Advanced Windows Debugging. Addison Wesley

Mario Hewardt & Daniel Pravat

A D V A N C E D W I N D O W S D E B U G G I N G

PRAISE

“Who says you can’t bottle experience? Between the covers is a wealth of information thatclearly demonstrates how to take a logical approach to finding and eliminating bugs. Thisis an absolute must-have book for anyone who develops, tests, or supports software forMicrosoft Windows.”—Bob Wilton, Escalation Engineer, Critical Problem Resolution Team,  Microsoft Corporation
“I have been fortunate enough to personally work with the authors on extremely demanding
systems projects for more than eight years. This volume contains the kind of stuff we
all wish we had known back at the beginning of those projects—the kind of stuff that the
debugging guru tells you over a coffee-spilled keyboard on February 29 only because an
extra day showed up and he has the afternoon free; the kind of stuff that only comes
from actually building and then debugging complex systems projects instead of just reading
about somebody else doing it.
“Most books leave the advanced cases as ‘exercises to the reader’ or to ‘other, more
advanced books,’ and those never seem to materialize. This book is one of those very rare
‘other’ books. Get two copies. You will always be lending the other one out.”
—Raymond McCollum, Architect, Microsoft Forefront Security Products
“This book by Microsoft authors Mario and Daniel is an excellent reference for both
intermediate and advanced debuggers. In-depth examples showing how to debug intricate
problems, such as stack and heap corruptions, make this book stand out among current
available literature on debugging Win32 software on Windows. The book is highly
practical and is filled with numerous debugging tricks and strategies.”
—Kinshuman, Development Lead, Windows Core OS Division
“I am pleased to see this guided tour through a comprehensive set of clever debugging
techniques. It does not only tell how to deal with tough diagnosis problems, but it also
explains the mechanisms behind the techniques used. The pragmatic approach taken in
Advanced Windows Debugging makes it a good resource to understand several key
Windows areas.”

—Adrian Marinescu, Software Architect, Microsoft Corporation
“Advanced Windows Debugging fills the need for good documentation about debugging
and fixing software defects. The book is based on the authors’ valuable experience of
tracking down the cause of various classes of software bugs. It includes representative
examples of typical defects, the tools used to investigate these defects, and step-by-step
instructions for using these tools. Software developers and testers will greatly benefit
from becoming familiar with these examples.”
—Daniel Mihai, Software Design Engineer, Developer Productivity Tools, Microsoft
“I wrote the WinDbg symbol handler, Symbol Server, and Source Server. Even so, I can’t
get my own wife to use WinDbg. She thinks it is hard to use, and, consequently, she
hasn’t learned of the potential of this toolset. I am buying a copy of this book, so she can
learn it. The chapters on postmortem debugging and memory corruption are essential
reading that provide real insight into the internals of the runtime and OS in the context
of a program fault. Mario and Daniel’s understanding of debugging comes from being
asked to resolve completely unexplained bugs in unfamiliar target programs. This is what
industrial strength debugging is all about.”

—Pat Styles, Microsoft

e-books shop
Advanced Windows Debugging


ABOUT THE AUTHORS
Mario Hewardt is a senior design engineer with Microsoft
Corporation and has worked extensively in the Windows system
level development arena for the past nine years. Throughout five
releases of Windows (starting with Windows 98), he has worked
primarily in the server and desktop management arena, focusing
the majority of his time on ensuring the reliability, robustness, and
security of the product.

Daniel Pravat is a senior design engineer with Microsoft
Corporation and was actively involved in releasing several windows
components in multiple Windows releases. Prior to joining
Microsoft, he developed telecommunication software for computerbased
telephony servers. He expects all software applications to be

reliable, predictable, and efficient.

ACKNOWLEDGMENTS
Writing a technical book is a large-scale effort, far more substantial than we had originally
anticipated. As authors, we provided the raw material and the first draft of the
book, but throughout the project, a number of people shared their insights and
expertise to make this book worth the time spent reading it.
Thanks to all the team members at Addison Wesley, especially Elizabeth Peterson,
Jana Jones, Curt Johnson, Joan Murray, and Gina Kanouse. Chris Zahn also played an
instrumental role in editing the book and in correcting our self-styled syntax.
As with any technical publication, technical accuracy is of utmost importance. We
were fortunate to have great engineers (many of them own the specific technology
areas discussed in the book) look at the material and provide feedback. Thanks go to
Mark Russinovich, Ivan Brugiolo, Pat Styles, Pavel Lebedynskiy, Daniel Mihai, Doug
Ellis, Cristi Vlasceanu, Adrian Marinescu, Saji Abraham, Kamen Moutafov,
Kinshuman Kinshumann, Bob Wilton, Raymond McCollum, Viorel Mititean, Andy
Cheung, Saar Picker, Drew Bliss, Jason Cunningham, Adam Edwards, Jen-Lung
Chiu, Alain Lissoir, and Brandon Jiang.
Special thanks go to Mark Russinovich for not only reviewing the book but also
writing the foreword. Mark’s remarkable body of work is well known among software
developers and has been a great influence on us and countless other engineers.
Ivan Brugiolo was also instrumental in reviewing and providing in-depth feedback.
Ivan was incredibly generous with his spare time, sharing knowledge that has
added considerable value to this book.
We also want to extend our gratitude to Alexandra Hewardt for designing and
implementing the book’s Web page.


PREFACE
Not long ago, we were reminiscing about a really tough problem we faced at work.
The Quality Assurance team was running stress tests on our product, and every four
or five days, a crash would rear its ugly head. Sure, we had debugged the crash as far
as we thought possible, and we had done extensive code reviews to try to figure it out,
but alas, not enough information could be gained to get to the bottom of it. After several
weeks of unfruitful attempts, we started looking for alternative approaches.
During a random hallway conversation, someone happened to casually mention a tool
called gflags. Having never heard of this tool before, we set out to do some research
to find out how it could help us get to the bottom of our crash. Unfortunately, the
learning process proved to be somewhat difficult. First, finding information about the
tool proved to be a real challenge. There was a ton of great information in the reference
documentation that came with the tools, but it was hard to figure out how to
actually get started. We quickly realized that without some basic guidance, there was
little hope for us to be able to utilize the tool. Naturally, we decided to ask the person
who had happened to mention the tool if he knew of any documentation or pointers.
He gave us some brief descriptions of the tool and, perhaps more importantly, the
names of other people who had worked with the tool extensively. What followed was
a series of long and instructive conversations, and bit by bit the basic idea behind the
tool started falling into place.
Did we ever get to the bottom of the crash? Yes—we did. As a matter of fact,
enabling the correct tool while running our stress tests pinpointed the problem to
such accuracy that it only took an hour of code reviewing to locate and fix the misbehaving
code. Had we known about this tool and how to use it from the start, we would
have saved several weeks of work. From that point on, we dedicated quite a lot of
time to furthering our understanding of the tools and how they can help while trying
to troubleshoot misbehaving code.
Over the years, the Windows debuggers and tools have matured and grown and
become increasingly powerful. The amount of timesaving features now available is
truly mind-boggling. What is equally mind-boggling is that after several years, the
native debuggers and tools are still relatively unknown to developers. The few developers
who do find out that these tools exist have to go through a similarly painful learning
process as we did years ago. We were fortunate to have the luxury of working with
engineers at Microsoft (some of whom wrote the tools), but without this luxury, many
hopeful developers end up at a dead end and are never able to reap the benefits of the
tools. This unfortunate problem of a lack of learning material also turned out to be a
great opportunity for a solution, and thus the idea for this book was born. The key to
enable developers to gain the knowledge required is to provide a central repository of
concise information that fully explains the ins and outs of the debugging tools and
processes. The book you are holding serves as that key and is the net result of three
years of writing and over 15 years of collective debugging experience.
We hope that you will enjoy reading this book as much as we enjoyed authoring
it and that it will open up the door to a truly amazing world of highly efficient software
troubleshooting and debugging. Knowing how to use the tools and techniques
described in this book is a critical part of a computer scientist’s work and can teach
you how to very efficiently troubleshoot some of the toughest problems in software.

Who Is This Book For?
The short answer to this question is anyone who is involved in any facet of software
development and has a strong desire to learn what is actually happening deep inside
Windows. Although the technical nature of the book might make you believe that its
content is only intended for advanced system engineers, this is absolutely not true. One
of the key points of this book is the removing of the magic. For various reasons, a lot of
software engineers believe that there is a magical relationship between the software
they are working on and the operating system. When a problem surfaces that requires
the analysis of operating system components (such as RPC/COM or the Windows heap
manager), this preconceived notion of magic prevents them from venturing inside
Windows to gain more information that can potentially help them solve the problem.
To make effective use of this book, you will have to learn how to remove this preconceived
notion and truly be of the mind-set that there is no magic behind-the-scenes.
The core Windows components should be viewed as an extension of your product and
not as a separate and magical layer. After all, it’s all just code—some of which just happened
to be written by other people. If you can adjust your mind-set to accept this, you
will have taken your first steps to mastering the art of Windows debugging.

Software Developers
Anyone from a low-level system developer to a high-level RAD developer will benefit
from reading this book. Whether your preference is writing Windows-based software
in assembly language or by using the .NET framework, there is a ton of useful information
to be learned about the tools and techniques behind Windows debugging.
Over the years, we’ve had several discussions with higher-level RAD developers who
claim that they really don’t see the need to learn about these low-level topics. After all,
the beauty of writing code at a higher level is that all of the low-level intricacies are
abstracted and hidden away from the developer. We couldn’t agree more. However,
our claim is that although abstractive programming allows the developer not to have
to focus on low-level details, it does not negate the need to know how the abstraction
really works. The substance behind this claim is simple. What you are working with is
really just that—an abstraction. Usage of this abstraction in a design that it was not
suited for can cause serious problems in your software; and, in such a case, without a
solid understanding of how the abstraction works, it can mean the difference between
shipping your product on time and slipping the release date by several months.
Another key factor when considering mastering the Windows debuggers and tools
is related to the debugging of live production servers. While every attempt should be
made to fix bugs before shipping a product, we all know that some bugs might slip
through the cracks. When these bugs do surface post release, it can be a real
headache tracking them down. Customers who encounter the bugs on live production
servers are typically very sensitive to downtime and configuration changes, making
it impossible to install a complex debugger package. The Debugging Tools for
Windows, on the other hand, enables live debugging with no server configuration
change and no installation requirements. In short, it enables customers to keep a pristine
server during the troubleshooting process.

Quality Assurance Engineers
Just as software developers will find the information in this book useful in their day-today
tasks, so will quality assurance engineers. Quality assurance typically runs a battery
of tests on any given component being tested. During this time, any number of bugs
can surface. Whether they are memory corruptions, resource leaks, or hangs, knowing
what extended instrumentation to enable during the test run can dramatically reduce
the time it takes for root cause analysis. For instance, imagine that quality assurance is
tasked with stress testing a credit card authorization service. One of the goals is that
the service must be capable of surviving one week of continuous and simultaneous
hammering by client requests. On day six, the service starts reporting errors for all
client requests. At this point, the developers responsible for the service are called in to
analyze the problem. It doesn’t take long for them to figure out that the server has run
out of memory, presumably due to a small memory leak that accumulates over time.
After six days of accumulated leaks, figuring out the source of the leak, however, is a
much bigger challenge that can take days of debugging and code reviewing. Had the
correct extended instrumentation been enabled while running these tests, the time it
would have taken to analyze the leak could have been greatly reduced.

Product Support Engineers
In much the same way that quality assurance uses the Windows debuggers and tools
to make root cause analysis more efficient, so can the product support engineers.
Product support faces many of the same problems that quality assurance and software
developers face on a day by day basis. The key difference, however, is the environmental
constraints that they work under. The constraints can include not having full
access to the server exhibiting the problems, having a limited amount of time available
for troubleshooting the server, having limited access to customer source code,
and other issues.
The information presented in this book will give product support engineers a
great deal of ammunition when tackling these tough problems. Knowing how to
debug customer problems with minimal downtime and minimal system configuration
changes enables product support engineers to much more efficiently and nonintrusively
gather the required data to get to the bottom of the problem.


Table of Contents
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xiii
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xv
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxvii
About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxviii
PART I: OVERVIEW
Chapter 1: Introduction to the Tools . . . . . . . . . . . . . . . . . . . . . . . . . .3
Leak Diagnosis Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
Debugging Tools for Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
UMDH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Microsoft Application Verifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Global Flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Process Explorer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Windows Driver Kits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Ethereal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
DebugDiag . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Chapter 2 Introduction to the Debuggers . . . . . . . . . . . . . . . . . . . .29
Debugger Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Basic Debugger Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
Remote Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .109
Debugging Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .121
Chapter 3 Debuggers Uncovered . . . . . . . . . . . . . . . . . . . . . . . . .123
User Mode Debugger Internals . . . . . . . . . . . . . . . . . . . . . . . . . . . .124
Controlling the Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .168
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .178
Chapter 4: Managing Symbol and Source Files . . . . . . . . . . . . . . .179
Managing the Symbols for Debugging . . . . . . . . . . . . . . . . . . . . . . .180
Managing Source Files for Debugging . . . . . . . . . . . . . . . . . . . . . . .188
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .196
PART II: APPLIED DEBUGGING
Chapter 5: Memory Corruption Part I—Stacks . . . . . . . . . . . . . . . .199
Memory Corruption Detection Process . . . . . . . . . . . . . . . . . . . . . . .201
Stack Corruptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .209
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .258
Chapter 6: Memory Corruption Part II—Heaps . . . . . . . . . . . . . . . .259
What Is a Heap? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .259
Heap Corruptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .281
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .314
Chapter 7: Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .317
Windows Security Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .318
Source of Security Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . .328
How Is the Security Check Performed? . . . . . . . . . . . . . . . . . . . . . .334
Identity Propagation in Client-Server Applications . . . . . . . . . . . . . . .334
Security Checks at System Boundaries . . . . . . . . . . . . . . . . . . . . . . .338
Investigating Security Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . .340
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .378
Chapter 8: Interprocess Communication . . . . . . . . . . . . . . . . . . . .379
Communication Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .380
Troubleshooting Local Communication . . . . . . . . . . . . . . . . . . . . . . .382
Troubleshooting Remote Communication . . . . . . . . . . . . . . . . . . . . . .396
Additional Technical Information . . . . . . . . . . . . . . . . . . . . . . . . . . .422
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .426
Chapter 9: Resource Leaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .427
What Is a Resource? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .427
High-Level Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .428
Reproducibility of Resource Leaks . . . . . . . . . . . . . . . . . . . . . . . . . .433
Handle Leaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .434
Memory Leaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .460
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .492
Chapter 10: Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .493
Synchronization Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .493
High-Level Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .505
Synchronization Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .510
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .550
PART III: ADVANCED TOPICS
Chapter 11: Writing Custom Debugger Extensions . . . . . . . . . . . . . .553
Introduction to Debugger Extensions . . . . . . . . . . . . . . . . . . . . . . . .553
Example Debugger Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . .556
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .594
Chapter 12: 64-Bit Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . .595
Microsoft 64-Bit Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .595
Windows x64 Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .602
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .629
Chapter 13: Postmortem Debugging . . . . . . . . . . . . . . . . . . . . . . . .631
Dump File Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .632
Using Dump Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .645
Windows Error Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .653
Corporate Error Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .682
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .690
Chapter 14: Power Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .691
Debug Diagnostic Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .691
!analyze Extension Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . .699
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .708
Chapter 15: Windows Vista Fundamentals . . . . . . . . . . . . . . . . . . .709
Chapter 1—Introduction to the Tools . . . . . . . . . . . . . . . . . . . . . . . .710
Chapter 2—Introduction to the Debuggers . . . . . . . . . . . . . . . . . . . .711
Chapter 6—Memory Corruptions—Part Heaps . . . . . . . . . . . . . . . . .717
Chapter 7—Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .723
Chapter 8—Interprocess Communication . . . . . . . . . . . . . . . . . . . . .736
Chapter 9—Resource Leaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .736
Chapter 10—Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . .737
Chapter 11—Writing Custom Debugger Extensions . . . . . . . . . . . . . .741
Chapter 13—Postmortem Debugging . . . . . . . . . . . . . . . . . . . . . . . .741
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .745
Appendix A: Application Verifier Test Settings . . . . . . . . . . . . . . . . .747
Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .747
Handles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .747
Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .749
Locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .757
Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .760
ThreadPool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .762
TLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .764
FilePaths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .764
HighVersionLie . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .765
InteractiveServices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .767
KernelModeDriverInstall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .768
Low Resource Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .769
LuaPriv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .771
DangerousAPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .774
DirtyStacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .775
TimeRollOver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .775
PrintAPI and PrintDriver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .776
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .777


Screenshot

e-books shop

Purchase Now !
Just with Paypal



Product details
 Price
 Pages
 839 p
 File Size
 9,681 KB
 File Type
 PDF format
 ISBN-13
 ISBN-10
 978-0-321-37446-2
 0-321-37446-0
 Copyright
 2008 Pearson Education, Inc 
  ●▬▬▬▬▬❂❂❂▬▬▬▬▬●
●▬▬❂❂▬▬●
●▬❂▬●

═════ ═════

Previous Post Next Post