BufferOverflowAttacks
DETECT, EXPLOIT, PREVENT
James C. Foster Vitaly Osipov Nish Bhalla Niels Heinen
Buffer Overflow Attacks |
FOREWORD
BY DAVE AITEL FOUNDER AND CEO IMMUNITY, INC.
PUBLISHED BY
Syngress Publishing, Inc.
800 Hingham Street Rockland, MA 02370
Contents
Foreword
Part 1 Expanding on Buffer Overflows
Chapter 1 Buffer Overflows: The Essentials
Introduction 3
The Challenge of Software Security 4
Microsoft Software Is Not Bug Free 6
The Increase in Buffer Overflows 8
Exploits vs. Buffer Overflows 10
Madonna Hacked! 10
Definitions 12
Hardware 12
Software 13
Security 18
Summary 20
Solutions Fast Track 20
Frequently Asked Questions 23
Chapter 2 Understanding Shellcode 25
Introduction 25
An Overview of Shellcode 26
The Tools 26
The Assembly Programming Language 27
Analysis 28
Analysis 29
Analysis 29
Windows vs. Unix Assembly 31
The Addressing Problem 32
Using the “call” and “jmp”Trick 32
Analysis 32
Analysis 33
Pushing the Arguments 33
The NULL Byte Problem 34
Implementing System Calls 35
System Call Numbers 35
System Call Arguments 36
Analysis 36
Analysis 37
Analysis 37
System Call Return Values 38
Remote Shellcode 38
Port-Binding Shellcode 38
Analysis 40
Socket Descriptor Reuse Shellcode 40
Analysis 40
Local Shellcode 41
execve Shellcode 41
setuid Shellcode 43
chroot Shellcode 44
Summary 49
Solutions Fast Track 49
Links to Sites 51
Mailing Lists 51
Frequently Asked Questions 52
Chapter 3 Writing Shellcode
Introduction 55
Shellcode Examples 56
The Write System Call 58
Analysis 60
Analysis 61
execve Shellcode 63
Analysis 63
Analysis 64
Analysis 66
Analysis 68
Analysis 70
Analysis 71
Port-Binding Shellcode 72
Analysis 73
Analysis 75
Analysis 76
Analysis 76
Analysis 77
Analysis 78
Analysis 81
Reverse Connection Shellcode 83
Analysis 85
Socket Reusing Shellcode 87
Analysis 88
Analysis 88
Reusing File Descriptors 89
Analysis 89
Analysis
Analysis
Analysis
Analysis
Analysis
Analysis
Encoding Shellcode 96
Analysis 97
Analysis 99
Analysis 101
Reusing Program Variables 102
Open Source Programs 102
Analysis 103
Closed Source Programs 104
Analysis 105
Analysis 105
OS-Spanning Shellcode 106
Analysis 107
Understanding Existing Shellcode 107
Analysis 109
Summary 112
Solutions Fast Track 112
Links to Sites 113
Mailing Lists 114
Frequently Asked Questions 114
Chapter 4 Win32 Assembly 117
Introduction 117
Application Memory Layout 118
Application Structure 120
Memory Allocation—Stack 121
Memory Allocation—Heap 122
Heap Structure 123
Windows Assembly 124
Registers 124
Indexing Registers 125
Stack Registers 125
Other General-Purpose Registers 125
EIP Register 126
Data Type 126
Operations 126
Hello World 127
Summary 129
Solutions Fast Track 130
Frequently Asked Questions 131
Section 1 Case Studies
Case Study 1.1 FreeBSD NN Exploit Code
Overview 133
Exploitation Code Dump 134
Analysis 136
References 137
Case Study 1.2 xlockmore User Supplied Format String
Vulnerability 138
Overview 138
xlockmore Vulnerability Details 139
Exploitation Code Dump 139
Analysis 141
References 141
Case Study 1.3 Frontpage Denial of Service Utilizing
WinSock 142
Overview 142
Code Dump 143
Analysis 144
Application Defense Hack.h Code Dump 145
Analysis 151
References 152
Case Study 1.4 cURL buffer overflow on FreeBSD
Overview 154
Exploit Code 155
Analysis 157
References 158
Part II Exploiting Buffer Overflows
Chapter 5 Stack Overflows
Introduction 161
Intel x86 Architecture and Machine Language Basics 163
Registers 164
Stacks and Procedure Calls 165
Storing Local Variables 167
Calling Conventions and Stack Frames 172
Introduction to the Stack Frame 172
Passing Arguments to a Function 173
Stack Frames and Calling Syntaxes 180
Process Memory Layout 181
Stack Overflows and Their Exploitation 183
Simple Overflow 185
Creating an Example Program with an Exploitable Overflow 189
Writing Overflowable Code 189
Disassembling the Overflowable Code 190
Performing the Exploit 192
General Exploit Concepts 192
Buffer Injection Techniques 193
Methods to Execute Payload 194
Designing Payload 198
What Is an Off-by-One Overflow? 204
Functions That Can Produce Buffer Overflows 211
Functions and Their Problems, or Never Use gets() 211
gets() and fgets() 211
strcpy() and strncpy(), strcat(), and strncat() 212
(v)sprintf() and (v)snprintf() 213
sscanf(), vscanf(), and fscanf() 213
Other Functions 214
Challenges in Finding Stack Overflows 215
Lexical Analysis 217
Semantics-Aware Analyzers 218
Application Defense! 220
OpenBSD 2.8 ftpd Off-by-One 220
Apache htpasswd Buffer Overflow 221
Summary 222
Solutions Fast Track 224
Links to Sites 225
Frequently Asked Questions 227
Chapter 6 Heap Corruption
Introduction 229
Simple Heap Corruption 230
Using the Heap—malloc(), calloc(), realloc() 231
Simple Heap and BSS Overflows 232
Corrupting Function Pointers in C++ 235
Advanced Heap Corruption—Doug Lea malloc 238
Overview of Doug Lea malloc 238
Memory Organization—Boundary Tags, Bins, Arenas 239
The free() Algorithm 244
Fake Chunks 246
Example Vulnerable Program 248
Exploiting frontlink() 250
Off-by-One and Off-by-Five on the Heap 251
Advanced Heap Corruption—System V malloc 252
System V malloc Operation 252
Tree Structure 253
Freeing Memory 255
The realfree() Function 257
The t_delete Function—The Exploitation Point 260
Application Defense! 263
Fixing Heap Corruption Vulnerabilities in the Source 263
Summary 266
Solutions Fast Track 267
Links to Sites 268
Frequently Asked Questions 270
Chapter 7 Format String Attacks
Introduction 273
What Is a Format String? 274
C Functions with Variable Numbers of Arguments 274
Ellipsis and va_args 275
Functions of Formatted Output 278
Using Format Strings 280
printf() Example 280
Format Tokens and printf() Arguments 281
Types of Format Specifiers 282
Abusing Format Strings 284
Playing with Bad Format Strings 286
Denial of Service 287
Direct Argument Access 287
Reading Memory 288
Writing to Memory 291
Simple Writes to Memory 291
Multiple Writes 294
Challenges in Exploiting Format String Bugs
Finding Format String Bugs 296
What to Overwrite 299
Destructors in .dtors 300
Global Offset Table entries 302
Structured Exception Handlers 304
Operating System Differences 305
Difficulties in Exploiting Different Systems 308
Application Defense! 308
The Whitebox and Blackbox Analysis of Applications 309
Summary 311
Solutions Fast Track 311
Links to Sites 313
Frequently Asked Questions 314
Chapter 8 Windows Buffer Overflows
Introduction 317
Background 318
Basic Stack Overflow 318
Analysis 322
Writing Windows Shellcode 327
Overcoming Special Characters (Example: NULL) 333
Client Server Application 338
Using/Abusing the Structured Exception Handler 350
Summary 355
Solutions Fast Track 355
Frequently Asked Questions 357
Section 2 Case Studies
Case Study 2.1 cURL Buffer Overflow on Linux
Overview 359
Exploit Code 360
Analysis 362
References 363
Case Study 2.2 SSLv2 Malformed Client Key Remote
Buffer Overflow Vuln. 364
Overview 364
OpenSSL Vulnerability Details 365
Exploitation Details 365
The Complication 366
Analysis 367
Improving the Exploit 368
Much Improved… but More to Come! 368
Complete Exploit Code for OpenSSL SSLv2
Malformed Client Key Remote Buffer Overflow 369
References 375
Case Study 2.3 X11R6 4.2 XLOCALEDIR Overflow
Overview 376
XLOCALEDIR Vulnerability Details and Analysis 377
Exploitation Code Dump 379
Analysis 381
References 381
Case Study 2.4 Microsoft MDAC Denial of Service
Overview 382
Code Dump 383
Analysis 385
Application Defense Hack.h Code Dump 386
Analysis 392
References 394
Case Study 2.5 Local UUX Buffer Overflow on HPUX
Overview 395
Exploit Code 396
Analysis 397
References 399
Part III Finding Buffer Overflows
Chapter 9 Finding Buffer Overflows in Source
Introduction 403
Source Code Analysis 404
Free Open Source Tools 406
Application Defense Snapshot 406
RATS 408
Flawfinder 412
Flawfinder Text Output 414
ITS4 417
Application Defense—Enterprise Developer 418
Secure Software 423
Architecture and Deployment423
Vulnerability Knowledgebase 424
Using CodeAssure 425
Setting Up Projects 427
Performing the Analysis 427
Vulnerability Review and Reporting 428
Managing Results 430
Remediation 432
Ounce Labs 432
Prexis’ Science of Automated Analysis 433
Prexis Architecture 434
Prexis Assessment Capability 434
Prexis Reporting and Remediation Capabilities 434
Prexis in Action 435
Vulnerability Assessment with Prexis 436
Project Configuration 436
Running an Assessment 436
Examining Assessment Results 437
Filtering Assessments 438
Remediation 438
Fortify Software 440
Fortify’s Source Code Analysis Suite 441
Using the Source Code Analysis Engine 441
Integrating with the Build Process 442
Running the Analysis 443
Understanding the Raw Output 443
Audit Workbench 444
Audit Guide 445
Software Security Manager 447
Summary 449
Solutions Fast Track 450
Links to Sites 452
Frequently Asked Question 453
Section 3 Case Studies
Case Study 3.1 InlineEgg I 455
Overview 455
Inline Egg Code 456
Analysis 456
References 457
Case Study 3.2 InlineEgg II
Overview 458
Inline Egg Code .459
Analysis 460
References 461
Case Study 3.3 Seti@Home Exploit Code 462
Overview 462
Exploitation Code Dump 462
Analysis 467
References 469
Case Study 3.4 Microsoft CodeBlue Exploit Code 470
Overview 470
Exploitation Code Dump 471
Analysis .474
References 475
Appendix A The Complete Data Conversion Table 477
Appendix B Useful Syscalls 485
exit( int ) 485
open( file, flags, mode ) 485
close( filedescriptor ) 485
read( filedescriptor, pointer to buffer, amount of bytes ) 486
write( filedescriptor, pointer to buffer, amount of bytes ) 486
execve( file, file + arguments, environment data ) 486
socketcall(callnumber, arguments ) 486
socket( domain, type, protocol ) .487
bind( file descriptor, sockaddr struct, size of arg 2 ) 487
listen ( file descriptor, number of connections allowed in queue ) 487
accept ( file descriptor, sockaddr struct, size of arg 2 ) 487
Index 489
-----------------------------------------------------------------------------------------------------
Part 1 Expanding on Buffer Overflows
Chapter 1 Buffer Overflows: The Essentials
Introduction 3
The Challenge of Software Security 4
Microsoft Software Is Not Bug Free 6
The Increase in Buffer Overflows 8
Exploits vs. Buffer Overflows 10
Madonna Hacked! 10
Definitions 12
Hardware 12
Software 13
Security 18
Summary 20
Solutions Fast Track 20
Frequently Asked Questions 23
Chapter 2 Understanding Shellcode 25
Introduction 25
An Overview of Shellcode 26
The Tools 26
The Assembly Programming Language 27
Analysis 28
Analysis 29
Analysis 29
Windows vs. Unix Assembly 31
The Addressing Problem 32
Using the “call” and “jmp”Trick 32
Analysis 32
Analysis 33
Pushing the Arguments 33
The NULL Byte Problem 34
Implementing System Calls 35
System Call Numbers 35
System Call Arguments 36
Analysis 36
Analysis 37
Analysis 37
System Call Return Values 38
Remote Shellcode 38
Port-Binding Shellcode 38
Analysis 40
Socket Descriptor Reuse Shellcode 40
Analysis 40
Local Shellcode 41
execve Shellcode 41
setuid Shellcode 43
chroot Shellcode 44
Summary 49
Solutions Fast Track 49
Links to Sites 51
Mailing Lists 51
Frequently Asked Questions 52
Chapter 3 Writing Shellcode
Introduction 55
Shellcode Examples 56
The Write System Call 58
Analysis 60
Analysis 61
execve Shellcode 63
Analysis 63
Analysis 64
Analysis 66
Analysis 68
Analysis 70
Analysis 71
Port-Binding Shellcode 72
Analysis 73
Analysis 75
Analysis 76
Analysis 76
Analysis 77
Analysis 78
Analysis 81
Reverse Connection Shellcode 83
Analysis 85
Socket Reusing Shellcode 87
Analysis 88
Analysis 88
Reusing File Descriptors 89
Analysis 89
Analysis
Analysis
Analysis
Analysis
Analysis
Analysis
Encoding Shellcode 96
Analysis 97
Analysis 99
Analysis 101
Reusing Program Variables 102
Open Source Programs 102
Analysis 103
Closed Source Programs 104
Analysis 105
Analysis 105
OS-Spanning Shellcode 106
Analysis 107
Understanding Existing Shellcode 107
Analysis 109
Summary 112
Solutions Fast Track 112
Links to Sites 113
Mailing Lists 114
Frequently Asked Questions 114
Chapter 4 Win32 Assembly 117
Introduction 117
Application Memory Layout 118
Application Structure 120
Memory Allocation—Stack 121
Memory Allocation—Heap 122
Heap Structure 123
Windows Assembly 124
Registers 124
Indexing Registers 125
Stack Registers 125
Other General-Purpose Registers 125
EIP Register 126
Data Type 126
Operations 126
Hello World 127
Summary 129
Solutions Fast Track 130
Frequently Asked Questions 131
Section 1 Case Studies
Case Study 1.1 FreeBSD NN Exploit Code
Overview 133
Exploitation Code Dump 134
Analysis 136
References 137
Case Study 1.2 xlockmore User Supplied Format String
Vulnerability 138
Overview 138
xlockmore Vulnerability Details 139
Exploitation Code Dump 139
Analysis 141
References 141
Case Study 1.3 Frontpage Denial of Service Utilizing
WinSock 142
Overview 142
Code Dump 143
Analysis 144
Application Defense Hack.h Code Dump 145
Analysis 151
References 152
Case Study 1.4 cURL buffer overflow on FreeBSD
Overview 154
Exploit Code 155
Analysis 157
References 158
Part II Exploiting Buffer Overflows
Chapter 5 Stack Overflows
Introduction 161
Intel x86 Architecture and Machine Language Basics 163
Registers 164
Stacks and Procedure Calls 165
Storing Local Variables 167
Calling Conventions and Stack Frames 172
Introduction to the Stack Frame 172
Passing Arguments to a Function 173
Stack Frames and Calling Syntaxes 180
Process Memory Layout 181
Stack Overflows and Their Exploitation 183
Simple Overflow 185
Creating an Example Program with an Exploitable Overflow 189
Writing Overflowable Code 189
Disassembling the Overflowable Code 190
Performing the Exploit 192
General Exploit Concepts 192
Buffer Injection Techniques 193
Methods to Execute Payload 194
Designing Payload 198
What Is an Off-by-One Overflow? 204
Functions That Can Produce Buffer Overflows 211
Functions and Their Problems, or Never Use gets() 211
gets() and fgets() 211
strcpy() and strncpy(), strcat(), and strncat() 212
(v)sprintf() and (v)snprintf() 213
sscanf(), vscanf(), and fscanf() 213
Other Functions 214
Challenges in Finding Stack Overflows 215
Lexical Analysis 217
Semantics-Aware Analyzers 218
Application Defense! 220
OpenBSD 2.8 ftpd Off-by-One 220
Apache htpasswd Buffer Overflow 221
Summary 222
Solutions Fast Track 224
Links to Sites 225
Frequently Asked Questions 227
Chapter 6 Heap Corruption
Introduction 229
Simple Heap Corruption 230
Using the Heap—malloc(), calloc(), realloc() 231
Simple Heap and BSS Overflows 232
Corrupting Function Pointers in C++ 235
Advanced Heap Corruption—Doug Lea malloc 238
Overview of Doug Lea malloc 238
Memory Organization—Boundary Tags, Bins, Arenas 239
The free() Algorithm 244
Fake Chunks 246
Example Vulnerable Program 248
Exploiting frontlink() 250
Off-by-One and Off-by-Five on the Heap 251
Advanced Heap Corruption—System V malloc 252
System V malloc Operation 252
Tree Structure 253
Freeing Memory 255
The realfree() Function 257
The t_delete Function—The Exploitation Point 260
Application Defense! 263
Fixing Heap Corruption Vulnerabilities in the Source 263
Summary 266
Solutions Fast Track 267
Links to Sites 268
Frequently Asked Questions 270
Chapter 7 Format String Attacks
Introduction 273
What Is a Format String? 274
C Functions with Variable Numbers of Arguments 274
Ellipsis and va_args 275
Functions of Formatted Output 278
Using Format Strings 280
printf() Example 280
Format Tokens and printf() Arguments 281
Types of Format Specifiers 282
Abusing Format Strings 284
Playing with Bad Format Strings 286
Denial of Service 287
Direct Argument Access 287
Reading Memory 288
Writing to Memory 291
Simple Writes to Memory 291
Multiple Writes 294
Challenges in Exploiting Format String Bugs
Finding Format String Bugs 296
What to Overwrite 299
Destructors in .dtors 300
Global Offset Table entries 302
Structured Exception Handlers 304
Operating System Differences 305
Difficulties in Exploiting Different Systems 308
Application Defense! 308
The Whitebox and Blackbox Analysis of Applications 309
Summary 311
Solutions Fast Track 311
Links to Sites 313
Frequently Asked Questions 314
Chapter 8 Windows Buffer Overflows
Introduction 317
Background 318
Basic Stack Overflow 318
Analysis 322
Writing Windows Shellcode 327
Overcoming Special Characters (Example: NULL) 333
Client Server Application 338
Using/Abusing the Structured Exception Handler 350
Summary 355
Solutions Fast Track 355
Frequently Asked Questions 357
Section 2 Case Studies
Case Study 2.1 cURL Buffer Overflow on Linux
Overview 359
Exploit Code 360
Analysis 362
References 363
Case Study 2.2 SSLv2 Malformed Client Key Remote
Buffer Overflow Vuln. 364
Overview 364
OpenSSL Vulnerability Details 365
Exploitation Details 365
The Complication 366
Analysis 367
Improving the Exploit 368
Much Improved… but More to Come! 368
Complete Exploit Code for OpenSSL SSLv2
Malformed Client Key Remote Buffer Overflow 369
References 375
Case Study 2.3 X11R6 4.2 XLOCALEDIR Overflow
Overview 376
XLOCALEDIR Vulnerability Details and Analysis 377
Exploitation Code Dump 379
Analysis 381
References 381
Case Study 2.4 Microsoft MDAC Denial of Service
Overview 382
Code Dump 383
Analysis 385
Application Defense Hack.h Code Dump 386
Analysis 392
References 394
Case Study 2.5 Local UUX Buffer Overflow on HPUX
Overview 395
Exploit Code 396
Analysis 397
References 399
Part III Finding Buffer Overflows
Chapter 9 Finding Buffer Overflows in Source
Introduction 403
Source Code Analysis 404
Free Open Source Tools 406
Application Defense Snapshot 406
RATS 408
Flawfinder 412
Flawfinder Text Output 414
ITS4 417
Application Defense—Enterprise Developer 418
Secure Software 423
Architecture and Deployment423
Vulnerability Knowledgebase 424
Using CodeAssure 425
Setting Up Projects 427
Performing the Analysis 427
Vulnerability Review and Reporting 428
Managing Results 430
Remediation 432
Ounce Labs 432
Prexis’ Science of Automated Analysis 433
Prexis Architecture 434
Prexis Assessment Capability 434
Prexis Reporting and Remediation Capabilities 434
Prexis in Action 435
Vulnerability Assessment with Prexis 436
Project Configuration 436
Running an Assessment 436
Examining Assessment Results 437
Filtering Assessments 438
Remediation 438
Fortify Software 440
Fortify’s Source Code Analysis Suite 441
Using the Source Code Analysis Engine 441
Integrating with the Build Process 442
Running the Analysis 443
Understanding the Raw Output 443
Audit Workbench 444
Audit Guide 445
Software Security Manager 447
Summary 449
Solutions Fast Track 450
Links to Sites 452
Frequently Asked Question 453
Section 3 Case Studies
Case Study 3.1 InlineEgg I 455
Overview 455
Inline Egg Code 456
Analysis 456
References 457
Case Study 3.2 InlineEgg II
Overview 458
Inline Egg Code .459
Analysis 460
References 461
Case Study 3.3 Seti@Home Exploit Code 462
Overview 462
Exploitation Code Dump 462
Analysis 467
References 469
Case Study 3.4 Microsoft CodeBlue Exploit Code 470
Overview 470
Exploitation Code Dump 471
Analysis .474
References 475
Appendix A The Complete Data Conversion Table 477
Appendix B Useful Syscalls 485
exit( int ) 485
open( file, flags, mode ) 485
close( filedescriptor ) 485
read( filedescriptor, pointer to buffer, amount of bytes ) 486
write( filedescriptor, pointer to buffer, amount of bytes ) 486
execve( file, file + arguments, environment data ) 486
socketcall(callnumber, arguments ) 486
socket( domain, type, protocol ) .487
bind( file descriptor, sockaddr struct, size of arg 2 ) 487
listen ( file descriptor, number of connections allowed in queue ) 487
accept ( file descriptor, sockaddr struct, size of arg 2 ) 487
Index 489
-----------------------------------------------------------------------------------------------------
■ Never be afraid. Microsoft has an entire army of marketing people telling you
how hard it is to find and write buffer overflows against their newest software.
My strategy is to fantasize about what I’m going to do with the exploit
once it’s working – it keeps me going.Writing exploits is going to require
mastery of a lot of difficult and boring technology. HP-UX uses wacky quadrant
memory access. Irix has inane caches. And learning the thousands of
assembly languages this requires is no small feat. If you think you can do it,
you can.
■ Don’t take yourself too seriously. Realize that no matter how good you get,
some fifteen year-old in Sweden is dedicating twenty hours a day to be better
than you. It’s not a competition, and you’ll burn out quick if you play the
game as such.
■ Find some friends. Like weight lifting or judo, this is not a skill you’re going
to keep up with if you do it all yourself. And you need people to tell you
where you’re weak.
------------------------------------------------
● Screenshot ●
how hard it is to find and write buffer overflows against their newest software.
My strategy is to fantasize about what I’m going to do with the exploit
once it’s working – it keeps me going.Writing exploits is going to require
mastery of a lot of difficult and boring technology. HP-UX uses wacky quadrant
memory access. Irix has inane caches. And learning the thousands of
assembly languages this requires is no small feat. If you think you can do it,
you can.
■ Don’t take yourself too seriously. Realize that no matter how good you get,
some fifteen year-old in Sweden is dedicating twenty hours a day to be better
than you. It’s not a competition, and you’ll burn out quick if you play the
game as such.
■ Find some friends. Like weight lifting or judo, this is not a skill you’re going
to keep up with if you do it all yourself. And you need people to tell you
where you’re weak.
------------------------------------------------
● Screenshot ●