Practical Programming for Total Beginners
by Al Sweigart
Part I: Python Programming Basics
Chapter 1: Python Basics
Chapter 2: Flow Control
Chapter 3: Functions
Chapter 4: Lists
Chapter 5: Dictionaries and Structuring Data
Chapter 6: Manipulating Strings
Part II: Automating Tasks
Chapter 7: Pattern Matching with Regular Expressions
Chapter 8: Reading and Writing Files
Chapter 9: Organizing Files
Chapter 10: Debugging
Chapter 11: Web Scraping
Chapter 12: Working with Excel Spreadsheets
Chapter 13: Working with PDF and Word Documents
Chapter 14: Working with CSV Files and JSON Data
Chapter 15: Keeping Time, Scheduling Tasks, and Launching Programs
Chapter 16: Sending Email and Text Messages
Chapter 17: Manipulating Images
Chapter 18: Controlling the Keyboard and Mouse with GUI Automation
Appendix A: Installing Third-Party Modules
Appendix B: Running Programs
Appendix C: Answers to the Practice Questions
Purchase Now !
Just with Paypal
Just with Paypal
Book Details
Price
|
2.00 USD |
|---|---|
Pages
| 505 p |
File Size
|
17,041 KB |
File Type
|
PDF format |
ISBN-10 ISBN-13
| 1-59327-599-4 978-1-59327-599-0 |
Copyright
| 2015 by Al Sweigart |
Al Sweigart is a software developer and tech book author living in San Francisco.
Python is his favorite programming language, and he is the
developer of several open source modules for it. His other books are freely
available under a Creative Commons license on his website
http://www.inventwithpython.com/. His cat weighs 14 pounds.
About the Tech Reviewer
Ari Lacenski is a developer of Android applications and Python software.
She lives in San Francisco, where she writes about Android programming at
http://gradlewhy.ghost.io/ and mentors with Women Who Code. She’s also a folk guitarist.
Table of Contents
Acknowledgments xxiii
Introduction 1
Whom Is This Book For? 2
Conventions . 2
What Is Programming? 3
What Is Python? . 4
Programmers Don’t Need to Know Much Math 4
Programming Is a Creative Activity . 5
About This Book 5
Downloading and Installing Python . 6
Starting IDLE . 7
The Interactive Shell 8
How to Find Help 8
Asking Smart Programming Questions . 9
Summary . 10
Part I: Python Programming Basics 11
1. Python Basics 13
Entering Expressions into the Interactive Shell 14
The Integer, Floating-Point, and String Data Types . 16
String Concatenation and Replication 17
Storing Values in Variables 18
Assignment Statements 18
Variable Names 20
Your First Program . 21
Dissecting Your Program 22
Comments . 23
The print() Function . 23
The input() Function 23
Printing the User’s Name . 24
The len() Function . 24
The str(), int(), and float() Functions . 25
Summary . 28
Practice Questions . 28
2. Flow Control 31
Boolean Values . 32
Comparison Operators 33
Boolean Operators 35
Binary Boolean Operators . 35
The not Operator . 36
Mixing Boolean and Comparison Operators 36
Elements of Flow Control 37
Conditions 37
Blocks of Code 37
Program Execution 38
Flow Control Statements . 38
if Statements 38
else Statements 39
elif Statements . 40
while Loop Statements 45
break Statements 49
continue Statements 50
for Loops and the range() Function . 53
Importing Modules 57
from import Statements . 58
Ending a Program Early with sys.exit() . 58
Summary . 58
Practice Questions . 59
3. Functions 61
def Statements with Parameters . 63
Return Values and return Statements 63
The None Value 65
Keyword Arguments and print() 65
Local and Global Scope 67
Local Variables Cannot Be Used in the Global Scope . 67
Local Scopes Cannot Use Variables in Other Local Scopes 68
Global Variables Can Be Read from a Local Scope 69
Local and Global Variables with the Same Name 69
The global Statement . 70
Exception Handling . 72
A Short Program: Guess the Number . 74
Summary . 76
Practice Questions . 76
Practice Projects 77
The Collatz Sequence . 77
Input Validation . 77
4. Lists 79
The List Data Type . 80
Getting Individual Values in a List with Indexes 80
Negative Indexes . 82
Getting Sublists with Slices 82
Getting a List’s Length with len() . 83
Changing Values in a List with Indexes . 83
List Concatenation and List Replication 83
Removing Values from Lists with del Statements 84
Working with Lists . 84
Using for Loops with Lists 86
The in and not in Operators 87
The Multiple Assignment Trick 87
Augmented Assignment Operators 88
Methods 89
Finding a Value in a List with the index() Method . 89
Adding Values to Lists with the append() and insert() Methods . 89
Removing Values from Lists with remove() . 90
Sorting the Values in a List with the sort() Method . 91
Example Program: Magic 8 Ball with a List . 92
List-like Types: Strings and Tuples 93
Mutable and Immutable Data Types . 94
The Tuple Data Type . 96
Converting Types with the list() and tuple() Functions . 97
References 97
Passing References . 100
The copy Module’s copy() and deepcopy() Functions 100
Summary . 101
Practice Questions . 102
Practice Projects 102
Comma Code 102
Character Picture Grid 103
5. Dictionaries and Structuring Data 105
The Dictionary Data Type . 105
Dictionaries vs. Lists 106
The keys(), values(), and items() Methods 107
Checking Whether a Key or Value Exists in a Dictionary . 109
The get() Method 109
The setdefault() Method . 110
Pretty Printing . 111
Using Data Structures to Model Real-World Things 112
A Tic-Tac-Toe Board 113
Nested Dictionaries and Lists . 117
Summary . 119
Practice Questions . 119
Practice Projects 120
Fantasy Game Inventory . 120
List to Dictionary Function for Fantasy Game Inventory 120
6. Manipulating Strings 123
Working with Strings . 123
String Literals . 124
Indexing and Slicing Strings 126
The in and not in Operators with Strings . 127
Useful String Methods 127
The upper(), lower(), isupper(), and islower() String Methods . 128
The isX String Methods . 129
The startswith() and endswith() String Methods . 131
The join() and split() String Methods . 131
Justifying Text with rjust(), ljust(), and center() . 133
Removing Whitespace with strip(), rstrip(), and lstrip() . 134
Copying and Pasting Strings with the pyperclip Module 135
Project: Password Locker 136
Step 1: Program Design and Data Structures . 136
Step 2: Handle Command Line Arguments 137
Step 3: Copy the Right Password . 137
Project: Adding Bullets to Wiki Markup 139
Step 1: Copy and Paste from the Clipboard 139
Step 2: Separate the Lines of Text and Add the Star . 140
Step 3: Join the Modified Lines 141
Summary . 141
Practice Questions . 142
Practice Project . 142
Table Printer 142
Part II: Automating Tasks 145
7. Pattern Matching with Regular Expressions 147
Finding Patterns of Text Without Regular Expressions . 148
Finding Patterns of Text with Regular Expressions 150
Creating Regex Objects . 150
Matching Regex Objects 151
Review of Regular Expression Matching 152
More Pattern Matching with Regular Expressions 152
Grouping with Parentheses 152
Matching Multiple Groups with the Pipe 153
Optional Matching with the Question Mark . 154
Matching Zero or More with the Star 155
Matching One or More with the Plus 155
Matching Specific Repetitions with Curly Brackets 156
Greedy and Nongreedy Matching 156
The findall() Method 157
Character Classes . 158
Making Your Own Character Classes 159
The Caret and Dollar Sign Characters . 159
The Wildcard Character 160
Matching Everything with Dot-Star 161
Matching Newlines with the Dot Character . 162
Review of Regex Symbols . 162
Case-Insensitive Matching 163
Substituting Strings with the sub() Method . 163
Managing Complex Regexes 164
Combining re.IGNORECASE, re.DOTALL, and re.VERBOSE . 164
Project: Phone Number and Email Address Extractor . 165
Step 1: Create a Regex for Phone Numbers 166
Step 2: Create a Regex for Email Addresses 166
Step 3: Find All Matches in the Clipboard Text 167
Step 4: Join the Matches into a String for the Clipboard 168
Running the Program 169
Ideas for Similar Programs 169
Summary . 169
Practice Questions . 170
Practice Projects 171
Strong Password Detection 171
Regex Version of strip() . 171
8. Reading and Writing Files 173
Files and File Paths 173
Backslash on Windows and Forward Slash on OS X and Linux 174
The Current Working Directory 175
Absolute vs. Relative Paths 175
Creating New Folders with os.makedirs() 176
The os.path Module . 177
Handling Absolute and Relative Paths . 177
Finding File Sizes and Folder Contents 179
Checking Path Validity 180
The File Reading/Writing Process . 180
Opening Files with the open() Function . 181
Reading the Contents of Files . 182
Writing to Files 183
Saving Variables with the shelve Module . 184
Saving Variables with the pprint.pformat() Function . 185
Project: Generating Random Quiz Files 186
Step 1: Store the Quiz Data in a Dictionary 187
Step 2: Create the Quiz File and Shuffle the Question Order . 188
Step 3: Create the Answer Options 189
Step 4: Write Content to the Quiz and Answer Key Files 189
Project: Multiclipboard . 191
Step 1: Comments and Shelf Setup . 192
Step 2: Save Clipboard Content with a Keyword . 192
Step 3: List Keywords and Load a Keyword’s Content . 193
Summary . 194
Practice Questions . 194
Practice Projects 194
Extending the Multiclipboard . 194
Mad Libs . 195
Regex Search 195
9. Organizing Files 197
The shutil Module 198
Copying Files and Folders . 198
Moving and Renaming Files and Folders . 199
Permanently Deleting Files and Folders . 200
Safe Deletes with the send2trash Module 201
Walking a Directory Tree . 202
Compressing Files with the zipfile Module 203
Reading ZIP Files . 204
Extracting from ZIP Files 205
Creating and Adding to ZIP Files . 205
Project: Renaming Files with American-Style Dates to
European-Style Dates 206
Step 1: Create a Regex for American-Style Dates . 206
Step 2: Identify the Date Parts from the Filenames 207
Step 3: Form the New Filename and Rename the Files 209
Ideas for Similar Programs 209
Project: Backing Up a Folder into a ZIP File 209
Step 1: Figure Out the ZIP File’s Name . 210
Step 2: Create the New ZIP File . 211
Step 3: Walk the Directory Tree and Add to the ZIP File . 211
Ideas for Similar Programs 212
Summary . 212
Practice Questions . 213
Practice Projects 213
Selective Copy . 213
Deleting Unneeded Files . 213
Filling in the Gaps 214
10. Debugging 215
Raising Exceptions 216
Getting the Traceback as a String . 217
Assertions 219
Using an Assertion in a Traffic Light Simulation 219
Disabling Assertions . 221
Logging . 221
Using the logging Module . 221
Don’t Debug with print() 223
Logging Levels . 223
Disabling Logging 224
Logging to a File 225
IDLE’s Debugger . 225
Go . 226
Step . 226
Over 226
Out 227
Quit . 227
Debugging a Number Adding Program 227
Breakpoints . 229
Summary . 231
Practice Questions . 231
Practice Project . 232
Debugging Coin Toss . 232
11. Web Scraping 233
Project: mapIt.py with the webbrowser Module . 234
Step 1: Figure Out the URL 234
Step 2: Handle the Command Line Arguments . 235
Step 3: Handle the Clipboard Content and Launch the Browser . 236
Ideas for Similar Programs 236
Downloading Files from the Web with the requests Module 237
Downloading a Web Page with the requests.get() Function . 237
Checking for Errors 238
Saving Downloaded Files to the Hard Drive . 239
HTML . 240
Resources for Learning HTML 240
A Quick Refresher 240
Viewing the Source HTML of a Web Page . 241
Opening Your Browser’s Developer Tools 242
Using the Developer Tools to Find HTML Elements 244
Parsing HTML with the BeautifulSoup Module . 245
Creating a BeautifulSoup Object from HTML 245
Finding an Element with the select() Method 246
Getting Data from an Element’s Attributes . 248
Project: “I’m Feeling Lucky” Google Search 248
Step 1: Get the Command Line Arguments and Request the Search Page 249
Step 2: Find All the Results 249
Step 3: Open Web Browsers for Each Result . 250
Ideas for Similar Programs 251
Project: Downloading All XKCD Comics . 251
Step 1: Design the Program . 252
Step 2: Download the Web Page 253
Step 3: Find and Download the Comic Image . 254
Step 4: Save the Image and Find the Previous Comic . 255
Ideas for Similar Programs 256
Controlling the Browser with the selenium Module . 256
Starting a Selenium-Controlled Browser . 256
Finding Elements on the Page 257
Clicking the Page . 259
Filling Out and Submitting Forms 259
Sending Special Keys . 260
Clicking Browser Buttons . 261
More Information on Selenium . 261
Summary . 261
Practice Questions . 261
Practice Projects 262
Command Line Emailer . 262
Image Site Downloader 263
2048 263
Link Verification . 263
12. Working with Excel Spreadsheets 265
Excel Documents . 266
Installing the openpyxl Module . 266
Reading Excel Documents 266
Opening Excel Documents with OpenPyXL 267
Getting Sheets from the Workbook . 268
Getting Cells from the Sheets . 268
Converting Between Column Letters and Numbers 270
Getting Rows and Columns from the Sheets . 270
Workbooks, Sheets, Cells . 272
Project: Reading Data from a Spreadsheet . 272
Step 1: Read the Spreadsheet Data 273
Step 2: Populate the Data Structure 274
Step 3: Write the Results to a File . 275
Ideas for Similar Programs 276
Writing Excel Documents . 277
Creating and Saving Excel Documents 277
Creating and Removing Sheets 278
Writing Values to Cells . 278
Project: Updating a Spreadsheet . 279
Step 1: Set Up a Data Structure with the Update Information . 280
Step 2: Check All Rows and Update Incorrect Prices . 281
Ideas for Similar Programs 281
Setting the Font Style of Cells 282
Font Objects . 282
Formulas 284
Adjusting Rows and Columns . 285
Setting Row Height and Column Width . 285
Merging and Unmerging Cells . 286
Freeze Panes . 287
Charts . 288
Summary . 290
Practice Questions . 291
Practice Projects 291
Multiplication Table Maker 291
Blank Row Inserter 292
Spreadsheet Cell Inverter 292
Text Files to Spreadsheet . 293
Spreadsheet to Text Files . 293
13. Working with PDF and Word Documents 295
PDF Documents . 295
Extracting Text from PDFs 296
Decrypting PDFs . 297
Creating PDFs 298
Project: Combining Select Pages from Many PDFs . 303
Step 1: Find All PDF Files 304
Step 2: Open Each PDF 304
Step 3: Add Each Page 305
Step 4: Save the Results 305
Ideas for Similar Programs 306
Word Documents 306
Reading Word Documents 307
Getting the Full Text from a .docx File . 308
Styling Paragraph and Run Objects 309
Creating Word Documents with Nondefault Styles . 310
Run Attributes 311
Writing Word Documents . 312
Adding Headings . 314
Adding Line and Page Breaks 315
Adding Pictures 315
Summary . 316
Practice Questions . 316
Practice Projects 317
PDF Paranoia 317
Custom Invitations as Word Documents 317
Brute-Force PDF Password Breaker 318
14. Working with CSV Files and JSON Data 319
The csv Module . 320
Reader Objects 321
Reading Data from Reader Objects in a for Loop . 322
Writer Objects . 322
The delimiter and lineterminator Keyword Arguments 323
Project: Removing the Header from CSV Files . 324
Step 1: Loop Through Each CSV File 325
Step 2: Read in the CSV File . 325
Step 3: Write Out the CSV File Without the First Row . 326
Ideas for Similar Programs 327
JSON and APIs . 327
The json Module . 328
Reading JSON with the loads() Function . 328
Writing JSON with the dumps() Function . 329
Project: Fetching Current Weather Data . 329
Step 1: Get Location from the Command Line Argument 330
Step 2: Download the JSON Data 330
Step 3: Load JSON Data and Print Weather 331
Ideas for Similar Programs 332
Summary . 333
Practice Questions . 333
Practice Project . 333
Excel-to-CSV Converter . 333
15. Keeping Time, Scheduling Tas ks,
and Launching Programs 335
The time Module . 336
The time.time() Function 336
The time.sleep() Function . 337
Rounding Numbers . 338
Project: Super Stopwatch . 338
Step 1: Set Up the Program to Track Times 339
Step 2: Track and Print Lap Times 339
Ideas for Similar Programs 340
The datetime Module . 341
The timedelta Data Type . 342
Pausing Until a Specific Date . 344
Converting datetime Objects into Strings . 344
Converting Strings into datetime Objects . 345
Review of Python’s Time Functions . 346
Multithreading 347
Passing Arguments to the Thread’s Target Function . 348
Concurrency Issues . 349
Project: Multithreaded XKCD Downloader 350
Step 1: Modify the Program to Use a Function . 350
Step 2: Create and Start Threads . 351
Step 3: Wait for All Threads to End 352
Launching Other Programs from Python 352
Passing Command Line Arguments to Popen() 354
Task Scheduler, launchd, and cron . 354
Opening Websites with Python 355
Running Other Python Scripts . 355
Opening Files with Default Applications 355
Project: Simple Countdown Program 357
Step 1: Count Down . 357
Step 2: Play the Sound File 357
Ideas for Similar Programs 358
Summary . 358
Practice Questions . 359
Practice Projects 359
Prettified Stopwatch 360
Scheduled Web Comic Downloader 360
16. Sending Email and Text Messages 361
SMTP . 362
Sending Email 362
Connecting to an SMTP Server . 363
Sending the SMTP “Hello” Message . 364
Starting TLS Encryption . 364
Logging in to the SMTP Server . 364
Sending an Email . 365
Disconnecting from the SMTP Server . 366
IMAP . 366
Retrieving and Deleting Emails with IMAP 366
Connecting to an IMAP Server . 367
Logging in to the IMAP Server . 368
Searching for Email 368
Fetching an Email and Marking It As Read 372
Getting Email Addresses from a Raw Message 373
Getting the Body from a Raw Message . 374
Deleting Emails 375
Disconnecting from the IMAP Server . 375
Project: Sending Member Dues Reminder Emails . 376
Step 1: Open the Excel File . 376
Step 2: Find All Unpaid Members 378
Step 3: Send Customized Email Reminders 378
Sending Text Messages with Twilio . 380
Signing Up for a Twilio Account 380
Sending Text Messages 381
Project: “Just Text Me” Module 383
Summary . 384
Practice Questions . 384
Practice Projects 385
Random Chore Assignment Emailer 385
Umbrella Reminder . 385
Auto Unsubscriber 385
Controlling Your Computer Through Email . 386
17. Manipulating Images 387
Computer Image Fundamentals . 388
Colors and RGBA Values 388
Coordinates and Box Tuples 389
Manipulating Images with Pillow . 390
Working with the Image Data Type 392
Cropping Images . 393
Copying and Pasting Images onto Other Images 394
Resizing an Image 397
Rotating and Flipping Images 398
Changing Individual Pixels 400
Project: Adding a Logo 401
Step 1: Open the Logo Image 401
Step 2: Loop Over All Files and Open Images . 402
Step 3: Resize the Images . 403
Step 4: Add the Logo and Save the Changes 404
Ideas for Similar Programs 406
Drawing on Images . 406
Drawing Shapes 406
Drawing Text . 408
Summary . 410
Practice Questions . 410
Practice Projects 411
Extending and Fixing the Chapter Project Programs 411
Identifying Photo Folders on the Hard Drive . 411
Custom Seating Cards 412
18. Controlling the Keyboard and Mouse
with GUI Automation 413
Installing the pyautogui Module 414
Staying on Track . 414
Shutting Down Everything by Logging Out . 414
Pauses and Fail-Safes . 415
Controlling Mouse Movement . 415
Moving the Mouse . 416
Getting the Mouse Position 417
Project: “Where Is the Mouse Right Now?” 417
Step 1: Import the Module . 418
Step 2: Set Up the Quit Code and Infinite Loop 418
Step 3: Get and Print the Mouse Coordinates 418
Controlling Mouse Interaction . 419
Clicking the Mouse . 420
Dragging the Mouse . 420
Scrolling the Mouse 422
Working with the Screen 423
Getting a Screenshot 423
Analyzing the Screenshot 424
Project: Extending the mouseNow Program 424
Image Recognition 425
Controlling the Keyboard . 426
Sending a String from the Keyboard . 426
Key Names . 427
Pressing and Releasing the Keyboard . 428
Hotkey Combinations . 429
Review of the PyAutoGUI Functions . 430
Project: Automatic Form Filler . 430
Step 1: Figure Out the Steps 432
Step 2: Set Up Coordinates . 432
Step 3: Start Typing Data 434
Step 4: Handle Select Lists and Radio Buttons 435
Step 5: Submit the Form and Wait . 436
Summary . 437
Practice Questions . 438
Practice Projects 438
Looking Busy . 438
Instant Messenger Bot . 438
Game-Playing Bot Tutorial . 439
A Installing Third-Party Modules 441
The pip Tool . 441
Installing Third-Party Modules . 442
B Running Programs 443
Shebang Line 443
Running Python Programs on Windows 444
Running Python Programs on OS X and Linux . 445
Running Python Programs with Assertions Disabled . 445
C Answers to the Practice Questions 447
Chapter 1 448
Chapter 2 448
Chapter 3 450
Chapter 4 450
Chapter 5 451
Chapter 6 451
Chapter 7 452
Chapter 8 453
Chapter 9 453
Chapter 10 454
Chapter 11 455
Chapter 12 456
Chapter 13 456
Chapter 14 457
Chapter 15 457
Chapter 16 458
Chapter 17 458
Chapter 18 458
Index 461
Bookscreen
Introduction
“You’ve just done in two hours what it takes
the three of us two days to do.” My college
roommate was working at a retail electronics
store in the early 2000s. Occasionally, the store
would receive a spreadsheet of thousands of product
prices from its competitor. A team of three employees
would print the spreadsheet onto a thick stack of paper and split it among
themselves. For each product price, they would look up their store’s price
and note all the products that their competitors sold for less. It usually took
a couple of days.
“You know, I could write a program to do that if you have the original
file for the printouts,” my roommate told them, when he saw them sitting
on the floor with papers scattered and stacked around them.
After a couple of hours, he had a short program that read a competitor’s
price from a file, found the product in the store’s database, and noted
whether the competitor was cheaper. He was still new to programming, and
he spent most of his time looking up documentation in a programming
book. The actual program took only a few seconds to run. My roommate
and his co-workers took an extra-long lunch that day.
This is the power of computer programming. A computer is like a Swiss
Army knife that you can configure for countless tasks. Many people spend
hours clicking and typing to perform repetitive tasks, unaware that the
machine they’re using could do their job in seconds if they gave it the right instructions.
Whom Is This Book For?
Software is at the core of so many of the tools we use today: Nearly everyone
uses social networks to communicate, many people have Internet-connected
computers in their phones, and most office jobs involve interacting with a
computer to get work done. As a result, the demand for people who can code
has skyrocketed. Countless books, interactive web tutorials, and developer
boot camps promise to turn ambitious beginners into software engineers
with six-figure salaries.
This book is not for those people. It’s for everyone else.
On its own, this book won’t turn you into a professional software developer
any more than a few guitar lessons will turn you into a rock star. But if
you’re an office worker, administrator, academic, or anyone else who uses a
computer for work or fun, you will learn the basics of programming so that
you can automate simple tasks such as the following:
• Moving and renaming thousands of files and sorting them into folders
• Filling out online forms, no typing required
• Downloading files or copy text from a website whenever it updates
• Having your computer text you custom notifications
• Updating or formatting Excel spreadsheets
• Checking your email and sending out prewritten responses
These tasks are simple but time-consuming for humans, and they’re
often so trivial or specific that there’s no ready-made software to perform
them. Armed with a little bit of programming knowledge, you can have
your computer do these tasks for you.