Discover the power of functional programming, generator functions, lazy evaluation, the built-in itertools library, and monads
Steven F. Lott
Purchase Now !
Just with Paypal
Just with Paypal
Book Details
Price
|
3.00 USD |
|---|---|
Pages
| 398 p |
File Size
|
2,582 KB |
File Type
|
PDF format |
ISBN
| 978-1-78862-706-1 |
Copyright
| 2018 Packt Publishing |
Steven F. Lott has been programming since the '70s, when computers were large,
expensive, and rare. He's been using Python to solve business problems for over 10 years.
His other titles with Packt Publishing include Python Essentials, Mastering Object-Oriented
Python, Functional Python Programming, and Python for Secret Agents. Steven is currently a
technomad who lives in city along the east coast of the U.S. You can follow his technology
blog (slott-softwarearchitect).
About the reviewer
Yogendra Sharma is a developer with experience in architecture, design, and development
of scalable and distributed applications. He was awarded a bachelors degree from the
Rajasthan Technical University in computer science. With a core interest in microservices
and Spring, he also has hands-on experience in technologies such as AWS Cloud, Python,
J2EE, NodeJS, JavaScript, Angular, MongoDB, and Docker.
Currently, he works as an IoT and Cloud Architect at Intelizign Engineering Services Pune.
Preface
Functional programming offers a variety of techniques for creating succinct and expressive
software. While Python is not a purely functional programming language, we can do a
great deal of functional programming in Python.
Python has a core set of functional programming features. This lets us borrow many design
patterns and techniques from other functional languages. These borrowed concepts can lead
us to create succinct and elegant programs. Python's generator expressions, in particular,
negate the need to create large in-memory data structures, leading to programs that may
execute more quickly because they use fewer resources.
We cant easily create purely functional programs in Python. Python lacks a number of
features that would be required for this. We dont have unlimited recursion, for example,
we dont have lazy evaluation of all expressions, and we dont have an optimizing compiler.
There are several key features of functional programming languages that are available in
Python. One of the most important ones is the idea of functions being first-class
objects. Python also offers a number of higher-order functions. The built-in ,
, and functions are widely used in this role, and lessobvious
are functions such as , , and .
Well look at the core features of functional programming from a Python point of view. Our
objective is to borrow good ideas from functional programming languages and use those
ideas to create expressive and succinct applications in Python.
Table of Contents
Copyright and Credits 3
Preface 1
Chapter 1: Understanding Functional Programming 8
Identifying a paradigm 9
Subdividing the procedural paradigm 10
Using the functional paradigm 11
Using a functional hybrid 14
Looking at object creation 15
The stack of turtles 16
A classic example of functional programming 17
Exploratory data analysis 20
Summary 21
Chapter 2: Introducing Essential Functional Concepts 22
First-class functions 23
Pure functions 23
Higher-order functions 24
Immutable data 25
Strict and non-strict evaluation 27
Recursion instead of an explicit loop state 29
Functional type systems 33
Familiar territory 33
Learning some advanced concepts 34
Summary 35
Chapter 3: Functions, Iterators, and Generators 36
Writing pure functions 37
Functions as first-class objects 39
Using strings 41
Using tuples and named tuples 42
Using generator expressions 44
Exploring the limitations of generators 46
Combining generator expressions 48
Cleaning raw data with generator functions 49
Using lists, dicts, and sets 50
Using stateful mappings 54
Using the bisect module to create a mapping 56
Using stateful sets 58
Summary 58
Chapter 4: Working with Collections 60
An overview of function varieties 61
Working with iterables 61
Parsing an XML file 63
Parsing a file at a higher level 65
Pairing up items from a sequence 67
Using the iter() function explicitly 70
Extending a simple loop 71
Applying generator expressions to scalar functions 74
Using any() and all() as reductions 76
Using len() and sum() 79
Using sums and counts for statistics 79
Using zip() to structure and flatten sequences 82
Unzipping a zipped sequence 84
Flattening sequences 84
Structuring flat sequences 86
Structuring flat sequences – an alternative approach 88
Using reversed() to change the order 89
Using enumerate() to include a sequence number 90
Summary 91
Chapter 5: Higher-Order Functions 92
Using max() and min() to find extrema 93
Using Python lambda forms 97
Lambdas and the lambda calculus 98
Using the map() function to apply a function to a collection 99
Working with lambda forms and map() 100
Using map() with multiple sequences 101
Using the filter() function to pass or reject data 103
Using filter() to identify outliers 104
The iter() function with a sentinel value 105
Using sorted() to put data in order 106
Writing higher-order functions 108
Writing higher-order mappings and filters 108
Unwrapping data while mapping 110
Wrapping additional data while mapping 112
Flattening data while mapping 114
Structuring data while filtering 116
Writing generator functions 117
Building higher-order functions with callables 120
Assuring good functional design 121
Review of some design patterns 123
Summary 124
Chapter 6: Recursions and Reductions 126
Simple numerical recursions 127
Implementing tail-call optimization 128
Leaving recursion in place 129
Handling difficult tail-call optimization 130
Processing collections through recursion 131
Tail-call optimization for collections 132
Reductions and folding a collection from many items to one 134
Group-by reduction from many items to fewer 136
Building a mapping with Counter 137
Building a mapping by sorting 138
Grouping or partitioning data by key values 140
Writing more general group-by reductions 143
Writing higher-order reductions 144
Writing file parsers 146
Parsing CSV files 148
Parsing plain text files with headers 150
Summary 153
Chapter 7: Additional Tuple Techniques 154
Using tuples to collect data 155
Using named tuples to collect data 157
Building named tuples with functional constructors 160
Avoiding stateful classes by using families of tuples 161
Assigning statistical ranks 165
Wrapping instead of state changing 167
Rewrapping instead of state changing 168
Computing Spearman rank-order correlation 170
Polymorphism and type-pattern matching 171
Summary 178
Chapter 8: The Itertools Module 179
Working with the infinite iterators 180
Counting with count() 181
Counting with float arguments 182
Re-iterating a cycle with cycle() 184
Repeating a single value with repeat() 186
Using the finite iterators 187
Assigning numbers with enumerate() 188
Running totals with accumulate() 190
Combining iterators with chain() 191
Partitioning an iterator with groupby() 192
Merging iterables with zip_longest() and zip() 194
Filtering with compress() 194
Picking subsets with islice() 196
Stateful filtering with dropwhile() and takewhile() 197
Two approaches to filtering with filterfalse() and filter() 198
Applying a function to data via starmap() and map() 199
Cloning iterators with tee() 201
The itertools recipes 201
Summary 203
Chapter 9: More Itertools Techniques 205
Enumerating the Cartesian product 206
Reducing a product 206
Computing distances 208
Getting all pixels and all colors 210
Performance analysis 212
Rearranging the problem 214
Combining two transformations 215
Permuting a collection of values 216
Generating all combinations 218
Recipes 220
Summary 221
Chapter 10: The Functools Module 222
Function tools 223
Memoizing previous results with lru_cache 223
Defining classes with total ordering 225
Defining number classes 228
Applying partial arguments with partial() 230
Reducing sets of data with the reduce() function 231
Combining map() and reduce() 232
Using the reduce() and partial() functions 234
Using the map() and reduce() functions to sanitize raw data 235
Using the groupby() and reduce() functions 236
Summary 239
Chapter 11: Decorator Design Techniques 241
Decorators as higher-order functions 241
Using the functools update_wrapper() functions 246
Cross-cutting concerns 246
Composite design 247
Preprocessing bad data 249
Adding a parameter to a decorator 251
Implementing more complex decorators 253
Complex design considerations 254
Summary 258
Chapter 12: The Multiprocessing and Threading Modules 259
Functional programming and concurrency 260
What concurrency really means 261
The boundary conditions 261
Sharing resources with process or threads 262
Where benefits will accrue 263
Using multiprocessing pools and tasks 264
Processing many large files 264
Parsing log files – gathering the rows 266
Parsing log lines into namedtuples 267
Parsing additional fields of an Access object 270
Filtering the access details 273
Analyzing the access details 275
The complete analysis process 276
Using a multiprocessing pool for concurrent processing 277
Using apply() to make a single request 280
Using the map_async(), starmap_async(), and apply_async() functions 280
More complex multiprocessing architectures 281
Using the concurrent.futures module 282
Using concurrent.futures thread pools 282
Using the threading and queue modules 283
Designing concurrent processing 284
Summary 286
Chapter 13: Conditional Expressions and the Operator Module 287
Evaluating conditional expressions 288
Exploiting non-strict dictionary rules 289
Filtering true conditional expressions 291
Finding a matching pattern 292
Using the operator module instead of lambdas 293
Getting named attributes when using higher-order functions 295
Starmapping with operators 296
Reducing with operator module functions 298
Summary 299
Chapter 14: The PyMonad Library 301
Downloading and installing 301
Functional composition and currying 302
Using curried higher-order functions 304
Currying the hard way 306
Functional composition and the PyMonad * operator 307
Functors and applicative functors 308
Using the lazy List() functor 310
Monad bind() function and the >> operator 313
Implementing simulation with monads 314
Additional PyMonad features 318
Summary 319
Chapter 15: A Functional Approach to Web Services 320
The HTTP request-response model 321
Injecting state through cookies 323
Considering a server with a functional design 324
Looking more deeply into the functional view 324
Nesting the services 325
The WSGI standard 326
Throwing exceptions during WSGI processing 329
Pragmatic WSGI applications 331
Defining web services as functions 331
Creating the WSGI application 332
Getting raw data 335
Applying a filter 337
Serializing the results 337
Serializing data into JSON or CSV formats 339
Serializing data into XML 340
Serializing data into HTML 341
Tracking usage 343
Summary 344
Chapter 16: Optimizations and Improvements 346
Memoization and caching 347
Specializing memoization 348
Tail recursion optimizations 350
Optimizing storage 352
Optimizing accuracy 353
Reducing accuracy based on audience requirements 353
Case study–making a chi-squared decision 354
Filtering and reducing the raw data with a Counter object 355
Reading summarized data 357
Computing sums with a Counter object 358
Computing probabilities from Counter objects 360
Computing expected values and displaying a contingency table 361
Computing the chi-squared value 363
Computing the chi-squared threshold 364
Computing the incomplete gamma function 365
Computing the complete gamma function 368
Computing the odds of a distribution being random 369
Functional programming design patterns 371
Summary 373
Other Books You May Enjoy 375
Bookscreen
Who this book is for
This book is for programmers who want to create succinct, expressive Python programs by
borrowing techniques and design patterns from functional programming languages. Some
algorithms can be expressed elegantly in a functional style; we canand shouldadapt
this to make Python programs more readable and maintainable.
In some cases, a functional approach to a problem will also lead to extremely highperformance
algorithms. Python makes it too easy to create large intermediate data
structures, tying up memory (and processor time.) With functional programming design
patterns, we can often replace large lists with generator expressions that are equally
expressive but take up much less memory and run much more quickly.