François Chollet
brief contents
PART 1 FUNDAMENTALS OF DEEP LEARNING
What is deep learning?
Before we begin: the mathematical building blocks of neural networks
Getting started with neural networks
Fundamentals of machine learning
PART 2 DEEP LEARNING IN PRACTICE
Deep learning for computer vision
Deep learning for text and sequences
Advanced deep-learning best practices
Generative deep learning
Conclusions
Purchase Now !
Just with Paypal
Just with Paypal
Book Details
Price
|
3.00 USD |
|---|---|
Pages
| 386 p |
File Size
|
11,170 KB |
File Type
|
PDF format |
ISBN
| 9781617294433 |
Copyright
| 2018 by Manning Publications Co |
This book was written for anyone who wishes to explore deep learning from scratch or
broaden their understanding of deep learning. Whether you’re a practicing machine-learning
engineer, a software developer, or a college student, you’ll find value in these pages.
This book offers a practical, hands-on exploration of deep learning. It avoids mathematical
notation, preferring instead to explain quantitative concepts via code snippets
and to build practical intuition about the core ideas of machine learning and deep learning.
You’ll learn from more than 30 code examples that include detailed commentary,
practical recommendations, and simple high-level explanations of everything you
need to know to start using deep learning to solve concrete problems.
The code examples use the Python deep-learning framework Keras, with Tensor-
Flow as a backend engine. Keras, one of the most popular and fastest-growing deeplearning
frameworks, is widely recommended as the best tool to get started with deep learning.
After reading this book, you’ll have a solid understand of what deep learning is,
when it’s applicable, and what its limitations are. You’ll be familiar with the standard
workflow for approaching and solving machine-learning problems, and you’ll know
how to address commonly encountered issues. You’ll be able to use Keras to tackle
real-world problems ranging from computer vision to natural-language processing:
image classification, timeseries forecasting, sentiment analysis, image and text generation,
and more.
Who should read this book
This book is written for people with Python programming experience who want to get
started with machine learning and deep learning. But this book can also be valuable
to many different types of readers:
If you’re a data scientist familiar with machine learning, this book will provide
you with a solid, practical introduction to deep learning, the fastest-growing
and most significant subfield of machine learning.
If you’re a deep-learning expert looking to get started with the Keras framework,
you’ll find this book to be the best Keras crash course available.
If you’re a graduate student studying deep learning in a formal setting, you’ll
find this book to be a practical complement to your education, helping you
build intuition around the behavior of deep neural networks and familiarizing
you with key best practices.
Even technically minded people who don’t code regularly will find this book useful as
an introduction to both basic and advanced deep-learning concepts.
In order to use Keras, you’ll need reasonable Python proficiency. Additionally, familiarity
with the Numpy library will be helpful, although it isn’t required. You don’t need
previous experience with machine learning or deep learning: this book covers from
scratch all the necessary basics. You don’t need an advanced mathematics background,
either—high school–level mathematics should suffice in order to follow along.
about the author
François Chollet works on deep learning at Google in Mountain
View, CA. He is the creator of the Keras deep-learning
library, as well as a contributor to the TensorFlow machinelearning
framework. He also does deep-learning research,
with a focus on computer vision and the application of
machine learning to formal reasoning. His papers have been
published at major conferences in the field, including the
Conference on Computer Vision and Pattern Recognition
(CVPR), the Conference and Workshop on Neural Information
Processing Systems (NIPS), the International Conference on Learning Representations
(ICLR), and others.
about the cover
The figure on the cover of Deep Learning with Python is captioned “Habit of a Persian
Lady in 1568.” The illustration is taken from Thomas Jefferys’ A Collection of the Dresses
of Different Nations, Ancient and Modern (four volumes), London, published between
1757 and 1772. The title page states that these are hand-colored copperplate engravings,
heightened with gum arabic.
Thomas Jefferys (1719–1771) was called “Geographer to King George III.” He was
an English cartographer who was the leading map supplier of his day. He engraved
and printed maps for government and other official bodies and produced a wide
range of commercial maps and atlases, especially of North America. His work as a map
maker sparked an interest in local dress customs of the lands he surveyed and
mapped, which are brilliantly displayed in this collection. Fascination with faraway
lands and travel for pleasure were relatively new phenomena in the late eighteenth
century, and collections such as this one were popular, introducing both the tourist as
well as the armchair traveler to the inhabitants of other countries.
The diversity of the drawings in Jefferys’ volumes speaks vividly of the uniqueness
and individuality of the world’s nations some 200 years ago. Dress codes have changed
since then, and the diversity by region and country, so rich at the time, has faded away.
It’s now often hard to tell the inhabitants of one continent from another. Perhaps, trying
to view it optimistically, we’ve traded a cultural and visual diversity for a more varied
personal life—or a more varied and interesting intellectual and technical life.
At a time when it’s difficult to tell one computer book from another, Manning celebrates
the inventiveness and initiative of the computer business with book covers
based on the rich diversity of regional life of two centuries ago, brought back to life by
Jefferys’ pictures.
Table of Contents
preface xiii
acknowledgments xv
about this book xvi
about the author xx
about the cover xxi
PART 1 FUNDAMENTALS OF DEEP LEARNING
1 What is deep learning? 3
1.1 Artificial intelligence, machine learning,
and deep learning 4
Artificial intelligence 4 ■ Machine learning 4 ■ Learning
representations from data 6 ■ The “deep” in deep learning 8
Understanding how deep learning works, in three figures 9
What deep learning has achieved so far 11 ■ Don’t believe
the short-term hype 12 ■ The promise of AI 13
1.2 Before deep learning: a brief history of machine
learning 14
Probabilistic modeling 14 ■ Early neural networks 14
Kernel methods 15 ■ Decision trees, random forests,
and gradient boosting machines 16 ■ Back to neural
networks 17 ■ What makes deep learning different 17
The modern machine-learning landscape 18
1.3 Why deep learning? Why now? 20
Hardware 20 ■ Data 21 ■ Algorithms 21 ■ A new
wave of investment 22 ■ The democratization of deep
learning 23 ■ Will it last? 23
2 Before we begin: the mathematical building blocks of
neural networks 25
2.1 A first look at a neural network 27
2.2 Data representations for neural networks 31
Scalars (0D tensors) 31 ■ Vectors (1D tensors) 31
Matrices (2D tensors) 31 ■ 3D tensors and higherdimensional
tensors 32 ■ Key attributes 32
Manipulating tensors in Numpy 34 ■ The notion
of data batches 34 ■ Real-world examples of data
tensors 35 ■ Vector data 35 ■ Timeseries data or
sequence data 35 ■ Image data 36 ■ Video data 37
2.3 The gears of neural networks: tensor operations 38
Element-wise operations 38 ■ Broadcasting 39 ■ Tensor
dot 40 ■ Tensor reshaping 42 ■ Geometric interpretation
of tensor operations 43 ■ A geometric interpretation of deep
learning 44
2.4 The engine of neural networks: gradient-based
optimization 46
What’s a derivative? 47 ■ Derivative of a tensor operation:
the gradient 48 ■ Stochastic gradient descent 48
Chaining derivatives: the Backpropagation algorithm 51
2.5 Looking back at our first example 53
2.6 Chapter summary 55
3 Getting started with neural networks 56
3.1 Anatomy of a neural network 58
Layers: the building blocks of deep learning 58 ■ Models:
networks of layers 59 ■ Loss functions and optimizers: keys
to configuring the learning process 60
3.2 Introduction to Keras 61
Keras, TensorFlow, Theano, and CNTK 62 ■ Developing
with Keras: a quick overview 62
3.3 Setting up a deep-learning workstation 65
Jupyter notebooks: the preferred way to run deep-learning
experiments 65 ■ Getting Keras running: two options 66
Running deep-learning jobs in the cloud: pros and cons 66
What is the best GPU for deep learning? 66
3.4 Classifying movie reviews: a binary classification
example 68
The IMDB dataset 68 ■ Preparing the data 69
Building your network 70 ■ Validating your approach 73
Using a trained network to generate predictions on new
data 76 ■ Further experiments 77 ■ Wrapping up 77
3.5 Classifying newswires: a multiclass classification
example 78
The Reuters dataset 78 ■ Preparing the data 79
Building your network 79 ■ Validating your approach 80
Generating predictions on new data 83 ■ A different way to
handle the labels and the loss 83 ■ The importance of
having sufficiently large intermediate layers 83 ■ Further
experiments 84 ■ Wrapping up 84
3.6 Predicting house prices: a regression example 85
The Boston Housing Price dataset 85 ■ Preparing the
data 86 ■ Building your network 86 ■ Validating
your approach using K-fold validation 87 ■ Wrapping up 91
3.7 Chapter summary 92
4 Fundamentals of machine learning 93
4.1 Four branches of machine learning 94
Supervised learning 94 ■ Unsupervised learning 94
Self-supervised learning 94 ■ Reinforcement learning 95
4.2 Evaluating machine-learning models 97
Training, validation, and test sets 97 ■ Things to
keep in mind 100
4.3 Data preprocessing, feature engineering,
and feature learning 101
Data preprocessing for neural networks 101 ■ Feature
engineering 102
4.4 Overfitting and underfitting 104
Reducing the network’s size 104 ■ Adding weight
regularization 107 ■ Adding dropout 109
4.5 The universal workflow of machine learning 111
Defining the problem and assembling a dataset 111
Choosing a measure of success 112 ■ Deciding on an
evaluation protocol 112 ■ Preparing your data 112
Developing a model that does better than a baseline 113
Scaling up: developing a model that overfits 114
Regularizing your model and tuning your hyperparameters 114
4.6 Chapter summary 116
PART 2 DEEP LEARNING IN PRACTICE
5 Deep learning for computer vision 119
5.1 Introduction to convnets 120
The convolution operation 122 ■ The max-pooling
operation 127
5.2 Training a convnet from scratch on a small dataset 130
The relevance of deep learning for small-data problems 130
Downloading the data 131 ■ Building your network 133
Data preprocessing 135 ■ Using data augmentation 138
5.3 Using a pretrained convnet 143
Feature extraction 143 ■ Fine-tuning 152 ■ Wrapping up 159
5.4 Visualizing what convnets learn 160
Visualizing intermediate activations 160 ■ Visualizing
convnet filters 167 ■ Visualizing heatmaps of class
activation 172
5.5 Chapter summary 177
6 Deep learning for text and sequences 178
6.1 Working with text data 180
One-hot encoding of words and characters 181 ■ Using
word embeddings 184 ■ Putting it all together: from raw
text to word embeddings 188 ■ Wrapping up 195
6.2 Understanding recurrent neural networks 196
A recurrent layer in Keras 198 ■ Understanding the
LSTM and GRU layers 202 ■ A concrete LSTM example
in Keras 204 ■ Wrapping up 206
6.3 Advanced use of recurrent neural networks 207
A temperature-forecasting problem 207 ■ Preparing the
data 210 ■ A common-sense, non-machine-learning
baseline 212 ■ A basic machine-learning approach 213
A first recurrent baseline 215 ■ Using recurrent dropout
to fight overfitting 216 ■ Stacking recurrent layers 217
Using bidirectional RNNs 219 ■ Going even further 222
Wrapping up 223
6.4 Sequence processing with convnets 225
Understanding 1D convolution for sequence data 225
1D pooling for sequence data 226 ■ Implementing a 1D
convnet 226 ■ Combining CNNs and RNNs to process long
sequences 228 ■ Wrapping up 231
6.5 Chapter summary 232
7 Advanced deep-learning best practices 233
7.1 Going beyond the Sequential model: the Keras
functional API 234
Introduction to the functional API 236 ■ Multi-input
models 238 ■ Multi-output models 240 ■ Directed acyclic
graphs of layers 242 ■ Layer weight sharing 246 ■ Models
as layers 247 ■ Wrapping up 248
7.2 Inspecting and monitoring deep-learning models using
Keras callbacks and TensorBoard 249
Using callbacks to act on a model during training 249
Introduction to TensorBoard: the TensorFlow visualization
framework 252 ■ Wrapping up 259
7.3 Getting the most out of your models 260
Advanced architecture patterns 260 ■ Hyperparameter
optimization 263 ■ Model ensembling 264 ■ Wrapping
up 266
7.4 Chapter summary 268
8 Generative deep learning 269
8.1 Text generation with LSTM 271
A brief history of generative recurrent networks 271 ■ How
do you generate sequence data? 272 ■ The importance of
the sampling strategy 272 ■ Implementing character-level
LSTM text generation 274 ■ Wrapping up 279
8.2 DeepDream 280
Implementing DeepDream in Keras 281 ■ Wrapping up 286
8.3 Neural style transfer 287
The content loss 288 ■ The style loss 288 ■ Neural style
transfer in Keras 289 ■ Wrapping up 295
8.4 Generating images with variational autoencoders 296
Sampling from latent spaces of images 296 ■ Concept vectors
for image editing 297 ■ Variational autoencoders 298
Wrapping up 304
8.5 Introduction to generative adversarial networks 305
A schematic GAN implementation 307 ■ A bag of tricks 307
The generator 308 ■ The discriminator 309 ■ The adversarial
network 310 ■ How to train your DCGAN 310 ■ Wrapping
up 312
8.6 Chapter summary 313
9 Conclusions 314
9.1 Key concepts in review 315
Various approaches to AI 315 ■ What makes deep learning
special within the field of machine learning 315 ■ How to
think about deep learning 316 ■ Key enabling technologies 317
The universal machine-learning workflow 318 ■ Key network
architectures 319 ■ The space of possibilities 322
9.2 The limitations of deep learning 325
The risk of anthropomorphizing machine-learning models 325
Local generalization vs. extreme generalization 327
Wrapping up 329
9.3 The future of deep learning 330
Models as programs 330 ■ Beyond backpropagation and
differentiable layers 332 ■ Automated machine learning 332
Lifelong learning and modular subroutine reuse 333
The long-term vision 335
9.4 Staying up to date in a fast-moving field 337
Practice on real-world problems using Kaggle 337
Read about the latest developments on arXiv 337
Explore the Keras ecosystem 338
9.5 Final words 339
appendix A Installing Keras and its dependencies on Ubuntu 340
appendix B Running Jupyter notebooks on an EC2 GPU instance 345
index 353
Bookscreen
preface
If you’ve picked up this book, you’re probably aware of the extraordinary progress
that deep learning has represented for the field of artificial intelligence in the recent
past. In a mere five years, we’ve gone from near-unusable image recognition and
speech transcription, to superhuman performance on these tasks.
The consequences of this sudden progress extend to almost every industry. But in
order to begin deploying deep-learning technology to every problem that it could
solve, we need to make it accessible to as many people as possible, including nonexperts—
people who aren’t researchers or graduate students. For deep learning to
reach its full potential, we need to radically democratize it.
When I released the first version of the Keras deep-learning framework in March
2015, the democratization of AI wasn’t what I had in mind. I had been doing research
in machine learning for several years, and had built Keras to help me with my own
experiments. But throughout 2015 and 2016, tens of thousands of new people
entered the field of deep learning; many of them picked up Keras because it was—and
still is—the easiest framework to get started with. As I watched scores of newcomers
use Keras in unexpected, powerful ways, I came to care deeply about the accessibility
and democratization of AI. I realized that the further we spread these technologies,
the more useful and valuable they become. Accessibility quickly became an explicit
goal in the development of Keras, and over a few short years, the Keras developer
community has made fantastic achievements on this front. We’ve put deep learning
into the hands of tens of thousands of people, who in turn are using it to solve important
problems we didn’t even know existed until recently.
The book you’re holding is another step on the way to making deep learning available
to as many people as possible. Keras had always needed a companion course to
simultaneously cover fundamentals of deep learning, Keras usage patterns, and deeplearning
best practices. This book is my best effort to produce such a course. I wrote it
with a focus on making the concepts behind deep learning, and their implementation,
as approachable as possible. Doing so didn’t require me to dumb down anything—
I strongly believe that there are no difficult ideas in deep learning. I hope
you’ll find this book valuable and that it will enable you to begin building intelligent
applications and solve the problems that matter to you.