MICHAEL WITTIG . ANDREAS WITTIG . FOREWORD BY BEN WHALEY
Praise for the First Edition
Fantastic introduction to cloud basics with excellent real-world examples.
—Rambabu Posa, GL Assessment
A very thorough and practical guide to everything AWS ... highly recommended.
—Scott M. King, Amazon
Cuts through the vast expanse of official documentation and gives you what you
need to make AWS work now!
—Carm Vecchio, Computer Science Corporation (CSC)
The right book to program AWS from scratch.
—Javier Muñoz Mellid, Senior Computer Engineer, Igalia
Brief Contents
PART 1 GETTING STARTED
1 ■ What is Amazon Web Services?
2 ■ A simple example: WordPress in five minutes
PART 2 BUILDING VIRTUAL INFRASTRUCTURE
CONSISTING OF COMPUTERS AND NETWORKING
3 ■ Using virtual machines: EC2
4 ■ Programming your infrastructure: The command-line,
SDKs, and CloudFormation
5 ■ Automating deployment: CloudFormation,
Elastic Beanstalk, and OpsWorks
6 ■ Securing your system: IAM, security groups, and VPC
7 ■ Automating operational tasks with Lambda 199
PART 3 STORING DATA IN THE CLOUD
8 ■ Storing your objects: S3 and Glacier
9 ■ Storing data on hard drives: EBS and instance store
10 ■ Sharing data volumes between machines: EFS
11 ■ Using a relational database service: RDS
12 ■ Caching data in memory: Amazon ElastiCache
13 ■ Programming for the NoSQL database service: DynamoDB
PART 4 ARCHITECTING ON AWS
14 ■ Achieving high availability: availability zones, auto-scaling,
and CloudWatch
15 ■ Decoupling your infrastructure: Elastic Load Balancing
and Simple Queue Service
16 ■ Designing for fault tolerance
17 ■ Scaling up and down: auto-scaling and CloudWatch
Purchase Now !
Just with Paypal
Just with Paypal
Book Details
Price
|
5.00 USD |
|---|---|
Pages
| 530 p |
File Size
|
23,730 KB |
File Type
|
PDF format |
ISBN
| 9781617295119 |
Copyright
| 2019 by Manning Publications Co |
Andreas Wittig and Michael Wittig are software and DevOps engineers
focusing on Amazon Web Services. The brothers started building on AWS
in 2013 when migrating the IT infrastructure of a German bank to AWS—
the first bank in Germany to do so. Since 2015, Andreas and Michael have
worked as consultants helping their clients to migrate and run their workloads
on AWS. They focus on infrastructure-as-code, continuous deployment,
serverless, Docker, and security. Andreas and Michael build SaaS
products on top of the Amazon’s cloud as well. Both are certified as AWS
Certified Solutions Architect - Professional and AWS Certified DevOps
Engineer - Professional. In addition, Andreas and Michael love sharing
their knowledge and teaching how to use Amazon Web Services through this book,
their blog (cloudonaut.io), as well as online- and on-site trainings (such as AWS in
about the cover illustration
The figure on the cover of Amazon Web Services in Action, Second Edition is captioned “Paysan
du Canton de Lucerne,” or a peasant from the canton of Lucerne in central Switzerland.
The illustration is taken from a collection of dress costumes from various
countries by Jacques Grasset de Saint-Sauveur (1757-1810), titled Costumes de Différent
Pays, published in France in 1797. Each illustration is finely drawn and colored by hand.
The rich variety of Grasset de Saint--Sauveur’s collection reminds us vividly of how
culturally apart the world’s towns and regions were just 200 years ago. Isolated from
each other, people spoke different dialects and languages. In the streets or in the
countryside, it was easy to identify where they lived and what their trade or station in
life was just by their dress.
The way we dress has changed since then and the diversity by region, so rich at the
time, has faded away. It is now hard to tell apart the inhabitants of different continents,
let alone different towns, regions, or countries. Perhaps we have traded cultural
diversity for a more varied personal life—certainly for a more varied and fast-paced
technological life.
At a time when it is hard to tell one computer book from another, Manning celebrates
the inventiveness and initiative of the computer business with book covers
based on the rich diversity of regional life of two centuries ago, brought back to life by
Grasset de Saint-Sauveur’s pictures.
about this book
Our book guides you from creating an AWS account to building fault-tolerant and
auto-scaling applications. You will learn about services offering compute, network,
and storage capacity. We get you started with everything you need to run web applications
on AWS: load balancers, virtual machines, file storage, database systems, and inmemory caches.
The first part of the book introduces the principles of Amazon Web Services and gives
you a first impression of the possibilities in the cloud. Next, you will learn about fundamental
compute and network services. Afterward, we demonstrate six different ways to
store your data. The last part of our book focuses on highly available or even faulttolerant
architectures that allow you to scale your infrastructure dynamically as well.
Amazon offers a wide variety of services. Unfortunately, the number of pages within
a book is limited. Therefore, we had to skip topics such as containers, big data, and
machine learning. We cover the basic or most important services, though.
Automation sneaks in throughout the book, so by the end you’ll be comfortable
with using AWS CloudFormation, an infrastructure-as-code tool that allows you to
manage your cloud infrastructure in an automated way; this will be one of the most
important things you will learn from our book.
Most of our examples use popular web applications to demonstrate important
points. We use tools offered by AWS instead of third-party tools whenever possible, as
we appreciate the quality and support offered by AWS. Our book focuses on the different
aspects of security in the cloud, for example by following the “least privilege” principle
when accessing cloud resources.
We focus on Linux as the operating system for virtual machines in the book. Our
examples are based on open source software.
Amazon operates data centers in geographic regions around the world. To simplify
the examples we are using the region US East (N. Virginia) within our book. You will
also learn how to switch to another region to exemplarily make use of resources in
Asia Pacific (Sydney).
Table of Contents
foreword xvii
preface xix
acknowledgments xxi
about this book xxiii
about the author xxvii
about the cover illustration xxviii
PART 1 GETTING STARTED
1 What is Amazon Web Services? 3
1.1 What is cloud computing? 4
1.2 What can you do with AWS? 5
Hosting a web shop 5 ■ Running a Java EE application in your
private network 7 ■ Implementing a highly available system 8
Profiting from low costs for batch processing infrastructure 9
1.3 How you can benefit from using AWS 10
Innovative and fast-growing platform 10 ■ Services solve common
problems 10 ■ Enabling automation 10 ■ Flexible capacity
(scalability) 11 ■ Built for failure (reliability) 11 ■ Reducing
time to market 11 ■ Benefiting from economies of scale 12
Global infrastructure 12 ■ Professional partner 12
1.4 How much does it cost? 12
Free Tier 13 ■ Billing example 13 ■ Pay-per-use opportunities 15
1.5 Comparing alternatives 15
1.6 Exploring AWS services 16
1.7 Interacting with AWS 19
Management Console 19 ■ Command-line interface 20
SDKs 21 ■ Blueprints 22
1.8 Creating an AWS account 22
Signing up 23 ■ Signing In 28 ■ Creating a key pair 29
1.9 Create a billing alarm to keep track of your AWS bill 33
2 A simple example: WordPress in five minutes 36
2.1 Creating your infrastructure 37
2.2 Exploring your infrastructure 44
Resource groups 44 ■ Virtual machines 45 ■ Load
balancer 47 ■ MySQL database 49 ■ Network filesystem 50
2.3 How much does it cost? 52
2.4 Deleting your infrastructure 54
PART 2 BUILDING VIRTUAL INFRASTRUCTURE CONSISTING OF COMPUTERS
AND NETWORKING
3 Using virtual machines: EC2 59
3.1 Exploring a virtual machine 60
Launching a virtual machine 60 ■ Connecting to your virtual
machine 72 ■ Installing and running software manually 75
3.2 Monitoring and debugging a virtual machine 76
Showing logs from a virtual machine 76 ■ Monitoring the load of
a virtual machine 77
3.3 Shutting down a virtual machine 78
3.4 Changing the size of a virtual machine 79
3.5 Starting a virtual machine in another data center 82
3.6 Allocating a public IP address 86
3.7 Adding an additional network interface to a virtual machine 88
3.8 Optimizing costs for virtual machines 92
Reserve virtual machines 93 ■ Bidding on unused virtual machines 95
Programming your infrastructure: The command-line, SDKs, and
CloudFormation 102
4.1 Infrastructure as Code 104
Automation and the DevOps movement 104 ■ Inventing an
infrastructure language: JIML 105
4.2 Using the command-line interface 108
Why should you automate? 108 ■ Installing the CLI 109
Configuring the CLI 110 ■ Using the CLI 113
4.3 Programming with the SDK 117
Controlling virtual machines with SDK: nodecc 118 ■ How
nodecc creates a virtual machine 119 ■ How nodecc lists virtual
machines and shows virtual machine details 120 ■ How nodecc
terminates a virtual machine 121
4.4 Using a blueprint to start a virtual machine 121
Anatomy of a CloudFormation template 122 ■ Creating your first
template 126
5 Automating deployment: CloudFormation, Elastic Beanstalk, and
OpsWorks 135
5.1 Deploying applications in a flexible cloud environment 136
5.2 Comparing deployment tools 137
Classifying the deployment tools 138 ■ Comparing the deployment
services 138
5.3 Creating a virtual machine and run a deployment script on
startup with AWS CloudFormation 139
Using user data to run a script on startup 140 ■ Deploying
OpenSwan: a VPN server to a virtual machine 140 ■ Starting
from scratch instead of updating 145
5.4 Deploying a simple web application with AWS Elastic
Beanstalk 145
Components of AWS Elastic Beanstalk 146 ■ Using AWS Elastic
Beanstalk to deploy Etherpad, a Node.js application 146
5.5 Deploying a multilayer application with AWS OpsWorks
Stacks 151
Components of AWS OpsWorks Stacks 152 ■ Using AWS
OpsWorks Stacks to deploy an IRC chat application 153
6 Securing your system: IAM, security groups, and VPC 165
6.1 Who’s responsible for security? 167
6.2 Keeping your software up to date 168
Checking for security updates 168 ■ Installing security updates on
startup 169 ■ Installing security updates on running virtual
machines 170
6.3 Securing your AWS account 171
Securing your AWS account’s root user 172 ■ AWS Identity and
Access Management (IAM) 173 ■ Defining permissions with an
IAM policy 174 ■ Users for authentication, and groups to organize
users 176 ■ Authenticating AWS resources with roles 177
6.4 Controlling network traffic to and from your virtual
machine 179
Controlling traffic to virtual machines with security groups 181
Allowing ICMP traffic 182 ■ Allowing SSH traffic 183
Allowing SSH traffic from a source IP address 184 ■ Allowing
SSH traffic from a source security group 185
6.5 Creating a private network in the cloud: Amazon Virtual Private
Cloud (VPC) 189
Creating the VPC and an internet gateway (IGW) 190 ■ Defining the
public bastion host subnet 192 ■ Adding the private Apache web server
subnet 194 ■ Launching virtual machines in the subnets 195
Accessing the internet from private subnets via a NAT gateway 196
7 Automating operational tasks with Lambda 199
7.1 Executing your code with AWS Lambda 200
What is serverless? 201 ■ Running your code on AWS Lambda 201
Comparing AWS Lambda with virtual machines (Amazon EC2) 202
7.2 Building a website health check with AWS Lambda 203
Creating a Lambda function 204 ■ Use CloudWatch to search
through your Lambda function’s logs 210 ■ Monitoring a
Lambda function with CloudWatch metrics and alarms 212
Accessing endpoints within a VPC 217
7.3 Adding a tag containing the owner of an EC2 instance
automatically 218
Event-driven: Subscribing to CloudWatch events 219 ■ Implementing
the Lambda function in Python 222 ■ Setting up a Lambda function
with the Serverless Application Model (SAM) 223 ■ Authorizing a
Lambda function to use other AWS services with an IAM role 224
Deploying a Lambda function with SAM 226
7.4 What else can you do with AWS Lambda? 227
What are the limitations of AWS Lambda? 227 ■ Impacts of the
serverless pricing model 228 ■ Use case: Web application 229
Use case: Data processing 230 ■ Use case: IoT back end 231
PART 3 STORING DATA IN THE CLOUD
8 Storing your objects: S3 and Glacier 235
8.1 What is an object store? 236
8.2 Amazon S3 237
8.3 Backing up your data on S3 with AWS CLI 238
8.4 Archiving objects to optimize costs 241
Creating an S3 bucket for the use with Glacier 241 ■ Adding a
lifecycle rule to a bucket 242 ■ Experimenting with Glacier and
your lifecycle rule 245
8.5 Storing objects programmatically 248
Setting up an S3 bucket 249 ■ Installing a web application that
uses S3 249 ■ Reviewing code access S3 with SDK 250
8.6 Using S3 for static web hosting 252
Creating a bucket and uploading a static website 253
Configuring a bucket for static web hosting 253 ■ Accessing a
website hosted on S3 254
8.7 Best practices for using S3 255
Ensuring data consistency 255 ■ Choosing the right keys 256
9 Storing data on hard drives: EBS and instance store 258
9.1 Elastic Block Store (EBS): Persistent block-level storage
attached over the network 259
Creating an EBS volume and attaching it to your EC2
instance 260 ■ Using EBS 261 ■ Tweaking performance 263
Backing up your data with EBS snapshots 266
9.2 Instance store: Temporary block-level storage 268
Using an instance store 271 ■ Testing performance 272
Backing up your data 272
10 Sharing data volumes between machines: EFS 274
10.1 Creating a filesystem 277
Using CloudFormation to describe a filesystem 277 ■ Pricing 277
10.2 Creating a mount target 278
10.3 Mounting the EFS share on EC2 instances 280
10.4 Sharing files between EC2 instances 283
10.5 Tweaking performance 284
Performance mode 285 ■ Expected throughput 285
10.6 Monitoring a filesystem 286
Should you use Max I/O Performance mode? 286 ■ Monitoring
your permitted throughput 287 ■ Monitoring your usage 288
10.7 Backing up your data 289
Using CloudFormation to describe an EBS volume 290 ■ Using the
EBS volume 290
11 Using a relational database service: RDS 294
11.1 Starting a MySQL database 296
Launching a WordPress platform with an RDS database 297
Exploring an RDS database instance with a MySQL engine 299
Pricing for Amazon RDS 300
11.2 Importing data into a database 300
11.3 Backing up and restoring your database 303
Configuring automated snapshots 303 ■ Creating snapshots
manually 304 ■ Restoring a database 305 ■ Copying a
database to another region 307 ■ Calculating the cost of snapshots 308
11.4 Controlling access to a database 308
Controlling access to the configuration of an RDS database 309
Controlling network access to an RDS database 310 ■ Controlling
data access 311
11.5 Relying on a highly available database 311
Enabling high-availability deployment for an RDS database 313
11.6 Tweaking database performance 314
Increasing database resources 314 ■ Using read replication to
increase read performance 316
11.7 Monitoring a database 318
12 Caching data in memory: Amazon ElastiCache 321
12.1 Creating a cache cluster 327
Minimal CloudFormation template 327 ■ Test the Redis cluster 328
12.2 Cache deployment options 330
Memcached: cluster 330 ■ Redis: Single-node cluster 331
Redis: Cluster with cluster mode disabled 332 ■ Redis: Cluster with
cluster mode enabled 332
12.3 Controlling cache access 334
Controlling access to the configuration 334 ■ Controlling network
access 334 ■ Controlling cluster and data access 335
12.4 Installing the sample application Discourse with
CloudFormation 336
VPC: Network configuration 337 ■ Cache: Security group, subnet group,
cache cluster 338 ■ Database: Security group, subnet group, database
instance 339 ■ Virtual machine—security group, EC2 instance 340
Testing the CloudFormation template for Discourse 342
12.5 Monitoring a cache 344
Monitoring host-level metrics 344 ■ Is my memory
sufficient? 345 ■ Is my Redis replication up-to-date? 345
12.6 Tweaking cache performance 346
Selecting the right cache node type 347 ■ Selecting the right
deployment option 347 ■ Compressing your data 348
13 Programming for the NoSQL database service: DynamoDB 349
13.1 Operating DynamoDB 351
Administration 352 ■ Pricing 352 ■ Networking 353
RDS comparison 353 ■ NoSQL comparison 354
13.2 DynamoDB for developers 354
Tables, items, and attributes 354 ■ Primary key 355
DynamoDB Local 356
13.3 Programming a to-do application 356
13.4 Creating tables 358
Users are identified by a partition key 358 ■ Tasks are identified by
a partition key and sort key 360
13.5 Adding data 361
Adding a user 363 ■ Adding a task 363
13.6 Retrieving data 364
Getting an item by key 365 ■ Querying items by key and
filter 366 ■ Using global secondary indexes for more flexible
queries 368 ■ Scanning and filtering all of your table’s data 371
Eventually consistent data retrieval 372
13.7 Removing data 373
13.8 Modifying data 374
13.9 Scaling capacity 375
Capacity units 375 ■ Auto-scaling 377
PART 4 ARCHITECTING ON AWS
14 Achieving high availability: availability zones, auto-scaling, and
CloudWatch 383
14.1 Recovering from EC2 instance failure with CloudWatch 385
Creating a CloudWatch alarm to trigger recovery when status checks
fail 387 ■ Monitoring and recovering a virtual machine based on
a CloudWatch alarm 388
14.2 Recovering from a data center outage 392
Availability zones: groups of isolated data centers 392 ■ Using
auto-scaling to ensure that an EC2 instance is always running 396
Recovering a failed virtual machine to another availability zone with
the help of auto-scaling 399 ■ Pitfall: recovering network-attached
storage 402 ■ Pitfall: network interface recovery 407
14.3 Analyzing disaster-recovery requirements 411
RTO and RPO comparison for a single EC2 instance 411
15 Decoupling your infrastructure: Elastic Load Balancing and Simple
Queue Service 413
15.1 Synchronous decoupling with load balancers 415
Setting up a load balancer with virtual machines 416
15.2 Asynchronous decoupling with message queues 420
Turning a synchronous process into an asynchronous one 421
Architecture of the URL2PNG application 422 ■ Setting up a
message queue 423 ■ Producing messages programmatically 423
Consuming messages programmatically 425 ■ Limitations of
messaging with SQS 428
16 Designing for fault tolerance 431
16.1 Using redundant EC2 instances to increase availability 434
Redundancy can remove a single point of failure 434
Redundancy requires decoupling 436
16.2 Considerations for making your code fault-tolerant 437
Let it crash, but also retry 437 ■ Idempotent retry makes fault
tolerance possible 438
16.3 Building a fault-tolerant web application: Imagery 440
The idempotent state machine 443 ■ Implementing a fault-tolerant
web service 444 ■ Implementing a fault-tolerant worker to consume
SQS messages 452 ■ Deploying the application 455
17 Scaling up and down: auto-scaling and CloudWatch 463
17.1 Managing a dynamic EC2 instance pool 465
17.2 Using metrics or schedules to trigger scaling 469
Scaling based on a schedule 471 ■ Scaling based on CloudWatch
metrics 472
17.3 Decouple your dynamic EC2 instance pool 475
Scaling a dynamic EC2 instance pool synchronously decoupled by a
load balancer 476 ■ Scaling a dynamic EC2 instances pool
asynchronously decoupled by a queue 480
index 487
Bookscreen
foreword
Throughout the late 1990s and early 2000s I worked in the rank and file of system
administrators endeavoring to keep network services online, secure, and available to
users. At the time, administration was a tedious, onerous affair involving cable slinging,
server racking, installing from optical media, and configuring software manually. It was
thankless work, often an exercise in frustration, requiring patience, persistence, and
plenty of caffeine. To participate in the emerging online marketplace, businesses of the
era bore the burden of managing this physical infrastructure, accepting the associated
capital and operating costs and hoping for enough success to justify those expenses.
When Amazon Web Services emerged in 2006, it signaled a shift in the industry.
Management of compute and storage resources was dramatically simplified, and the
cost of building and launching applications plummeted. Suddenly anyone with a
good idea and the ability to execute could build a global business on world-class infrastructure
at a starting cost of just a few cents an hour. The AWS value proposition was
immediately apparent, ushering in a wave of new startups, data center migrations, and
third-party service providers. In terms of cumulative disruption of an established market,
a few technologies stand above all others, and AWS is among them.
Today, the march of progress continues unabated. In December 2017 at its annual
re:Invent conference in Las Vegas, Werner Vogels, CTO of Amazon, announced to more
than 40,000 attendees that the company had released 3,951 new features and services
since the first conference in 2012. AWS has an $18 billion annual run rate and 40% yearover-
year growth. Enterprises, startups, and governments alike have adopted the AWS
cloud en masse. The numbers are staggering, and AWS shows no signs of slowing down.
Needless to say, this growth and innovation comes at the expense of considerable
complexity. The AWS cloud is composed of scores of services and thousands of features,
enabling powerful new applications and highly efficient designs. But it is accompanied
by a brand-new lexicon with distinct architectural and technical best practices.
The platform can bewilder the neophyte. How does one know where to begin?
Amazon Web Services in Action, Second Edition, slices through the complexity of AWS
using examples and visuals to cement knowledge in the minds of readers. Andreas
and Michael focus on the most prominent services and features that users are most
likely to need. Code snippets are sprinkled throughout each chapter, reinforcing the
programmable nature of the cloud. And because many readers will be footing the bill
from AWS personally, any examples that incur charges are called out explicitly
throughout the text.
As a consultant, author, and at heart an engineer, I celebrate all efforts to introduce
the bewildering world of cloud computing to new users. Amazon Web Services in
Action, Second Edition is at the head of the pack as a confident, practical guide through
the maze of the industry’s leading cloud platform.
With this book as your sidekick, what will you build on the AWS cloud?
—BEN WHALEY, AWS COMMUNITY HERO AND AUTHOR
preface
When we started our career as software developers in 2008, we didn’t care about operations.
We wrote code, and someone else was responsible for deployment and operations.
There was a huge gap between software development and IT operations. On top
of that, releasing new features was a huge risk because it was impossible to test all the
changes to software and infrastructure manually. Every six months, when new features
needed to be deployed, we experienced a nightmare.
Time passed, and in 2012 we became responsible for a product: an online banking
platform. Our goal was to iterate quickly and to be able to release new features to the
product every week. Our software was responsible for managing money, so the quality
and security of the software and infrastructure was as important as the ability to innovate.
But the inflexible on-premises infrastructure and the outdated process of deploying
software made that goal impossible to reach. We started to look for a better way.
Our search led us to Amazon Web Services, which offered us a flexible and reliable
way to build and operate our applications. The possibility of automating every part of
our infrastructure was fascinating. Step by step, we dove into the different AWS services,
from virtual machines to distributed message queues. Being able to outsource tasks like
operating an SQL database or a load balancer saved us a lot of time. We invested this
time in automating testing and operations for our entire infrastructure.
Technical aspects weren’t the only things that changed during this transformation
to the cloud. After a while the software architecture changed from a monolithic application
to microservices, and the separation between software development and operations
disappeared. Instead we built our organization around the core principle of
DevOps: you build it, you run it.
We have worked as independent consultants since 2015, helping our clients get the
most out of AWS. We’ve accompanied startups, mid-sized companies, and enterprises
on their journey to the cloud. Besides designing and implementing cloud architectures
based on AWS services, we are focusing on infrastructure as code, continuous
deployment, Docker, serverless, security, and monitoring.
We enjoyed writing the first edition of our book in 2015. The astonishing support
from Manning and our MEAP readers allowed us to finish the whole book in only
nine months. Above all, it was a pleasure to observe you—our readers—using our
book to get started with AWS or deepen your knowledge.
AWS is innovating and constantly releases new features or whole new services.
Therefore, it was about time to update our book in 2017. We started to work on the
second edition of our book in June. Within six months we updated all chapters, added
three more chapters, and improved the book based on the feedback of our readers
and our editors.
We hope you enjoy the second edition of Amazon Web Services in Action as much as we do!