Transcript Web Scale Computing
Webscale Computing
Mike Culver
Amazon Web Services
Amazon Has Three Parts 1 3 2
Ideas Meet Pagers
What if Gravity Sets In?
How Do You Survive This?
Don’t Do the Muck
What is Cloud Computing?
o Remote computing capacity o On-demand o Infinite scale o XML Web services
Fallacies of Distributed Computing
1. The network is reliable. 2. Latency is zero. 3. Bandwidth is infinite. 4. The network is secure. 5. Topology doesn't change. 6. There is one administrator. 7. Transport cost is zero. 8. The network is homogeneous.
Wikipedia.com
Bandwidth Consumed by Amazon Web Services Bandwidth Consumed by Amazon’s Global Websites 2001 2002 2003 2004 2005 2006 2007 2008
Amazon Web Services Are...
Building block services that allow developers to innovate and make money
Infrastructure As a Service Amazon Simple Storage Service Amazon Elastic Compute Cloud Amazon Simple Queue Service Amazon SimpleDB Commerce As a Service Amazon Flexible Payments Service Fulfillment Web Service People As a Service Amazon Mechanical Turk Alexa Web Services Alexa Web Information Service Alexa Top Sites Alexa Site Thumbnail Alexa Web Search Platform
Amazon Simple Storage Service (Amazon S3)
Highly scalable data storage in-the-cloud Programmatic access via web services API Simple to get going, simple to use Highly available and durable Pay-as-you-go: Storage: $0.15 / GB / month Data Transfer: starts at $0.17 / GB Requests: nominal charges
Billions of Objects Stored 14 Billion 18 Billion 22 Billion 10 Billion .8 Billion Q3 06 5 Billion Q1 07 Q2 07 Q4 07 Q1 08 Q2 08
Amazon S3 Namespace object bucket Amazon S3 object bucket bucket object object object object
Amazon S3 Namespace mculver-images Amazon S3 media.mydomain.com
Beach.jpg
2005/party/hat.jpg
img1.jpg
img2.jpg
public.blueorigin.com
index.html
img/pic1.jpg
Internet Backup on a Stick Or Stick Backed Up on the Net?
Open Source Backup
1 TB: Basecamp and Campfire Tools
Amazon Elastic Compute Cloud (Amazon EC2)
Resizable compute capacity in the cloud Obtain and boot new server instances in minutes Quickly scale capacity, up or down, as your computing requirements change Full root access to a blank Linux machine Simple Web service management interface Changes the economics of computing
Predictions Cost Money Infrastructure Cost $ Large Capital Expenditure Opportunity Cost You just lost customers Predicted Demand Traditional Hardware Actual Demand Automated Virtualization time
Resilient Infrastructure Requires Physical Isolation
Availability Zone A Availability Zone B Availability Zone C
Three Flavors of Amazon Machine Images
Public AMIs:
Use pre-configured, template AMIs to get up and running immediately. Choose from Fedora, Movable Type, Ubuntu configurations, and more
Private AMIs:
Create an Amazon Machine Image (AMI) containing your applications, libraries, data and associated configuration settings
Paid AMIs:
Set a price for your AMI and let others purchase and use it (Single payment and/or per hour)
Virtual Machine Choices
Small Bits RAM Disk EC2 Compute Units I/O Performa nce Firewall Per Hour 32 1.7 GB 160 GB 1 Medium Yes $0.10
Standard Large 64 7.5 GB 850 GB 4 Extra Large 64 15 GB 1690 GB 8 High Yes $0.40
High Yes $0.80
High CPU Medium Extra Large 32 1.7 GB 350 GB 5 64 7 GB 1690 GB 20 High Yes $0.20
High Yes $0.80
$5 or $30,000 For a Prototype?
$30,000 worth of in-house servers DISA paid a total of $5
Build Your Own Virtual ISP
Operating Systems as a Service
4 TB Data 100 Nodes 11 Million PDFs 100 instances x 24 hours x $0.10 / Hr = $240
From 50 to 3400 Instances in 3 Days
Amazon Simple Queue Service (Amazon SQS)
A distributed queue in the cloud Used for storing messages traveling between computers Reliable Runs within Amazon's high-availability data centers Messages are stored redundantly across multiple servers and locations Scalable to millions of messages a day Simple: Only 6 methods Platform agnostic Provides access control and message locking
Amazon SimpleDB
Limited Beta
Databases Should Just Work But They’d Better Scale…
Easy: no administration Web Service API Flexible (no schemas) Scalable: create new domains as Your data grows Request throughput increases Durable Multiple nodes Replicated data centers
Amazon SimpleDB item
123 456 789
description
Sweater Dress shirt Shoes
color
Blue, Red White, Blue Black
material
Leather PUT (item, 123), (description, Sweater), (color, Blue), (color, Red) PUT (item, 456), (description, Dress shirt), (color, White), (color, Blue) PUT (item, 789), (description, Shoes), (color, Black), (material, Leather)
Query
Domain = MyStore [‘description’ = ‘Sweater’]
Learn More About AWS
Explore http://aws.amazon.com
Read our blog at http://aws.typepad.com
Email me at [email protected]