Transcript COS304 Agenda  Windows Azure Storage  Blobs, Drives, Tables, Queues   Cloud Storage System  Scalable  Durable  Highly Available  Pay for what.

COS304
Agenda
 Windows Azure Storage
 Blobs, Drives, Tables, Queues

 Cloud Storage System
 Scalable
 Durable
 Highly Available
 Pay for what you use
 Blobs
 Drives
 Tables – Massively scalable structured storage.
 Queues
Container
Blobs
https://<account>.blob.core.windows.net/<container>
Account
Table
Entities
https://<account>.table.core.windows.net/<table>
Queue
Messages
https://<account>.queue.core.windows.net/<queue>
blocks
Local file
Block 1
Block 2
Block
3
Cloud blob
Block 5
pages
5120
0
0
0
0
3072
1024
2048
5120
4096
3072
5120
• Goals
• By default – authentication required
• Public containers – everyone has read access
• SAS – share url with subset without sharing your key!
ID=1
A
ID=1
A
ID=2
BB
ID=2
BB
Base blob = alphabets.txt
#1 snapshot=2011-04-10T19:26:24.8690267Z
ID=1
A
ID=2
BB
#1 snapshot=2011-0410T19:26:24.8690267Z
ID=1
A
ID=1
A
ID=2
BB
ID=2
BB
ID=3
CCC
ID=3
CCC
Base blob = alphabets.txt
Base blob = alphabets.txt
#2 snapshot=2011-0510T19:26:24.8690267Z
ID=1
A
ID=2
BB
#1 snapshot=2011-0410T19:26:24.8690267Z
ID=1
A
ID=1
A
ID=2
BB
ID=2
BB
ID=3
CCC
ID=3
CCC
Base blob = alphabets.txt
ID=1
A
ID=2
BB
ID=3
CCC
#2 snapshot=2011-0510T19:26:24.8690267Z
#3 snapshot=2011-0510T19:28:24.8690267Z
Agenda


 Partitioning & Scalability

Single Blob Partition
• Throughput up to 60 MB/s
Single Queue/Table Partition
• Up to 500 transactions (entities or messages) per second
Storage Account
•
•
•
•
SLA – 99.9% Availability
Capacity – Up to 100 TBs
Transactions – Up to 5000 entities per second
Bandwidth – Up to 3 gigabits per second
 Distribute traffic to multiple partitions
 Use multiple storage accounts
 When limit is hit, app may see frequent ‘503 server busy’;
applications should implement exponential back-off
Storage Partitioning – How to Scale?
Understanding partitioning is key to understanding scale
Every data object has a
partition key
Partition key is unit of scale
System load balances
Server Busy
• Different for each data type (blobs, entities, queues)
• A partition can be served by a single server
• System load balances partitions based on traffic
• Load balancing can take a few minutes to kick in
• Can take a couple of seconds for partition to be available on a
different server
• Our system load balances to meet your traffic needs
• Single partition limits have been reached
• Use exponential back-off on “Server Busy”
VIP
Legend
- Partition
- Server Load
Distributed File System
Partition Keys In Each Abstraction
Blobs – Container name + Blob name
Container Name
Blob Name
image
annarbor/bighouse.jpg
image
annarbor/bighouse.jpg
image
annarbor/denard.jpg
backup
annarbor/bighouse.jpg
• Every blob and its snapshots are in
a single partition
Snapshot
2009-12-03T15:26:19.4466877Z
Partition Keys In Each Abstraction
Entities –
TableName + PartitionKey
• Entities with same PartitionKey value are
served from same partition
Table Name
PartitionKey
RowKey
Zipcode
City
CustomerOrder
Alaska
Tina Fey
99501
Anchorage
CustomerOrder
Alaska
Sarah Palin
99501
Anchorage
CustomerOrder
Washington
Bill Johnson
98053
Redmond
Customers
Washington
Bill Johnson
98053
Redmond
Partition Keys In Each Abstraction
Messages –
Queue Name
• All messages for a single queue belong
to the same partition
Queue
Message
jobs
Message1
jobs
Message2
workflow
Message1
12000
11000
10000
Entities / Sec
9000
8000
Successful
Inserts (SP)
7000
6000
5000
4000
Successful
Inserts (MP)
3000
2000
1000
0
1
2
5
10
Extra Large VMs (15 Threads per VM)
16
12000
11000
10000
Entities / Sec
9000
8000
7000
Successful Gets
(SP)
6000
5000
4000
Successful Gets
(MP)
3000
2000
1000
0
1
2
5
10
Extra Large VMs (15 Threads per VM)
16




Blobs
Drives
Tables
Queues
http://blogs.msdn.com/windowsazurestorage/
Blue Section
http://www.microsoft.com/cloud/
http://www.microsoft.com/privatecloud/
http://www.microsoft.com/windowsserver/
http://www.microsoft.com/windowsazure/
http://www.microsoft.com/systemcenter/
http://www.microsoft.com/forefront/
http://northamerica.msteched.com
www.microsoft.com/teched
www.microsoft.com/learning
http://microsoft.com/technet
http://microsoft.com/msdn