Transcript KEY VALUE STORAGE IN THE CLOUD AGENDA • review cloud storage • review providers • review differences between storage clouds • jclouds.

KEY VALUE STORAGE IN THE CLOUD
AGENDA
•
review cloud storage
•
review providers
•
review differences between storage clouds
•
jclouds
review cloud storage
what is key value storage?
•
global name space
•
key, value with metadata
•
http accessible
•
sites on demand
•
unlimited scaling
key value storage concepts
•
GLOBAL NAME SPACE
•
organizes and sets
boundaries for content
•
called a “bucket” or
“container”
•
KEY/ VALUE
•
called objects, blobs, or files
•
unstructured content
•
can be small or large
•
can have metadata
(key, value pairs)
review vendors
THE VENDORS
QuickTime™ and a
BMP decompressor
are needed to see this picture.
QuickT i me™ and a
BMP decom pressor
are needed to see this picture.
review differences between
storage clouds
FILE SIZE
Can you store 5 GB or unlimited?
UPLOAD LARGE FILES
One shot or chunked?
DOWNLOAD LARGE FILES
One shot or chunked?
CONTENT DELIVERY NETWORK
Cache content close to users?
GLOBAL PRESENCE
Where do you write? What level of redundancy?
SLA
How many 9’s?
Maintenance notification
Replication count
Consistency Model
Immediate or Eventual?
AUTHORIZATION
SUB-ACCOUNTS AND QUOTAS
ACCESS CONTROLS
PROTOCOLS
•
ReST
•
SOAP
•
WebDAV
•
CIFS
•
NFS
pricing
•
PARKING FEES
•
TEXT MESSAGING
•
AIR TIME
•
SUPPORT
Value Adds
Promotions with other services and tools
api complexity
How difficult is it to use?
code and sign the http request
QuickT i me™ and a
BMP decom pressor
are needed to see this picture.
PUT /sushi.avi HTTP/1.1 Host:
adriansmovies.s3.amazonaws.comConten
t-Length: 734859264Date: Wed, 01 Mar
2006 12:00:00 GMTAuthorization:
signature x-amz-meta-Chef: Kawasaki
PUT /adriansmovies/sushi.avi HTTP/1.1 Host:
<account>.blob.core.windows.netContentLength: 734859264
Date: Wed, 01 Mar 2006 12:00:00
GMTAuthorization: SharedKey <app>:signature
x-ms-meta-Chef: Kawasaki
QuickTime™ and a
BMP decompressor
are needed to see this picture.
PUT /<api
version>/<account>/adriansmovies/sus
hi.avi HTTP/1.1 Host:
storage.clouddrive.com TransferEncoding: chunkedX-Auth-Token:
session-token X-Object-Meta-Chef:
Kawasaki
POST /namespace/adriansmovies/sushi.avi HTTP/1.1
Content-Length: 734859264Date: Wed, 01 Mar 2006
12:00:00 GMT
x-emc-uid: <uid> x-emc-signature: signature x-emcmeta: Chef=Kawasaki
CODE AND SIGN THE HTTP
REQUEST
GET /ws/IMFS/GetStorageNodeExtended.ashx?&fileOverwrite=true&ipRestricted=true&destFolderPath= adriansmovies&sizeBytes=
734859264&firstByteExpiration=6000&lastByteExpiration=259200&sessionToken=session-token HTTP/1.1
POST /Upload.ashx?uploadToken=from_above&destFolderPath=adriansmovies HTTP/1.1
Host: from_above
Content-Length: 734859382
Content-Type=multipart/form-data; boundary=--jclouds-Authorization=Basic GpjbG9=
----jclouds-Content-Disposition: form-data; name="sushi.avi"; filename="sushi.avi"
Content-Type: application/octetstring
...
PUT /ws/Metadata/SetMetadata.ashx?&path=Folders/adriansmovies/sushi.avi&sessionToken=session-token&metadata=Chef:Kawasaki HTTP/1.1
CODE AND SIGN THE HTTP
REQUEST
POST /<api version>/containers/id_of_ adriansmovies/contents HTTP/1.1
Content-Length: 734859382
Content-Type=multipart/form-data; boundary=--jclouds-Authorization=Basic GpjbG9=
----jclouds-Content-Disposition: form-data; name="sushi.avi"; filename="sushi.avi"
Content-Type: application/octetstring
...
PUT /<api version>/files/from_above/metadata/Chef HTTP/1.1 Content-Length:
8
Content-Type: text/plain
Authorization: Basic GpjbG9=
Kawasaki
do you want to
•
Deal with Errors
•
Deal with Concurrency
•
Deal with Cloud Complexity
enterprise grade tooling is
critical
•
JCLOUDS BLOBSTORE
•
open source
•
feels like java
•
portability between clouds
•
deal with web complexity
•
unit testability
•
thread-safe and scalable
jclouds-blobstore
•
No HTTP, weird URLs, or complicated authorization
•
Portable atomic commands for blobs and containers.
•
Easy to switch clouds (or stub!)
BlobStoreContext context = S3ContextFactory.createContext(account, key);
Map<String, InputStream> map = context.createInputStreamMap("adriansmovies");
InputStream myphoto = map.get(“sushi.avi”);
BlobMap blobMap = context.createBlobMap("adriansmovies");
BlobMetadata sushiMetadata = map.get(“sushi.avi”).getMetadata();
String chef = sushiMetadata.getUserMetadata(“chef”);
String contentType = sushiMetadata.getContentType();
context.close();
architecture
questions?
•
http://www.jclouds.org
QuickTime™ and a
BMP decompressor
are needed to see this picture.
DEMO
•
Goals
•
Show multi-cloud
•
Show Google AppEngine support
•
Show something cool
DEMO architecture
Lets try it out
•
Send a tweet @jclouds
•
Wait a minute
•
Check to see if it landed in the cloud
•
http://jclouds-tweetstore.appspot.com/tweets/get
what’s next?
•
•
COMPUTE
•
VCloud Express, EC2 and
GoGrid Support
•
Create Abstraction Layer via
Dasein
STORAGE
•
Content Delivery Network
•
Eucalyptus Support
appendix
other notes of interest
feature and pricing
comparison