Transcript Designing and Delivering Scalable and Resilient Web Services Ron Jacobs Sr. Technical Evangelist, Microsoft http://blogs.msdn.com/rjacobs.

Designing and Delivering Scalable
and Resilient Web Services
Ron Jacobs
Sr. Technical Evangelist, Microsoft
http://blogs.msdn.com/rjacobs
Agenda
The Problem
A Possible Solution
New Possibilities
What to do next
Simple
Do the simplest thing
that will possibly work
Scalability
Able to support the
required quality of
service as the system
load increases
-Wiktionary
Typical Web Architecture
Users
• Need to get routed to same
machine (i.e. sticky sessions)
Web
Tier
• Each machine round trips for
data
• Data that is cached is stored
in the memory of one server
Data
Tier
• CPU and disk can get
saturated due to traffic
• Service access is slow
Web Explosion
Where did
my
IIS/ASP.NET
shopping
cart go? Application
Web
Site’s too
slow!!
IIS/ASP.NET
IIS/ASP.NET
Application
Application
Servers are
crashing
Database
Services
are slow
Database is
hot!!
Agenda
The Problem
A Possible Solution
New Possibilities
What to do next
Data Near Processing
Cache
Cache
Browser
Smart Client
Cache
Cache
Cache
Database
Good but…
• Cache is scoped to machine / process
– Machines die
– Processes recycle
• Cache memory is limited
What if?
• You could have as much cache as you wanted?
• You could share a giant cache across servers,
services and even clients?
• What if this was something you could simply
add to the platform for 1free?
1Some
features may require certain
editions of Windows Server
Windows Server AppFabric
AppFabric
HIGH AVAILABILITY
CACHING
WORKFLOW HOSTING
MONITORING
SERVICE HOSTING
SCALE OUT
MANAGEMENT
AppFabric Cache – Formerly known as Code Name “Velocity”
What is AppFabric Caching?
• An explicit, distributed, in-memory application cache for all
kinds of data
Caching clients
can be across
machines or
processes
Unified Cache View
Clients Access
the Cache as if it
was a large
single cache
Cache Layer
distributes data
across the
various cache
nodes
AppFabric Cache
Users
• No need for sticky sessions
Web
Tier
• Cached data available to
all servers
Caching
Tier
• Multiple machines means
scale and high-availability
• Lots of cache memory
Data
Tier
• Reduces load on database
• Duplicate Service Calls
eliminated
Data Distribution - Partitioned Cache
ASP.Net App
Caching Client
Caching Client
D I C
D E F
Cache Service
Cache Service
A B C
…
G A F
ASP.Net App
Web Tier
Caching Client
G H I
Cache Service
ASP.Net App
BH E
Cache
Tier
Scale on Data Size - More machines => More memory to cache
Scale on Cache Throughput - More machines => keys distributed across
more machines => better throughput
Scale Test Output
Load
1 Cache Server
As load increases,
throughput fails
to scale
latency increases
Throughput
Latency
Caching
Tier
Add Second Cache Server
Load
Throughput
Latency
Caching
Tier
Load Max
Throughput
increases
Latency
decreases
Add Third Cache Server
Load
Throughput
Latency
Caching
Tier
Associated Press
• Caches metadata and
news
• Serves 16 million hits
per day
• Increased the amount
of cached data 6
times.
System.Web.Cache
AppFabric DataCache
Deployment
Update
Web.config
<hosts>
<host name="BL1CDB8083714“
cachePort="22233"
cacheHostName="DistributedCacheService"/>
…..
Copy Client DLLs
…
.NET 3.5 SP1 OR .NET 4
Application
Application
Application
Caching Access Layer
Caching Access Layer
Caching Access Layer
</hosts>
.NET 4
Configuration Store
Caching Service
Caching Service
Install
Configure
AppFabric
Caching Service
<localCache isEnabled=“true" ../>
<security … />
Usage Pattern – Cache Aside
(Explicit Caching)
// Read from Cache
Toy toyObj = (Toy)
catalog.Get("toy-101");
Application
Caching Access Layer
Caching Service
// If Not present in the cache
if (toyObj == null)
{
// Read from backend..
toyObj = ReadFromDatabase();
// Populate Cache
catalog.Put("toy-101", toyObj);
return toyObj;
}
Database
Administration
• PowerShell cmdlets are used to administer the
cache cluster
• Rich set of cmdlets for
– Cache cluster management
– Cache creation and monitoring
Hello AppFabric Cache
Using PowerShell
Remember – PowerShell can also be
called from .NET Code!
AppFabric Cache
Codeplex Tool
http://mdcadmintool.codeplex.com/
Security
• Domain Based Security Option
– Domain Account / Local Account based Authentication
– Only authorized servers can join the cluster
– Only authorized clients can connect to the cluster
grant-cacheallowedclientaccount RedDomain\Machine1$
grant-cacheallowedclientaccount RedDomain\John
• Transport Level Security
– Turn on/off Signing or Encryption
• Can turn off Cache Security
– Use Firewalls, IPSec, VLANs to protect cache
AppFabric Caching Logical
Hierarchy
AppFabric
Caching
Service
AppFabric
Caching
Service
AppFabric
Caching
Service
AppFabric
Caching
Service
Named Cache :
Product Catalog
Named Cache :
Electronics Inventory
Regions
Key Payload
Region A
Tags
121 xxxx “Toy” “Child”
123 yyyy “Toy” “Chair”..
Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects
• Host
– Physical processes hosting AppFabric
Caching instance.
• Named Caches
– Can span across machines
– Defined in the configuration file
• Cache Item
– Key, Payload (Object ), Tags, TTL,
Timestamps, Version
• Regions
– Physically co-located Container of
Cache Items
– May be implicit or explicitly created
AppFabric Caching API
// Create instance of cachefactory (reads appconfig)
DataCacheFactory fac = new DataCacheFactory();
// Get a named cache from the factory
DataCache catalog = fac.GetCache("catalogcache");
// Simple Get/Put
catalog.Put("toy-101", new Toy("Puzzle", .,.));
// From the same or a different client
Toy toyObj = (Toy)catalog.Get("toy-101");
// Region based Get/Put
catalog.CreateRegion("toyRegion");
// Both toy and toyparts are put in the same region
catalog.Put("toy-101", new Toy( .,.), “toyRegion”);
Catalog.Put("toypart-100", new ToyParts(…), “toyRegion”);
Toy toyObj = (Toy)catalog.Get("toy-101“,"toyRegion");
Access APIs – Tagging Items
Tag hotItem
= new Tag("hotItem");
catalog.Put("toy-101", new Toy("Puzzle"),
new Tag[]{hotItem}, “toyRegion”);
catalog.Put("toy-102", new Toy("Bridge"), “toyRegion”);
// From the same or a different client
List<KeyValuePair<string, object>> toys =
catalog.GetAnyMatchingTag("toyRegion", hotItem);
Types of Data
Reference
Activity
Resource
Primary Read Only
Read-Write Not shared
Read-Write, Shared
Catalog Data
Shopping Cart
Auction Data/Seat
Assignment
Grocery Shop
Web Tier
Shopping Cart
Grocery Catalog
Grocery Inventory
Distributed Cache
Reference Data – Performance
• Catalog data doesn’t change often
• Unnecessary network cost to access from different machines
• Solution – Local Cache
Application
Application
Get(K2)
Put(K2, v3)
AppFabric Caching Client
Get(K2)
AppFabric Caching Client
Local Cache
K2, V2
Routing Table
Routing Table
Cache1
Cache2
Cache3
Primary for K1,V1
Primary for K2,V2
Primary for K3,V3
K1, V1
K2, V2
V3
K3, V3
Reference Data – Bulk Get
• Bulk Fetch from region
– 200-300k ops per second
– Fewer network calls
Catalog.BulkGet(
new List<string>(){“toy-101”, “toy-102”} ,
“toyRegion”);
Activity Data – Session Integration
Load Balance Requests
No more sticky routing
<sessionState mode="Custom“ customProvider="SessionStoreProvider">
<providers>
<add name="SessionStoreProvider"
type=“Microsoft.Data.Caching.DataCacheSessionStoreProvider,
Application
Application
Application
Caching
Access
Layer
Caching
Access Layer
Caching Access Layer
ClientLibrary“
cacheName="<YourNamedCache>"/>
</providers>
</sessionState>
…
Caching Service
Caching Service
Cache Service
Allows session state to be shared amongst multiple applications
Drop in AppFabric
Caching
SessionStoreProvider
Session State
stored in
AppFabric
Caching
Scale your
Session Store
Dynamically
Highly
Available
Activity Data - Availability
Application
Application
(K2, V2)
Get(K2)
PUT
AppFabric Caching Client
AppFabric Caching Client
Routing Table
Routing Table
Cache1
Cache2
Cache3
Primary for
Primary for
Primary for
K2, V2
Replication
Agent
K2, V2
K1, V1
K3, V3
K2, V2
Secondary for
Secondary for
K2, V2
K3, V3
Secondary for
K1, V1
Resource Data - Optimistic Locking
•
•
•
•
GetCacheItem returns a version object
Every update to an object internally increments it's version
Supply the version obtained along with the Put/Remove
Put/Remove will succeed only if the passed in version matches the
version in the cache
Version Based Update
Time
Client1
Client2 (Different Thread or process)
T0
CacheItem item =
catalog.GetCacheItem(“PlayerRegion”,
”Zune”);
CacheItem item =
catalog.GetCacheItem(“PlayerRegion”,
”Zune”);
T1
((ZuneObject)item.Object).inventory --;
((ZuneObject)item.Object).inventory--;
T2
T3
catalog.Put(“PlayerRegion”, “Zune”,
item.Object, item.Version);
catalog.Put(“PlayerRegion”, “Zune”,
item.Object, item.Version);
// Version mismatch
// Client must retry again
Two clients access the same
item
Both update the item
Second Client gets in first;
put succeeds because item
version matches; atomically
increments the version
First client tries put;
Fails because the versions
don’t match
Resource Data - Pessimistic
Locking
Client1:
GetAndLock ("k1")
Client2:
GetAndLock ("k1")
Client3:
Get ("k1")
GetAndLock gets
lock handle
Other GetAndLock
on same item fails
K1
Regular Get
succeeds
• Take locks on non-existent keys
– Allows you to co-ordinate calls for data
Data Race
GET
GET
GET
Lock Non-Existent Key
GET/LOCK
Caching Service
Cache Service
GET/LOCK
Cache Service
GET/LOCK
Caching Service
CALL
Cache Service
Cache Service
Composite Race
WAIT
WAIT
Caching Service
PUT
UNLOCK
Cache Service
Cache Service
Composite Race
GET
GET
Resource/Activity Data –
Tracking Changes
• Cache Event notifications
• Register on any client to notify changes
• Batched Notifications
DataCache.RegisterCacheLevelCallback(
int filter, DataCacheChangeCallback delegate);
DataCache.RegisterRegionLevelCallback(
String region, int filter, DataCacheChangeCallback delegate);
DataCache.RegisterKeyLevelCallback(
String region, String key, int filter, DataCacheChangeCallback delegate);
Scalable Notifications
Call Delegate
Store Last LSN
Register Notification for
Key “K3"
Application
Map Keys
to Partition
AppFabric Caching Client
Routing Table
Poll
Required Nodes
Partition: P2
Last LSN: 19
Nodes Return List of
Changes LSN Order
Cache1
Cache2
Cache3
Primary for
Primary for
Primary for
Change Log
Change Log
33 Add K1
34 Del K22
K1, V1
Change Log
Partition P1
1 Add K2
2 Del K32
K2, V2
(Partition P2)
18 Del K32
19 Del K43
K3, V3
Agenda
The Problem
A Possible Solution
New Possibilities
What to do next
Pre-Fetch
Hospital
Data Center
Pre-Fetch
Remote Clinic
Hospital
Slow!!
Data Center
WAN
Pre-Fetch
Hospital
Cache Service
Remote Clinic
Data Center
WAN
Agenda
The Problem
A Possible Solution
New Possibilities
What to do next
Web Platform Installer
Select Enterprise
Install AppFabric
Install AppFabric
endpoint.tv
AppFabric on MSDN
http://msdn.microsoft.com/AppFabric