Transcript Windows Server AppFabric Caching Velocity Users Load Balancer Sticky Routing Application / Web Tier Application Application Application ASP.Net App Session Cart Hosted in memory Database Data Tier Catalog sits in Database.

Windows Server AppFabric Caching
Velocity
Users
Load Balancer
Sticky Routing
Application
/ Web Tier
Application
Application
Application
ASP.Net App
Session Cart
Hosted in
memory
Database
Data Tier
Catalog sits in
Database
Flu Hits!
Web
Site’s tooUsers
slow!!
Where did
my
shopping
cart go?Application
Application
…
Web Tier
(ASP.Net)
Application
Servers are
crashing
Data Tier
Database
Database is
hot!!
Windows Server
AppFabric Caching
can help
>
>
>
>
>
>
>
>
>
>
>
Caching clients
can be across
machines or
processes
Unified Cache View
Clients Access
the Cache as if it
was a large
single cache
Cache Layer
distributes data
across the
various cache
nodes
Users
Application
Caching Access
Layer
Caching Access
Layer
Caching
Service
Caching
Service
Database
…
Application
Caching Access
Layer
Web Tier
(ASP.Net)
Data Tier
Cloud
Caching
Service
Application
Cache Tier
>
>
>
>
>
>
>
>
>
Oper
Throughput
Latency
Bulk Fetch
Read
28, 000/ sec (2k)
1-2ms
200k/sec (512 bytes)
Write
20,000 / sec (2k)
3 ms
-
Operation
Servers
Throughput
Read 2k
1
28,000 ops/ sec
Read 2k
2
52, 500 ops/ sec
Read 2k
3
78, 000 ops/ sec
5. Proven Technology
Associated Press Custom News live on CTP3
>
>
>
5. Proven Technology Contd..
> Dell.com
>
>
>
>
>
6. Do all this at low cost!
CACHING
>
>
>
>
>
>
>
>
>
>
CTP1
CTP2
CTP3
Beta
June ‘08
RTM
PDC ‘08
Mar ‘09
• PDC
• Nov ‘09
2010 Wave 1
Azure
CTP
* CY10
Post RTM
>
>
>
>
>
>
>
>
Users
Copy Client DLLs and
add app.config
<hosts>
<host name="BL1CDB8083714“
cachePort="22233"
cacheHostName="DistributedCacheService"/>
…..
…
Application
Caching Access Layer
Application
Application
Caching Access Layer
Caching Access Layer
</hosts>
Caching Service
Caching Service
Enable
Application
Server Cache
Role
Caching Service
<localCache isEnabled=“true" ../>
<security … />
Web Tier
Cache Tier
Configuration Store
(Can be database, File share, etc.)
Stores Global Cache Policies
Stores Current Partitioning Information
(new in Beta)
>
>
>
>
grant-cacheallowedclientaccount
grant-cacheallowedclientaccount
>
>
>
>
RedDomain\Machine1$
RedDomain\John
>
(new in Beta)
>
>
>
// Create instance of cachefactory (reads appconfig)
// Get a named cache from the factory
// Simple Get/Put
catalog.Put("toy-101", new Toy("Puzzle", .,.));
// From the same or a different client
Toy toyObj = (Toy)catalog.Get("toy-101");
// Region based Get/Put
catalog.CreateRegion("toyRegion");
// Both toy and toyparts are put in the same region
catalog.Put("toy-101", new Toy( .,.), “toyRegion”);
Catalog.Put("toypart-100", new ToyParts(…), “toyRegion”);
Toy toyObj = (Toy)catalog.Get("toy-101“,"toyRegion");
AppFabric
Caching
Service
AppFabric
Caching
Service
AppFabric
Caching
Service
Named Cache :
Product Catalog
Named Cache :
Electronics Inventory
Regions
AppFabric
Caching
Service
Key Payload
Region A
Tags
121 xxxx “Toy” “Child”
123 yyyy “Toy” “Chair”..
Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects
>
>
>
>
>
>
>
>
>
>
> Add Tags to Items
(New in Beta)
>
Tag hotItem
= new Tag("hotItem");
catalog.Put("toy-101", new Toy("Puzzle"),
new Tag[]{hotItem}, “toyRegion”);
catalog.Put("toy-102", new Toy("Bridge"), “toyRegion”);
// From the same or a different client
List<KeyValuePair<string, object>> toys =
catalog.GetAnyMatchingTag("toyRegion", hotItem);
// Read from Cache
Toy toyObj = (Toy)
catalog.Get("toy-101");
Application
Caching Access Layer
Caching Service
// If Not present in the cache
if (toyObj == null)
{
// Read from backend..
toyObj = ReadFromDatabase();
// Populate Cache
catalog.Put("toy-101", toyObj);
return toyObj;
}
Database
>
>
>
>
>
>
>
>
>
Users
ASP.Net App
Caching Client
Caching Client
D I C
D E F
Cache Service
Cache Service
A B C
…
G A F
ASP.Net App
Web Tier
Caching Client
G H I
Cache Service
ASP.Net App
BH E
Cache
Tier
Scale on Data Size - More machines => More memory to cache
Scale on Cache Throughput - More machines => keys distributed
across more machines => better throughput
Server 2 Added
Throughput Increases
Latency Decreases
Until Server Saturation
Single Server
Throughput Increases
with Increasing Load
Until Server Saturation
Server 3 Added
Throughput Increases
Latency Decreases
Load
Throughput
Latency
16 KB objects, Response
Time, sec
Throughput Test
Increase Load vs Requests / sec
5000
4000
0.5
3000
Without Velocity
2000
With Velocity
0.4
1000
0
00:00
05:00
10:00
15:00
20:00
0.3
Time
Scale Test
# of Nodes vs Throughput
5000
4000
0.2
3000
0.1
2000
1000
0
0
1
2
3
4
5
6
0
Nodes
http://blogs.msdn.com/velocity/
http://www.griddynamics.com/velocity
1000
2000
Requests / sec
3000
4000
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Reference
Activity
Resource
Primary Read Only
Read-Write Not shared
Read-Write, Shared
Catalog Data
Shopping Cart
Auction Data/Seat
Assignment
Grocery Shop
Web Tier
Shopping Cart
Grocery Catalog
Distributed Cache
Grocery Inventory
>
>
>
Application
Application
Get(K2)
Get(K2)
Put(K2, v3)
AppFabric Caching Client
AppFabric Caching Client
Local Cache
K2, V2
Routing Table
Routing Table
Cache1
Cache2
Cache3
Primary for K1,V1
Primary for K2,V2
Primary for K3,V3
K1, V1
K2, V2
V3
K3, V3
> Enumerate all items in Region
> Bulk Fetch from region (New in Beta)
> 200-300k ops per second
Catalog.BulkGet(
new List<string>(){“toy-101”, “toy-102”} ,
“toyRegion”);
Load Balance Requests
No more sticky routing
<sessionState mode="Custom“ customProvider="SessionStoreProvider">
<providers>
<add name="SessionStoreProvider"
Application
Application
Application
type=“Microsoft.Data.Caching.DataCacheSessionStoreProvider,
Caching
Access
Layer
Caching
Access Layer
Caching Access Layer
ClientLibrary“
cacheName="<YourNamedCache>"/>
</providers>
</sessionState>
…
Caching Service
Caching Service
Cache Service
New in Beta – Allow session state to be shared amongst multiple
applications
Drop in AppFabric
Caching
SessionStoreProvider
Session State
stored in
AppFabric
Caching
Scale your
Session Store
Dynamically
Highly
Available
Application
Application
(K2, V2)
Get(K2)
PUT
AppFabric Caching Client
AppFabric Caching Client
Routing Table
Routing Table
Cache1
Cache2
Cache3
Primary for
Primary for
Primary for
K2, V2
Replication
Agent
K1, V1
K3, V3
K2, V2
Secondary for
Secondary for
K2, V2
K3, V3
Secondary for
K1, V1
>
>
>
>
Version Based Update
Time
Client1
Client2 (Different Thread or process)
T0
CacheItem item =
catalog.GetCacheItem(“PlayerRegion”,
”Zune”);
CacheItem item =
catalog.GetCacheItem(“PlayerRegion”,
”Zune”);
T1
((ZuneObject)item.Object).inventory -;
((ZuneObject)item.Object).inventory--;
T2
T3
catalog.Put(“PlayerRegion”, “Zune”,
item.Object, item.Version);
catalog.Put(“PlayerRegion”, “Zune”,
item.Object, item.Version);
// Version mismatch
// Client must retry again
Two clients access the same
item
Both update the item
Second Client gets in first;
put succeeds because item
version matches; atomically
increments the version
First client tries put;
Fails because the versions
don’t match
Client1:
GetAndLock ("k1")
Client2:
GetAndLock ("k1")
Client3:
Get ("k1")
GetAndLock gets
lock handle
Other GetAndLock
on same item fails
K1
Regular Get
succeeds
> New in Beta – Take locks on non-existent keys
> Allows you to co-ordinate creating new
object amongst multiple clients
>
>
>
(New in Beta)
DataCache.RegisterCacheLevelCallback(
int filter, DataCacheChangeCallback delegate);
DataCache.RegisterRegionLevelCallback(
String region, int filter, DataCacheChangeCallback delegate);
DataCache.RegisterKeyLevelCallback(
String region, String key, int filter, DataCacheChangeCallback delegate);
Register
Notification for
Key “K3"
Call Delegate
Store Last LSN
Application
Map Keys
to Partition
(say P2)
AppFabric Caching Client
Routing Table
Poll
Required
Nodes
Partition: P2
Last LSN: 19
Nodes Return
List of Changes
LSN Order
Cache1
Cache2
Cache3
Primary for
Primary for
Primary for
Change Log
Change Log
33 Add K1
34 Del K22
K1, V1
Change Log
Partition P1
1 Add K2
2 Del K32
K2, V2
(Partition P2)
18 Del K32
19 Del K43
K3, V3
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Solution >
>
>
>>FUTURE
Application
Caching Service
Caching
Worker Role
Caching Service
AppFabric Caching
Access Layer
AppFabric Caching
Access Layer
…
Application
AppFabric Caching
Access Layer
Caching Service
Application
ASP.Net Web
Tier
Application &
AppFabric
Caching
deployed Onpremise
Data on SQL Azure
>>FUTURE
Application
Caching Service
Caching
Worker Role
Caching Service
AppFabric Caching
Access Layer
AppFabric Caching
Access Layer
…
Application
AppFabric Caching
Access Layer
Caching Service
Application
Web Role
Application &
AppFabric Caching
on Windows Azure
AppFabric Caching
Worker Role
Data on SQL Azure
>>FUTURE
Application
Caching Service
Caching
Worker Role
AppFabric
Service
Bus
Caching Service
AppFabric Caching
Access Layer
AppFabric Caching
Access Layer
…
Application
AppFabric Caching
Access Layer
Caching Service
Application
Web Role
Application &
AppFabric Caching
on Windows Azure
AppFabric Caching
Worker Role
Data on-premises
>
>
>
>
>
>
>
>
>>FUTURE
>
>
>
>
>
>
>
>
>>FUTURE
>
>
>
>
>
>
>
>
>
>
>>FUTURE
Job
Input
Split
Method
Keys
Calculation
Operation
AppFabric
Caching
Intermediate
Store
Final
Results
Market
Data
Market Data
Central Market
Data Store
(~1 TB Tick Data)
Final Results
Store
AppFabric
Caching
Scratch
Rollup
Operation
Scratch
GridDynamics testing
on CTP1
50 times improvement
in some cases
>
>
>
>
>
>
>
Web
Scenarios
• Distributed/Global
object cache
• Low latency
access
• High scale
• Availability
• Cache for reference
and activity data
• Scale IIS/ASP.Net
applications
Enterprise /
HPC
Scenarios
• Persistence
• LINQ
enabled cache
• Integrate with
HPC server
• Heterogeneous
client support
• Co-locate
compute and data
Software +
Services
Scenarios
• Application Cache
for Windows Azure
• More data services
• BI, Streaming,
Reporting
• OData (REST API)
>
>
>
>
>
>
>
>
http://msdn.microsoft.com/appfabric
>
>
channel9.msdn.com/learn
Built by Developers for Developers….
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market
conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.