Transcript BB06  Abolade Gbadegesin Principal Software Architect Microsoft Corporation Mesh Services User-centric data and services with more than 460M Users Users Identity Devices Directory Applications Storage Synchronization Communications and Presence Search & Geospatial.

BB06
 Abolade Gbadegesin
Principal Software Architect
Microsoft Corporation
Mesh
Services
User-centric data and services with more than 460M Users
Users
Identity
Devices
Directory
Applications
Storage
Synchronization
Communications
and Presence
Search &
Geospatial








Accounts


User-data storage



Communications
& Presence



Notifications


Conceptual elements
builds on Live ID authentication to
capture user-device relationships
provisioned during web sign-up
1
identity
device
claim
2
device
provisioned during software install
3
Resources, collections and links
2
collection
Uri
EntityTag
SubscriptionUri
…
resource
resource
1
scope for grouping resources
and tracking changes
3
represented as feeds and items
Uri
LastUpdateTime
…
metadata common to all resources
Uri
Label
…
link
resource
4
primitive for representing &
navigating relationships
Conceptual view and building blocks
Live ID Service
HTTP
Live ID
Client Runtime Library
Mesh Service
Live Mesh Client
Accounts
app
HTTP
users, devices, apps
Accounts
(local cache)
Conceptual elements
1
items in data feed
represent files
mesh object
local
file
system
mesh
data
feed
3
data feed mapping
2
‘maps’ data feed to a file system location
mesh object defines
permission scope for
sharing feeds
Enabling bidirectional feed synchronization
<rss version="2.0">
<channel>
<title>To Do List</title>
<description>A list of items to do</description>
<link> http://example.com/partial.xml </link>
<item>
<title>Buy groceries</title>
<description>Get milk, ... </description>
<sx:sync id="item_1_myapp_2005-05-21T11:43:33Z" updates="3">
<sx:history sequence="1" when="2005-05-21T09:43:33Z" by="a9"/>
<sx:conflicts ...>
<item>
...
</item>
</sx:conflicts>
</sx:sync>
</item>
</channel>
</rss>
version vector and conflicts for each feed item
Conceptual view and building blocks
Mesh Service
Live fx Client
User-data storage
010010
010100
111100
HTTP, FeedSync
User-data storage
(local replica)
feeds, content,
permissions
Mesh folders app
handles mesh folders
Scalable storage
local
file
system
Mesh Service
Live ID
Service
Accounts
app
Notifications
Messenger
Relay
Service
Mesh UX
Live fx Client
Mesh
Web Desktop
Communications
& Presence
Notifications
User-data storage
010
101
Communications
& Presence
Mesh
Remote Desktop
Scalable storage
Services fabric
Accounts
User-data storage
Local file system
Mesh Folders
Microsoft Update
Live ID
Client
Runtime
Library
Messenger
NAT
Traversal
Getting from concepts to implementation




Workload analysis: Startup and file updates
Live fx Client
Mesh Accounts
Startup
update local cache
subscribe for change notifications
synchronize local replicas
subscribe for change notifications
File
change synchronize feed
upload updated files
Mesh User-Data Storage
Workload analysis: Content replication
Live fx Client
File
change
continued
Mesh Notifications
Mesh User-Data Storage
fire change notifications
deliver change notifications
Content
replication synchronize local replica
query availability of updated files
Mesh Communications
and Presence
query availability of peers
fetch updated files from peers
relay or direct connection
Scale-out strategy





Partitioning resources
each resource lives in a ‘home’ datacenter
each resource has a stable canonical
URI based on its resource identifier
mapping a resource identifier to its ‘home’
datacenter requires a lookup in a replicated database
https://accounts.mesh.com/Identities/FKEXBSLJR3VUXEXMTPSSOVO6GM/DeviceClaims/CSOW2TEBYUSE5BD3I5QOS724UU
https://accounts.clus9.bay.mesh.com/Identities/FKEXBSLJR3VUXEXMTPSSOVO6GM/DeviceClaims/CSOW2TEBYUSE5BD3I5QOS724UU
each resource has a location URI that routes
requests directly to its ‘home’ datacenter
requests to the ‘wrong’ datacenter get redirected,
and clients cache the redirect URI for future use
General service design principles






Front
end role
Back
end role
Building
block
Infrastructure and roles
Account APIs
User-Data Storage
front end
roles
Content upload &
download
Web Desktop
commodity hardware
xcopy deployment
automated rollback
self-monitoring
fail-fast recovery
self-healing
Scalable storage
Services
fabric
partitioning units
Services fabric SKU:
2P Dual Core
8GB RAM
4x 500GB SATA II HDD
Windows Server
Device presence
store
Subscription store
back end
roles
Notification
queue store
Dictionary state
store
Partitioning &
Recovery Manager
Watchdogs
infrastructure
roles
scale
unit
Front
end role
Back
end role
Building
block
Scale unit layout
soft state stores
reliable
state
stores
Accounts
Scalable
Storage
Dictionary
state store
Web
Desktop
Accounts
API
Partitioning &
Recovery
Manager
User-Data
Structured
Storage
Device presence
store
Notification
queue store
soft
state
stores
Communications
& Presence
Notifications
API
User-Data
Storage API
User-Data
Blob
Storage
Content
upload &
download
Services fabric
Live ID
Messenger Relay
Hotmail
building
block
services
Watchdogs
Subscription store
Soft state partitioning and recovery
hash nodes & resources
to the same output space
Node 1
Node 2
node ‘nearest’ a resource
is responsible for it
{ECCA8793-3208-48A9-A1A1-038D0AD704C5}
/MeshObjects/FJ3GZXUNDC4EPPPPUMQ2QNCG6I/Subscriptions/GLW6BC6HRWFU5LEW7HTX5FPC3Y
consistent hashing
node loss tracking
resource
owner
nodes
Device presence
store
Partitioning &
Recovery Manager
Notification
queue store
Owner library
Owner library
Lookup library
Lookup library
Communications
and Presence
owner library obtains leases
lookup library queries assignments
Notifications API
resource
client
nodes
Common software stack
multiple processes
Mesh
service code
Building
block code
mesh service process
Mesh services
runtime
Mesh service application
singleton process
Tracing &
log
collection
Services
fabric
controller
AP configuration
Mesh services host process
NETFX 3.5 SP1
SMB.sys
Windows Server
Front end software stack
multiple processes
Mesh
service code
Building
block code
mesh service process
Mesh services
runtime
Mesh resource
handlers
Accounts
Mesh
content
handlers
singleton process
TLS/TCP
handlers
Tracing &
log
collection
Notifications
Soft state lookup
Services
fabric
controller
Scalable storage
client
Subscriptions
AP configuration
Communications
& Presence
User-data
storage
User-data
content
upload &
download
HTTP request dispatcher
TCP
notification
streaming
TLS/TCP
stream
dispatcher
Mesh services host process
WCF
syndication
NETFX 3.5 SP1
HTTP.sys
Windows Server
sockets
SMB.sys
Back end software stack
multiple process
Mesh
service code
Building
block code
mesh service process
Mesh services
runtime
Mesh resource managers
Soft state
owner/lookup
Notification queue manager
Device presence manager
singleton process
Tracing &
log
collection
Dictionary store manager
Services
fabric
controller
Scalable storage
client
Subscription manager
Subscriptions
Partitioning and recovery manager
Work queues
AP configuration
Sockets dispatcher
Mesh services host process
NETFX 3.5 SP1
sockets
Windows Server
SMB.sys
Experiences and lessons learned
On availability:
Design for loosely coupled dependence on building blocks
Diligently validate client/cloud upgrade scenarios
Invest in pre-production stress and functional coverage in
environments that look like production
Design for throttling based on both dynamic thresholds and
static bounds
Experiences and lessons learned
On monitoring:
Continuously refine performance counters, logs, and log
processing tools
Monitor end-user-visible operations (Keynote)
Build end-to-end tracing across tiers
Self-healing is hard: Invest in tuning watchdogs and thresholds
Experiences and lessons learned
On deployment:
Deployments every other week, client upgrades every month
Major functionality roughly each quarter
Took advantage of gradual ramp to learn lessons early
Extending the mesh cloud
You saw this
working today:
Microsoft
Services
Connector
federate user authentication
with other identity providers
(e.g. Active Directory)
personal and enterprise
user-data storage
offerings:
on-premises software
and online service
Stay tuned!
synchronize user-data storage with
non-Microsoft services
(e.g. Smugmug)
You saw this in the Sync
Framework session
Federating identity and policy
Azure Platform Services
Enterprise
Apps
ISV Apps
Windows Azure
Live
Identity
Provider
Microsoft
Federation
Gateway
Soon: enterprise
credentials in Live Mesh
Live
Services
Online
Services
Live
Mesh
CRM
Online
Future: enterprise
policy on Mesh Services
Microsoft
Services
Connector
Enterprise On-Premises
Active
Directory
Live Mesh
Desktop
Exchange Your Apps Sharepoint
Synchronizing clouds
SyncFX store providers
MeshMesh
Services
Services
Future: Mesh Services as
a sync hub for your apps
Accounts
app
Notifications
Sync apps
Communications
& Presence
Live Framework
User-data storage
010
101
Cloud
Sync



comprehensive
and compelling suite
build on the Mesh Services today
Live Framework
Sync Framework
feedback
personal and enterprise
offerings



http://www.mesh.com
http://blogs.msdn.com/livemesh






http://msdn.microsoft.com/sync
www.microsoftpdc.com
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market
conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.