Transcript Pat Helland Partner Architect Microsoft Corporation Understand the use and interpretation of data It is different inside services than outside services Many new issues.

Pat Helland
Partner Architect
Microsoft Corporation
Understand the use and interpretation of data
It is different inside services than outside services
Many new issues arise in the world between services
Versioning, meta-data (schema), immutability, idempotence, and
much more
Think carefully about the interactions across services
XML, SQL, and Objects all have their place
Understand their strengths and weaknesses and use them
together to create the best services
Slide 2
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 3
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 4
Service-orientation
Independent services
Chunks of code and data
Interconnected via messaging
Four basic tenets:
Boundaries are explicit
Services are autonomous
Services share schema
and contract
Not implementation
Service compatibility
is based on policy
Service
Policy
Schema and
Contract
Slide 5
Services communicate with messages
Nothing else
No other knowledge about partner
May be heterogeneous
Service-A
Service-B
Slide 6
Data is different inside from outside
Outside the service
Passed in messages
Understood by sender and receiver
Independent schema definition important
Extensibility important
Inside the service
Private to service
Encapsulated by
service code
MSG
Data
SQL
MSG
Data Outside
the Service
Data Inside
the Service
Slide 7
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 8
Services only do limited things for their partners
This is how they bound their trust
Encapsulation is about bounding trust
Business logic ensures only the desired operations happen
No changes to the data occur except through locally controlled
business logic!
Service
Things I’ll Do for Outsiders
• Deposit
• Withdrawal
• Transfer
• Account Balance Check
Slide 9
Encapsulating change
Ensures integrity of the service’s work
Ensures integrity of the service’s data
Encapsulating exported data for read
Ensures privacy by controlling what’s exported
Allows planning for loose coupling and expirations
E.G. Wednesday’s price-list
Sanitized Data
for Export
Data
Exported Data
Private
Internal
Data
Business Request
Some propose atomic transactions across services
E.G. WS-Transactions
Requires holding locks
Lots of trust in timely unlock
Doesn’t sound autonomous and independent to me…
Debate is the definition of the word service
Requires autonomy and independence?
Allows intimacy across service boundaries?
There will be code connected by 2-phase commit
Same service or in different services?
For this talk, I presume no cross-service txs
Simply the definition of the word “service”
Slide 11
Services are connected by messaging
The only interaction between two services is by the messages that they
exchange
Schema: the formats of the individual messages
Contracts :the allowable sequences of messages
Service
Tentative
Place-Order
One
Of
Reject-Order
Contract
Accept-Order
One
Of
Confirm
Cancel
Place-Order
Place-Order
Slide 12
Messages contain operators
Requests a business operation
Operators provide business semantics
Part of the contract between the two services
Operator messages contain operands
Details needed to do the business operation
The sending service must put them into the message
Service
Deposit
Operands
Operator
Slide 13
Operands come from reference data
New kind of data in SOA
Except it’s not new;
we’ve done variations of SOA for decades…
We’re just getting better at it!
Reference data is versioned and
each version is immutable
Immutable images are shared across many services
We will talk about the creation, publication, and
management of reference data
Slide 14
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 15
Transactions make you feel alone
No one else manipulates the data when you are
Transactional serializability
The behavior is as if a serial order exists
Tg
Te
Ta
Tj
Ti
Tf
Tc
Tb
Ti Doesn’t Know About These
Transactions and They Don’t
Know About Ti
Td
These Transactions
Precede Ti
Tn
Tl
Th
Tk
Transaction
Serializability
Tm
To
These Transactions
Follow Ti
Slide 16
Transactions live in the “now” inside services
Time marches forward
Transactions commit
Advancing time
Transactions see
the committed
transactions
Service
A service’s
biz-logic lives
in the “now”
Each Transaction
Only Sees a Simple
Advancing of Time
with a Clear Set of
Preceding
Transactions
Slide 17
Messages contain unlocked data
Assume no shared transactions
Unlocked data may change
Unlocking it allows change
Messages are not from the “now”
They are from the past
There is no simultaneity at a distance!
• Similar to speed of light
• Knowledge travels at speed of light
• By the time you see a distant object it may have changed!
• By the time you see a message, the data may have changed!
Services, transactions, and locks bound simultaneity!
• Inside a transaction, things appear simultaneous (to others)
• Simultaneity only inside a transaction!
• Simultaneity only inside a service!
Slide 18
All data from distant stars is from the past
• 10 light years away; 10 year old knowledge
• The sun may have blown up 5 minutes ago
• We won’t know for 3 minutes more…
All data seen from a distant service is from the “past”
By the time you see it, it has been unlocked and may change
Each service has its own perspective
Inside data is “now”; outside data is “past”
My inside is not your inside; my outside is not your outside
Going to SOA is like going from Newtonian to Einstonian physics
• Newton’s time marched forward uniformly
• Instant knowledge
• Before SOA, distributed computing many systems look like one
• RPC, 2-phase commit, remote method calls…
• In Einstein’s world, everything is “relative” to one’s perspective
• SOA has “now” inside and the “past” arriving in messages
Slide 19
A sequence of versions describing changes to data
Updates from
one service
Data Owning Service
Wednesday’s
Price-List
Owner controlled
Owner changes
the data
Sends changes
as messages
Data is seen
as advancing
versions
Wednesday’s
Price-List
Price-List
Wednesday’s
Price-List
Tuesday’s
Price-List
Monday’s
Price-List
Wednesday’s
Price-List
Tuesday’s
Price-List
Listening
Partner
Service-1
Listening
Partner
Service-5
Listening
Partner
Service-8
Tuesday’s
Price-List
Monday’s
Price-List
Listening
Partner
Service-7
Slide 20
Messages may contain operators
Requests for business functionality part of the contract
Service-B sends an operator to Service-A
If Service-A accepts the operator, it is part of its future
It changes the state of
Service-A
Service-B is hopeful
It wants Service-A to do
the work
When it receives a reply,
its future is changed!
Hopeful for
the Future…
Decides
to Issue
Request
Ever
Hopeful,
Waiting
for a
Response
Invoking
Partner
Service-B
Invoked
Partner
Service-A
Operator
Request
Operator
Response
Hopes Fulfilled,
the Future
Is Now
Blithely
Ignorant
and
Minding
Its Own
Business
A Future
Forever
Altered
by the
Processing
of the
Request
from
Service-B
Slide 21
Operands may live in the past
Values published as reference data
Come from Service-A’s past
Service-B Preparing a Request for Service-A
Deposit
Friday’s
Price-List
Published:
11PM Thursday
Operands
Operator
On Friday, Operands
Are Extracted from
the Price-List Published
on Thursday
Operands may live in the future
They may contain a proposed value submitted to Service-A
Slide 22
Everything between services lives in the past or future
Operators live in the future
Operands live in the past or the future
It’s not meaningful to speak of “now” between services
No shared transactions  no simultaneity
Life in the “then”
Past or future
Not now
Service-1
Each service has
a separate “now”
Service-4
Different temporal
environments!
Service-2
Service-3
No Notion
of “Now”
in Between
Services!
Slide 23
Services Make the “Now” Meet the “Then”
Each Service Lives in Its Own “Now”
Messages Come and Go Dealing with the “Then”
The Business-Logic of the Service Must Reconcile This!!
Example: accepting an order
• A biz publishes daily prices
• Probably want to accept
yesterday’s prices for a while
• Tolerance for time differences
must be programmed
Example:
“Usually ships in 24 hours”
• Order processing has old info
• Available inventory not accurate
• Deliberately “fuzzy”
• Allows both sides to cope with
difference in time domains!
The world is no longer flat!
• SOA is recognizing that there is more than one computer
• Multiple machines mean multiple time domains
• Multiple time domains mandate we cope with ambiguity to
allow coexistence, cooperation, and joint work
Slide 24
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 25
Reference data is published across service boundaries
For each collection of reference data:
One service creates and publishes the data
Other services receive periodic versions of the data
Purposes for Reference Data
Operands
Historic Artifacts
Shared Collections of Data
Slide 26
As discussed above, messages across services invoke
business operations…
Each service-to-service message is an operator
Each operator message is filled with operands
Parameters, options, customer-id, parts-being-ordered, etc
The data for these operands is published as
reference data
Service
Deposit
Operands
Operator
Slide 27
Historic artifacts report on what happened in the past
Sometimes these snapshots need to be sent to other services
Examples:
Sales quarterly results
Monthly bank statements
Any and all monthly bills
Well…
Both requests for payment (operations) and the
historic artifact of how much power you used…
Inventory status at end of quarter
Service
(Bank)
Bank
Statement
Jul-2007
Slide 28
Many services may need access to the same data
The data is changing…
Someone owns updating and distributing the data…
Examples:
Customer database
Employee database
Parts database and price-list
HR Service
Authoritative
Employee
Vers#23
UpdateData – Vers#24
Employees
Sales Service
Ref Vers#24
of Employee
Data
Ref Vers#24
Vers#23
of
Employee
Update!
Data
Authoritative
Customer
Data
Slide 29
The owner of data periodically publishes
Using whatever messaging technique it wants
Publications are always versioned
The version numbers increase
1
A’s-Data
Vers-X
Service-B
2
A’s-Data
Vers-Y
Service-A
3
A’s-Data
Vers-Z
4
Request
Uses: Vers-Z
Slide 30
If a non-owner wants a change it must do a biz-operation
This is a request sent to the owning service
The owning service may agree to the operation causing change
to the data in question
If it changes, this affects the next version`
1
A’s-Data
Vers-X
Service-B
Request
Uses: Vers-X
Please Make
Data Change
2
A’s-Data
Vers-Y
Owning
Service-A
3
Slide 31
What is optimistic concurrency control?
Data is read
Changes are made and submitted to the data’s owner
If the original data hasn’t changed, the new changes are applied
This assumes the remote system should be able to write
directly on the data
This is a trusting relationship… not autonomous!
Autonomy and updates to data
Autonomy means independent control
My local biz-logic decides how my data changes!
If you want a change, ask me to do a business op
It’s my data…I’ll decide how it changes!
Slide 32
What about a salesperson updating
a customer’s address?
Shouldn’t that just be optimistic concurrency control?
No! It should invoke business logic with a request!
Not all fields of the customer record should be updated
by sales people
Requests across service boundaries invoke business logic when
the customer address is changed
Slide 33
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 34
Data may be immutable
Once written, it is unchangeable
•Windows Vista, SP1
• The Same Set of Bits
Every Time
Immutable data needs an ID
From the ID, comes the same data
No matter when, no matter where
Versions are immutable
Each new version is identified
Given the identifier, the same data comes
Recent NY Times
• Maybe Today’s,
Maybe Yesterday’s
Version independent identifiers
Let you ask for a recent version
New York Times; 7/24/07
Latest SP of Vista
• Specific Version of the Paper
-- Contents Don’t Change
• Definitely Vista,
Results Vary Over Time
Version
Independent
Slide 35
Retries are a fact of life
Zero or more delivery semantics
Messages must be immutable
Retries must not see differences…
Once it’s sent, you can’t un-send!
Service-A
Once It’s Outside,
It’s Immutable!
Slide 36
OK to cache immutable data
It’s never wrong
Never have to invalidate!
Caching should only be used for immutable data
Caching data that changes may lead to anomalies
Consider caching data labeled with
a version dependent ID
Because versions are immutable it will work
Store the mapping from version independent to version
dependent in an accurate location
Slide 37
Databases design for normalized data
Can be changed without “funny behavior”
Each data item lives in one place
Sometimes data should be de-normalized
If data is immutable it’s OK
De-normalization is
OK if you aren’t going to update!
Emp # Emp Name
47
Joe
18
Sally
91
Pete
66
Mary
Classic problem
with de-normalization
Emp Phone Mgr # Mgr Name Mgr Phone
5-1234
13
Sam
6-9876
3-3123
38
Harry
5-6782
2-1112
13
Sam
6-9876
5-7349
02
Betty
4-0101
Can’t update
Sam’s phone #
since there are
many copies
Slide 38
Immutability isn’t enough!
We need a common understanding
President Bush  1990 vs. President Bush  2007
Stable data has a clearly understood meaning
The schema must be clearly understood
The interpretation of values must be unambiguous
Suggestion
• Timestamping or
versioning makes
stable data
Advice
• Don’t recycle
customer-ids
Observation
• A monthly bank statement
is stable data
Observation
• Anything called
“current” is not stable
Slide 39
Outside data must be stable
Consistent interpretation across valid spaces and
times
Inside data may be stable
Notably, when it is the same data as outside data…
Sometimes data inside is not stable
Classic normalization for vibrant update
Needs to be cast into a stable shape to send outside
Slide 40
Bounding the valid times
It may have an expiration
Bounding the valid locations
Restrictions on where the data is valid
When valid, the data should be:
Immutable (the ID yields the same bits)
Stable (the meaning is clear)
Data Valid For Service-X Only
Price-List Valid
Until Dec 31st
“Offer Good Until Next Tuesday”
“Offer Good to Washington State Residents Only”
Slide 41
Identify the
Message
Put Unique ID in All Messages
Part of the Unique ID May Be a Version…
Immutable
Data
Don’t Change the Data Associated with
the Unique ID; Never Return Different Bits
OK to
Cache
The Same Bits Will Always Be Returned
Define Valid
Ranges
Valid for a Certain Time Period and Over
Some Space; OK to Always Be Valid
Must Be
Stable
Must Ensure There Is Never Any Confusion
About the Meaning (Within Valid Range)
Slide 42
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 43
SQL’s DDL (data definition language) is transactional
Changes are made using transactions
The structure of the data may be changed
The interpretation after the DDL change is different
DDL lives within the time scope of the database
The database’s shape evolves over time
DDL is the change agent for this evolution
SQL lives in the “now”
Each transaction’s execution is meaningful only within the
schema definition at the moment of its execution
Serializability makes this crisp and well-defined
Slide 44
When a message is sent, it must be immutable
It is crossing temporal boundaries
Retries mustn’t give different results
The message’s schema must be immutable
It makes a mess if the interpretation of the message changes
Message
Message
Schema
Service-A
Immutable Message
Immutable Schema for the Message
Slide 45
Immutable schema needs an identifier
It must be possible to unambiguously identify the schema
This must occur across the namespaces of sender and receiver
The schema definition must never change
Given the identifier, the same schema is returned
URIs (Universal Resource Identifiers) work well
Guaranteed to be unique
If you follow the rules
URLs (Universal Resource Locators) are cool
URLs are URIs
Also tell you a location to get the stuff (e.g. the schema)
Slide 46
Schema make contain sub-schema
Inside the message are chunks of data
A purchase-order may contain customer information
They have their own definitions
The sub-schema are referenced by identifier
This leads to a tree of references to immutable schema
It’s really a DAG (Directed Acyclic Graph)
Sometimes, different sub-schema reference the same stuff
Purchase Order
Customer
Delivery Addr
SKUs
Customer
Name
Address
Credit Rating
SKU
Part
Color
Size
Address
Number/Street
City/State
Postal Code
Country
Slide 47
Frequently, schema is versioned
A new format of the schema is created
It is given a new identifier
Version independent schema identifiers
Specify a set of versions for a type of schema
The set may evolve over time
Version dependent schema identifiers
Specify a specific version of a specific schema
The version-dependent schema is immutable
Messages should always specify
a version-dependent schema
This ensures no ambiguity
Slide 48
Extensibility is the addition of non-schema specified
information into the message
The schema does not specify the additional stuff
The sender wanted to add it anyway
Adding extensions is like scribbling in the margins
Sometimes adding notes to a form helps!
Sometimes it does no good at all!
Service-A
Purchase Order
Customer
Delivery Addr
SKUs
Purchase Order
Customer
Delivery Addr
Don’t Deliver in Morning
SKUs
Slide 49
XML-Infoset
Semantics of XML, not syntax
Tree: parents, children, elements, & attributes
Allows (encourages) schema
Any representation OK
XML-Schema
Datatype library and schema definition
Composed schema uniquely identified (URI)
PSVI – Post Schema Validated Infoset
Infoset after validation against schema
Can leverage schema knowledge
Slide 50
Messages must be immutable
Retries must not see differences…
Message schema must be immutable
Must be able to unambiguously interpret the message
XML-Infosets and XML-Schema work well for messaging!
Service-A
Immutable Messages
Immutable Schema
Schema for
the outside
is very different
from schema for
the inside!!!
Slide 51
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 52
Different services frequently have different schema
How do you represent a customer?
What information about a customer is relevant?
What special info does your business-unit keep about
customers?
Is there an enterprise-wide customer-id?
Just disambiguating the identity of a unique customer is fun!
How do you represent:
Address?
Status level
(e.g. Gold-customer)?
Special considerations?
Customers for General Electric:
• Purchasers of aircraft engines
• Purchasers of refrigerators
• Purchasers of nuclear power plants
• Purchasers of light bulbs
• Purchasers of locomotives
Slide 53
Some data
requires
transformation
Reader-makes-right
Receiver fixes message
Based on both schemas
Best possible mapping
N-squared combinatorics
Reading
Service
Other data
copies through
Everybody knows everybody
N*(N-1) mappings
Gets big fast!
Svc
Svc
Svc
Svc
Svc
Svc
Svc
Svc
12 Services
12 * 11 = 132
message transformers
Svc
Svc
Svc
Svc
Slide 54
Canonical schema is a standard representation
Map everything to the canonical and back out
Far fewer transformers required
Using a canonical schema is double-lossy
Info loss going in and info loss going out
Svc
Svc
Svc
Svc
Svc
Svc
Two
transformations
gives twice
the data loss
Canonical
Svc
Source
Service
Svc
Svc
Svc
Svc
Svc
12 Services
2 * N; 2*12=24
message transformers
Canonical
Schema
Transforming
Service
Destination
Service
Slide 55
Classic Solution:
Implement Canonical Schema
Practical to Build the Transformers
2 * N (for N Different Schemas)
Specialize When Needed
If Doubly-Lossy Hurts, Then Reader-Makes-Right
Specialize Only Where Important
Hybrid Trades Work for Fidelity
The cool thing about services:
• Rationalizing schema occurs for outside data
• Inside a service, you have what you want!
• You only have to cooperate on the outside!
Slide 56
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 57
When data arrives from the outside, you store it inside
Most services keep incoming data
Keep for processing
Keep for auditing
Internal Data
Incoming
Data
Slide 58
Shredding the message
The incoming data is broken down to relational form
Empowers query and business intelligence
Auditing considerations
Typically, don’t want to change the message image
Preserve for auditing
May keep unshredded version also for non-repudiation
Extensibility
The sender added stuff you didn’t expect
May or may not know how utilize extensions
Extensibility fights shredding!
Hard to map extensions to planned relational tables
OK to partially shred
Yields partial query benefits
Slide 59
Atomic transactions consume messages
Each message is consumed
Data may be changed
Records outgoing responses
All of a service’s state lives in the database!
It is transactionally modified
Service
Service-Private
Data
Service Logic
Transaction
Slide 60
Resource-Oriented-Data
Lives Longer Than One
Long-Running-Operation
Changed by
Long-Running-Operations
Activity-Oriented-Data
Gets Modified by
Long-Running-Operations
Retires On Completion of
Long-Running-Operation
These classes of data have
different characteristics
We’re gonna examine some of the
consequences of these differences…
Inventory
of SKU#
71946
####
Customer
Info for
Cust #8319
Bank Balance “Sally’s Auto”
#Info#
of Account#
01600-18653
#$$$#
Shopping
Basket
#1834953
####
Account-Recv
Inv#173-45 for
PO#307654-03
#Info#
#Payments#
PO#307654-03
Order for
Joe’s Flowers
#Master#
#LineItems#
Shipping Order
#86403 tied to
PO#307654-03
#Info#
#Shipped#
Slide 61
Activity-oriented and resource-oriented data
live inside services
They are encapsulated within the service
Access is mediated by business logic
Internal Data
Inventory
Resource
of SKU#
Oriented
71946
Data
####
Shopping
Activity
Basket
Oriented
#1834953
Data
####
Slide 62
Data sometimes retires
It becomes read-only
It is referenced less and less frequently
It may get archived to tape
Rarely, it is deleted
New regulations will make this very rare
Activity-oriented-data retires when the
long-running operation completes
This may take weeks or months
Resource-oriented-data sometimes retires
The resource-oriented-data may live a long time
Skus, accounts, employees, customers, etc
These may live for years
Sometimes they leave
Some resources have bounded life
Hotel occupancy info; airplane seat assignments
Slide 63
Updates of only activity-oriented data
This is partitioned by long-running work
Almost never have concurrent work!!
You can have it but it’s not a performance issue…
Activity-oriented data have low concurrency updates!
Updates affecting resource-oriented data
The resource-oriented data MAY be in high demand
Possible that it is so desired that there is a queue for update
Resource-oriented data MAY have high concurrency updates
Slide 64
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 65
Xml
Schematized representation of messages
Schema supports independent definition and extensibility
SQL
Stores relational data by value
Allows you to “relate” fields by values
Incredibly query capabilities
World-class management of data
Data
SQL
Objects
Very powerful software engineering tool
Based on encapsulation
Slide 66
Relational is bounded
Operations within the database
Value comparisons only meaningful inside
Tightly managed schema
XML-Infoset is unbounded
Open (extensible) schema
Contributions to schema from
who-knows-where
References (not just values)
Uris known to be unique
XML-Infosets can be interpreted anywhere
Slide 67
SQL is anti-encapsulated
UPDATE WHERE
Query/update by joining anything with anything
Triggers/stored-procs are not strongly tied to
protected data
XML is anti-encapsulated
Please examine my public schema!
Components/objects offer encapsulation
Long tradition of cheating
Reference passing to shared objects
Whacking on shared database
Slide 68
Anti-encapsulation is OK in its place
SQL’s anti-encapsulation is only seen by the local biz-logic
XML’s anti-encapsulation only applies to the “public” behavior and
data of the service
Encapsulation is strongly enforced by the service
No visibility is allowed to the internals of the service!
Sanitized Data
for Export
Exported Data
Business Request
Data
The Service
Private
Is aInternal
Data
Black Box!
Slide 69
Persistent Objects
Encapsulated by Logic
Kept in SQL
Uses Optimistic Concurrency (Low Update)
Stored as Collection of Records
May Use Records in Many Tables
Keys of Records Prefixed with Unique ID
This is the Object ID
Encapsulation by Convention
SQL
Table-A
ID-X
ID-Y
ID-Z
<key>
<key>
<key>
Database-Key
Persistent Objects
Work Great for
Activity-Oriented-Data
Table-B
ID-X <key1>
ID-X <key2>
ID-X <key3>
<record>
<record>
<record>
ID-Y <key1>
ID-Y <key2>
<record>
<record>
Database-Key
<record>
<record>
<record>
Persistent Object
ID=Y
Slide 70
Interesting!
Request
Response
Reference
Data
Activity
Oriented
Resource
Oriented
Requires Open Schema
for Interoperability
Yes
Yes
No
No
Encapsulation Useful
No
No
Yes
Yes
Immutable or
Low Concurrent Update or
Highly Concurrent Update
Immutable
Immutable
(versions)
Fodder for Business
Intelligence Analysis
Probably
Probably
Definitely
Definitely
Represent in XML?
Yes
Yes
Probably
Not
Probably
Not
Durable storage in SQL?
XML copy
in SQL
XML copy
in SQL
Definitely
Definitely
Encapsulated Access?
No
No
Yes: Use
Obj Persist
Yes:
Stateless
Low
Highly
Concurrent Concurrent
Update
Update
Maybe copied using tools like XML  SQL stuff; shredding what you want
to do BI Analysis on. Maybe transformed in and out using business logic.
It is fantastic to compare anything to anything and combine
anything with anything in Relational (within the bounded database)
It is possible to have independent definition of schema and data in
XML
XML-Infosets. You can independently extend, too.
Components/ Provide encapsulation of data behind logic. Ensure enforcement of
Objects
business rules. Eases composition of logic.
SQL
Strengths and
Weaknesses
Arbitrary
Queries
SQL
Outstanding
Bounded Schema
XML
Independent
Data Definition
Encapsulation
(Controls Data)
Impossible:
Not via SQL
Centralized Schema Enforced by DBA
Problematic:
Outstanding
Schema
inconsistency
Unbounded Schema
Impossible:
Impossible
Objects
Encapsulated Data Can’t see the data! Can’t see the data!
Impossible:
Open Schema
Outstanding
Each model’s strength is simultaneously its weakness!
You can’t enhance one to add features of the other without breaking it!
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 73
Use XML-Infosets across services
Standard or proprietary marshaling formats
Use ML-Schema
Interactions across services include:
Requests,
Responses, and
Reference-data
Service-A
Slide 74
Enterprise-class services
Encapsulate a collection of private data
Store your data in a SQL database!
Industrial strength store, performance,
and management
Store incoming and
outgoing messages in SQL
Remember incoming and
outgoing for auditing
Remember them for Biz Intelligence
Transactionally record
outgoing messages for retry
Data
SQL
Slide 75
Build your services with objects
Offer encapsulation of parts within the service
They are the best for software engineering!
Encapsulation
Services are a big chunk
Completely encapsulate some durable data
Objects make littler chunks
Inside the service
Make it easier to build
Slide 76
Services Need All Three!
XML-Infosets: Between the services
Objects:
Implementing the business logic
SQL:
Storing private data and messages
Data
SQL
XML-InfoSets for
Objects Implement SQL Holds
Messages Between Services
the Biz Logic
the Data
 Introduction: The Shift Towards Services
 Behavior: Encapsulation and Trust
 Data: Then and Now
 Outside Data: Reference Data
 Outside Data: Sending Messages
 Outside Data: XML and Schema
 Outside Data: Commonality of Schema and Understanding
 Inside Data
 Inside/Outside: Representations of Data
 Inside/Outside: Tying It All Together
 Conclusion
Slide 78
Data is different inside from outside
Outside the service
Passed in messages
Understood by sender and receiver
Independent schema definition important
Extensibility important
Inside the service
Private to service
Encapsulated by
service code
MSG
Data
SQL
MSG
Data Outside
the Service
Data Inside
the Service
Slide 79
Understand the use and interpretation of data
It is different inside services than outside services
Many new issues arise in the world between services
Versioning, meta-data (schema), immutability, idempotence, and
much more
Think carefully about the interactions across services
XML, SQL, and Objects all have their place
Understand their strengths and weaknesses and use them
together to create the best services
Slide 80