Transcript Application design and the end-to

Application design and the
end-to-end arguments
David D. Clark
MIT CFP
MIT Communications Futures Program
Bi-annual meeting, May 30-31, 2007
Philadelphia, PA
This talk is about:

The evolution of thinking around the endto-end arguments (E2EA).




A quick review of E2EA…
Design principles for modern applications.
What these have to do with each other.
What we might learn about distributed
applications and “network services”.
Some readings






van Schewick, Architecture & Innovation: The role of the
end-to-end arguments in the original Internet, PhD, TUB
Tim Moors, A critical review of “End-to-end arguments in
system design”, 2002 ICC
Chen and Jackson, editors, Commentaries on “Active
network and end-to-end design”, !EEE Network,
May/June 1998
Kempf and Austein, The rise of the middle and the
future of end-to-end, RFC 3724
Reed, The end of the end-to-end argument, 2002
www.reed.com
Blumenthal and Clark, Rethinking the design of the
Internet: the end to end arguments vs. the brave new
world, ACM ToIT, 2001
Relating E2EA and applications



To a purist, the E2EA do not speak to the
design of applications.
The E2EA are concerned with the
communications subsystem and “the rest”.
But today’s applications don’t seem very
E2E.

Does this matter?
Two quotes from the first paper
“In a system that includes communications, one
usually draws a modular boundary around the
communication subsystem and defines a firm
interface between it and the rest of the system.”
“The function in question can completely and
correctly be implemented only with the
knowledge and help of the application standing
at the end points of the communication system.”
So what is an end-point?
The ends of the transport connection?
 Often true but not the definition.
The “ends” of the application?
 Often true but…
The people using the system?

“Excuse me, someone dropped a glass. Would
you please say that again?”
Consider this case

A person in St. Louis (middle of US) doing a
“careful file transfer” from San Francisco to
Boston.






Compute checksum in SF and send to StL to save
Move file from SF to Bos
Compute checksum in Bos and send to StL.
Compare in StL.
Now where are the ends?
I claim the real end is the user in St. Louis.

There are at least 3 transport connections.
Why is this being careful?


Original argument: the communications
subsystem and the disks are unreliable
(for lots of good reasons).
The check of correct operation has to be
done at a layer that is aware of the desired
semantics of the operation.


What does it mean to the application to be
“reliable”?
But why are the actions of the end-point
reliable?
In search of reliability

The E2EA seem to equate the end-points
with points that are reliable and
trustworthy, as well as points where the
application code runs.


Probably sensible, but not fundamental.
I tried to sneak in a new word: “trustworthy”.
A case study--email

Email is delivered in stages:




From sender to sender’s SMTP server.
From sender’s server to receiver’s server.
From receiver’s server to receiver via POP or
IMAP.
Good reasons for this design.

Asynchronous delivery, delivery to multiple
clients, etc.
The shape of email
Sender
SMTP
server
PoP/
IMAP
server
Receiver
But is it “E2E”?



Answer 1: No. there is no overall confirmation
wrapped around the “outside”.
Answer 2: Yes, by definition. E2E does not apply
to apps. All of the servers are part of “the rest”. It
is E2E if the communications subsystem is E2E.
Answer 3: Yes. We have used reliable,
application-aware parts to compensate for
unreliable ones.



We deem the servers to be trustworthy and reliable.
I trust mine, you trust yours…
Answer 4: No. We trust more than we need to.
Is email reliable?


The more basic question.
Usually yes. But when it is not…


Case study from Africa. Flakey ISP led users
to invent a “whole new idea”. E2E sequence
numbers and acks.
Spam checkers eat good mail. How do we
compensate?
Summary so far


A possible re-framing of E2E is “trust-to-trust”.
Original: “The function in question can completely and
correctly be implemented only with the knowledge and
help of the application standing at the end points of the
communication system.”

Revised: “The function in question can completely and
correctly be implemented only with the knowledge and
help of the application standing at points where it can be
trusted to perform its job properly.”
Trust--a recognized issue.
“The decision to implement reliable transfer
in the transport layer is not justified on the
basis of the end-to-end arguments, but
rather on the basis of trust”.
-Tim Moors, A critical review of “End-to-end
arguments in system design” 2002
Evolution of E2E objectives



Correctness--”careful file transfer”.
Generality--support many apps.
Open--no constraints on user action.


Innovation--lower barriers.




Lessig and Lemley
van Schewick
Simplicity--fewer bugs and lower cost in core.
Minimality--don’t depend on more than
necessary.
Optional function--only add, don’t prevent.
Changing our tune


Original: “The function in question can
completely and correctly be implemented only
with the knowledge and help of the application
standing at the end points of the communication
system.”
1998: “ A function or service should be carried
out within a network layer only if it is needed by
all clients of that layer, and it can be completely
implemented in that layer”.
Re-evaluate the network


Is it that the communication subsystem is
“unreliable”, or that it is operated by a stakeholder who may not have interests aligned with
the end-user?
End-to-end changes from a technique to
achieve correct operation to a technique for
tussle.
-van Schewick, Architecture and Innovation:
the role of the end-to-end arguments in the
original internet, has an excellent analysis of
this:
Application design



The original argument--the
communications subsystem and the rest-seems to be specific to the network layer.
A discussion of trust and alignment of
interests is not restricted to any layer.
So perhaps this broader interpretation can
help us understand how to design
applications.
First fact of life today


Applications are not simple, two-party transfers.
They are “full of “ servers and services.




Email
Web: caches, proxies, Akamai, etc.
IM, games, etc.
Begs the question--is the design compatible with
actual trust assumptions?

T2T would argue: put the function where “I” trust it will
be carried out properly.
Second fact of life today

We don’t trust each other.


Contrast with original “careful file transfer”.
Each party may delegate to servers that it
trusts.
The shape of email
Sender
PoP/
IMAP
server
SMTP
server
My sphere
of trust
Receiver
Your sphere
of trust
Any trust
here?
Two examples

Email: protocols allow functions to be
assigned to different servers.



Can re-arrange to match trust (and other)
assumptions.
The story from Africa: ISP “forced” users to
use their mail server.
IM: the most popular design requires
users to trust (use) a third-party server,
whether or not they want to.

Sort of…
If we equate E2E with trust

In particular, the assignment of function to nodes
that are trusted to carry them out .






Then third party servers and services are not
intrinsically in violation of E2E.
Trust--depends on choice.
Minimality--why trust them?
Open--can anyone deploy them?
Generality--does not seem to be an issue.
Simplicity--keep the core simple. Is this
relevant?
Third fact of life today

We cannot trust our own end-computers.




Perhaps the final insult to classic E2E.
And we may not want to maintain them.
So trust may imply movement away from
end-computers on to specialized servers.
Simplicity may imply similar movement.
Back to lack of mutual trust
What do we do when we don’t trust each other but
want to interact?

A real life question.
1) Put up lots of defenses and proceed with
caution.
2) Utilize trusted third parties.

Real life is full of these--credit card companies,
registries of deeds.
Some classic E2E arguments (minimality, trust the
edges) might imply a preference for 1.

This make sense?
The shape of trust in email
Sender
SMTP
server
My sphere
of trust
PoP/
IMAP
server
Receiver
Your sphere
of trust
Reasoning by visual analogy: two “distributed ends”.
Sort of E2E-like…
Third parties
My sphere
of trust
Customer
Credit
card co.
Your sphere
of trust
Merchant
This is “trust in the middle”, not the edges.
Reasoning by analogy

The E2EA arguments focus on the end in the
context of a communications subsystem that is:



These factors do not generalize.



Perhaps untrustworthy.
“Naturally” unreliable.
Services in the middle can be picked based on
determination of trust.
They are not “naturally” unreliable.
The residual argument is minimality.

There may be a high price for being unwilling to trust
a third party.
Yes, there is a relevant E2EA


It is about moving function to places where
it can be trusted to be carried out.
It implies choice by the end-users.

End-user choice empowers the user.
Trust as the user pleases.
 Select applications, servers and services as the
user pleases.


The goals of generality, innovation, lack of
constraint are served by choice.
The unstated “other half”

The E2EA presume that you can, if you
choose, build a general, applicationindependent useful service.



Like packets.
But the discovery of packets is not a
consequence of the E2EA. It is the success of
packets that make the E2EA relevant.
How does this relate to applications?
Generalities at the application

Open platforms for services and servers.


Services that cross applications.




Like PlanetLab.
Identity management
Trusted agents.
Location and presence management.
Does any form of the E2EA speak to the
design of these?
Design options

Allow for options and alternatives.



Different actors offering the “same” service.
Different forms of the service for different
contexts.
Let the end-user have choice and pick.

Each end may want to pick independently.
Generalizing the E2EA…
In 3 easy steps.
 E2E -> T2T.
Position function where it can be trusted to do its job.
 Position trusted function around untrusted/unreliable
components to compensate and correct for them.
There can be more than 2 points of trust.
 T2T2T, etc.
 Balance minimality with utility.



The actual end-points (computers) may not be fully
trustworthy.


The user (person) at the end is the ultimate locus of trust.
This trust is expressed through choice, among other ways.