Transcript Document
Application Layer CT 542 Content • DNS – Domain Name Server – DNS Name Space – DNS Records – Name Servers • E- Mail – Architecture and Services – Message format – Message transfer • SMTP, SSMTP – Message final delivery • POP3, IMAP, Web mail • WWW – Architecture – Static web documents • HTML overview – Dynamic web documents – HTTP DNS • IP addresses are difficult to remember – [email protected] would be difficult to remember – If JohnDoe’s mail server moves to another machine, then his e-mail address would not be valid anymore – Something like [email protected] would be appropriate • Need some mechanisms to translate wuzwuz.ucg.ie to the IP address – To solve this problem DNS was invented • It is a hierarchical, domain-based naming scheme and a distributed database system for implementing this naming scheme • Usage: – Map name onto an IP address, an application program calls an library procedure, called resolver, passing it the name as a parameter (i.e. gethostbyname() is an resolver) – The resolver sends UDP packet to a local DNS server which looks up the name and returns the IP address to the reslover – The resolver returns the IP address to the application, which can establish an TCP/IP connection with the destination (or send UDP packets) DNS Name Space • Internet is divided into over 200 top level domains – Each domain is divided into sub-domains, which are further partitioned, etc.. – All domains can be represented by a tree • The leaves of the tree represent domains that have no sub-domains (but contain machines) • A leaf domain may contain a single host or represent a company and contain thousands of hosts • Top level domains could be generic and country domains Domain Names • Can be either absolute (ends with a period i.e. eng.sun.com.) or relative (it doesn’t end with a dot) – Relative ones have to be interpreted in a context to find the true meaning – Both of them refers to a specific node in the tree and all the nodes under it • Are case insensitive (edu, Edu, EDU are same thing) • Components names can be up to 63 characters and full names should not exceed 255 characters • There is no rule against registering under two top level domains (sony.com and sony.nl) • Each domain controls how it allocates the domains under it – i.e. Japan makes a domains ac.jp and co.jp that mimic edu and com – Netherlands doesn’t make this distinction • To create a new domain, permission is required from the domain that will include it; once created, it can create sub-domains without having to ask permission from the higher up domains. Resource Records • Every domain, either single host or a top level domain, should have a set of resource records • When a resolver gives a domain name to DNS, what it gets back are the resource records associated with that name; thus the primary function of DNS is to map domain names onto resource records • A resource record has five parts: – – – – – Domain name Time to Live Class Type Value Resource Records • The Domain_name tells the domain to which this record applies – Normally many records exist for each domain and each copy of the database holds info about multiple domains – This field is the primary search key used to satisfy queries – The order of the records in the database is not important • Time_to_live field gives an indication of how stable the record is – Info that is highly stable, is assigned a large value, such as 86400 (number of seconds in a day) – Info that is highly volatile is assigned a small value, such as 60 (1 minute) • Class field is always IN for Internet information Resource Records • Type field tells what kind of record this is HINFO Address MX The CNAME PTR NS record record records record record records specifies points specify isallow allow the to the another most people name aliases name important servers. name tothe of tofind the bename (like created. out host i.e., record. what CNAME). every prepared It kind holds example: DNS to of Unlike adatabase accept machine 32 bit CNAME e-mail IPnormally and address OS for (which system the for hassome an is aan NS An SOA record provides of In the primary source of host; specified record macro domain Every definition), for corresponds domain; each Internet of 86400 it the PTR is host to. top-level used is TXT must aname DNS because records have domains data at not allow type. least every domains one Itzone, ismachine IP used address; toto identify is associate prepared somethemselves hosts a to name accept have with ecs.mit.edu IN CNAME lcs.mit.edu information about the server’s the e-mail address ofintwo itsanor more mail. IP arbitrary address If IPsomeone addresses ways. to allow wants (being lookups toconnected send of the e-mail IPtoaddress to multiple [email protected] andnetworks, return thehaving name , theone of sending the type A host administrator, a unique serial number and various flags and timeouts Creates alias for lcs.mit.edu realreverse domain name) resource needs corresponding toan find record a machine. mail per network server This atconnection); microsoft.com is (the called DNS that lookup. can is willing be madetotoaccept cyclee-mail. throughMX those (for first record can request provide to this return information first record, for second request to return the second A type record, etc…) Resource Records Example Name servers • One DNS server could service all requests – In practice it will be overloaded • To solve this, DNS name space is divided in non overlapping zones – Each zone contains some part of the tree and name servers holding zone info • A zone would have a primary DNS (gets info from disk) • One or more secondary DNS (get info from the primary DNS) Name Servers • In the example, a resolver on flits.cs.vu.nl is looking for IP address of linda.cs.yale.edu (using recursive query; some servers don’t implement recursive query and return the address of the next server to try) – The resolver sends a query containing the domain name sought, the type (A) and class (IN) – The query is forwarded by the local name server to the name server for domain edu, that is found in its database….etc • Once the records get back to cs.vu.nl name server, they will be entered in a local cache, in case they are needed later; this info is not authoritative, since changes at cs.yale.edu will not propagate to all the caches in the world; this is the reason why time_to_live should not be too big E-Mail • • • • Architecture and services User agent Message formats Message transfer agents – SMTP • Final delivery E-Mail Architecture • E-mail system consists of two parts – User agents, which allow people to read and send email • Local programs that provide a command based or graphical method for interacting with e-mail system – Message transfer agents, which move the messages from source to destination • Are typically system daemons or processes that run in background, having the job to move messages E-Mail functions • E-mail system functions – Composition – refers to the process of creating messages and answers; although any text editor can be used for the text of the message, the system itself can provide assistance with addressing and numerous header fields attached to each message – Transfer – refers to moving messages from the originator to the recipient; this requires establishing a connection to the destination or some intermediate machine, outputting the message and releasing the connection E-mail functions • E-mail system functions – Reporting – has to do with telling the originator what happened to the message; Was it delivered? Was it rejected? Was it lost? – Displaying – showing the incoming message is important, so the people can read their e-mail; sometime conversion or a special viewer is required (i.e. if the message is a PS message or an audio file) – Disposition – what the recipient does after the message has been received; possibilities include throwing it away before reading it, throwing it away after reading it, saving it and so on. E-mail architecture and functions • Distinction between envelope and its contents; – envelope encapsulates the message and contains info needed for transporting the message, such as destination address, priority and security level – Message has two parts: headers (interpreted by the user agent) and body (info for the human recipient) E-mail user agent • Sending e-mail – User must provide the message and the destination address (user@dns-address) – User agents may support mailing lists • Receiving e-mail – When an user agent is started, it looks at user’s mailbox before displaying anything – Then it may announce the number of messages in the mailbox E-mail message format • Basic ASCII e-mail message using RFC 822 – Messages consists of a primitive envelope (described in RFC821), some number of header fields, a blank line and then the message body – Each header field (logically) consists of a single line of ASCII text, a colon and, for most fields, a value (see next slide) – RFC822 was designed long ago and doesn’t clearly distinguish between the envelope fields and the header fields • This was revised in RFC 2822, however, wasn’t possible to completely redo it due to the widespread usage E-mail message format RFC 822 header fields related to message transport. Some fields used in the RFC 822 message header. E-mail message format • MIME – the Multipurpose Internet Mail Extensions – Solves problems related to send and receive of messages in non Latin alphabets, languages with no alphabets (Chinese or Japanese), messages not containing text at all (video or audio) – RFC1341 and updated in RFCs 2045-2049 • The basic idea of MIME is to continue to use RFC 822 format, but to add structure to the message body and define encoding rules for non-ASCII messages – By not deviating the RFC822, MIME messages can be sent using the existing mail programs and protocols – All it has to be changed is the receiving and sending programs, which each user can do it MIME message format RFC 822 headers added by MIME. Content-Type: header specifies the nature of the message body. Severn types are Content-Transfer-Encoding: header tells how the body is wrapped for Content-Description: defined by the RFC 2045, header each is an of ASCII which has string onetelling or more what subtypes. is the message; The typethis and transmission through a network that may object to most characters other than header isisneeded subtype separated so the by recipient a slash, aswill in: know whether it is worth decoding and letters, numbers and punctuation marks. Five schemas are provided (the simplest reading the message Content-Type: video/mpeg is ASCII text. ASCII characters use 7 bits (one bit is not used) and can be carried Content-Id: header the content directly by the mailbeidentifies protocol, provided line exceeds 1000are characters. The subtype must given explicitly inthat the no header; no defaults provided. See the for clarification. Seefollowing next slide slides for clarification. MIME – Content Transfer Encoding • Content-Transfer-Encoding: header specifies how the body is wrapped for transmission – the simplest is ASCII text. ASCII characters use 7 bits (one bit is not used) and can be carried directly by the mail protocol, provided that no line exceeds 1000 characters – The next simplest scheme is the same thing, but using 8 bit characters, all values from 0 to 255. This encoding scheme violates the original Internet e-mail protocol, but it is used by some parts of the Internet that implement some extensions to the original protocol. While declaring the encoding doesn’t make it legal, at least explain things when something goes wrong. Still this schema is respecting the 1000 chars per line rule – Even worst are messages that don’t respect the 1000 characters per line rule; Executable programs fall in this category. No guarantee is given that messages in binary will arrive correctly, but some people try MIME – Content Transfer Encoding – The correct way to encode binary messages is to use base64 encoding, sometimes called ASCII armor. In this scheme, groups of 24 bits are broken up into four 6 bits units, with each unit being sent as a valid ASCII character. The coding is ‘A’ for 0, ‘B’ for 1 and so on, followed by the 26 lower-case letters, the ten digits, and finally ‘+’ and ‘–’ for 62 and 63; The ‘==‘ and ‘=‘ sequences are used to indicate that the last group contained only 8 respectively 16 bits. Carriage returns and line feeds are ignored and can be inserted freely in the encoded data, to keep the lines short enough. Arbitrary binary data can be sent safely using this scheme. – For messages that are almost entirely ASCII but with a few non ASCII characters, the base64 encoding is somewhat inefficient; instead an encoding known as quoted-printable encoding is used. This is just 7 bits ASCII, with all the characters above 127 encoded as an equal sign followed by the character’s value as two hexadecimal digits. MIME message format • MIME Content-Type: header types and subtypes Text typeVideo: for ASCII text; Text/Plain is for allowing ordinarily messages to beonly transported MIME withfor a few extra Image Audio Application Message Multipart –isused type type type types to allows allows transmit is aare catchall one messages forstill message sound for pictures; toformats and to contain be movies; JPEG fully that more and encapsulated require video than GIF contains external just support one inside part, processing were another visual with builtinthe in information, not (useful nearly beginning covered all by not and headers; Text/Enriched allows for a simple markup language to be included in the text; this language provide system existing any message the sound end of the of browsers, forwarding); track; other each part types. RFC therefore, very 3003 When Anwell octet-stream isathey adding defined. complete are ainnew is the RFC822 just audio listafrom sequence message subtype: the beginning. of isaudio/mpeg un-interpreted encapsulated Other fortypes in supporting bytes. the are outer Toadded eindependent way to express boldface, italics, etc…(i.e. the <bold>time</bold> has come is displayed like the time to mail interpret message, with list then MP3 rfc822 files the type user should agentbe can used; just and partial callRFC3023 an subtype external can PostScript be usedinterpreter to break messages hasthe come ); Postscript, RFC 2854 adds another subtype: text/html defines text/xml into pieces; external-body subtype is used to provide FTP (url) address for the body MIME example • A birthday greeting is transmitted as both text and as a song; the receiver user agent will fetch the sound file (birthday.snd) Email message transfer • Message transfer mechanism is concerned with relaying messages from the originator to the destination – This can be done by establishing an transport level connection between the source and the destination and then just transfer the message • SMPT – Simple Mail Transfer Protocol – Source machine establishes a TCP connection on port 25 on destination machine, where SMPT daemon listens. This daemon accepts the incoming connections and copies messages from them into the appropriate mailboxes – If a message can’t be delivered, an error report containing the first part of the undeliverable message is returned to the sender – It is a simple ASCII protocol SMTP Protocol • Connection establishment (on port 25) • Data exchange – the client machine (operating as a client) waits for the destination machine (operating as a server) to talk first; – the server begins by sending a line of text giving its identity and telling whether is prepared to receive mail; • if it is not, then the client releases the connection and tries again latter – If the server is willing to accept mail, then the client announces whom the e-mail is coming from and whom it is going to – If such recipient exists at the server end, then the client get the goahead to send the message – The client sends the message, server acknowledges it • Connection is released • Stages of SMTP protocol – C – client – S – server SMTP typical problems • Some old implementations can’t handle more than 64KB message length • If the server and client have different timeouts, one of them may give up while the other is still busy, unexpectedly terminating the connection • In some situations infinite mail storms can be triggered – If host 1 is holding mailing list A and host 2 holds mailing list B and each list contains an entry for other one, then a message sent to either list could generate a never ending amount of e-mail traffic unless it is checked • RFC281 defines ESMTP (Extended SMTP) – Clients wanting to use it should start initially with EHLO instead HELO; if this is rejected then the server is regular SMTP server Final delivery • Assuming that all machines can send and receive mail all the time, the e-mail model so far works • This model breaks for people accessing Internet over a dialup connection – What happens when Elinor wants to send Carolyn e-mail and Carolyn is not currently online? • One solution is to have a message transfer agent on ISP machine; since this transfer agent can be online all the time, e-mail can be sent 24 hours a day – This solution creates another problem: how does the user gets email from ISP’s message transfer agent • Solution to create another protocol that allows user transfer agents (on client PCs) to contact the message transfer agent (on ISP’s machine) and allow email to be copied from ISP to the user • One such protocol is POP3 (Post Office Protocol Version 3), RFC 1939 Final delivery (a) Sending and reading mail when the receiver has a permanent Internet connection and the user agent runs on the same machine as the message transfer agent. (b) Reading e-mail when the receiver has a dial-up connection to an ISP. POP3 • Starts when the user starts the mail reader • Mail reader calls up the ISP (if there is no connection) and establishes a TCP connection with the message transfer agent on port 110; – Authorization • Having user logged in by sending its username and password – Transactions • User collecting the e-mails and marking them for deletion – Update • Causes the e-mails to be deleted POP3 • Sequence to receive three messages IMAP • POP3 works fine for users with one e-mail account with one ISP, accessed from one PC – If mail was accessed from different locations, user may loose emails, security issues may appear, etc • An alternative final delivery protocol, IMAP (Internet Message Access Protocol), defined in RFC2060 – Instead assuming that all messages will be downloaded and work offline after that (like POP3), IMAP assumes that all e-mail will remain on the server indefinite in multiple mailboxes – Provides extensive mechanisms to read messages or parts of messages, mechanisms to create, destroy and manipulate multiple mailboxes. POP3 vs. IMAP Web Mail • Various companies (i.e. Hotmail and Yahoo) provide e-mail service using Web mail. • Normal message transfer agents are listening on port 25 for incoming SMTP connections • Messages are delivered using special web pages; when the user goes to the e-mail Web page, a form is presented in which the user is asked for a login name and password. WWW • • • • • • Architectural Overview Static Web Documents Dynamic Web Documents HTTP – The HyperText Transfer Protocol Performance Enhancements The Wireless Web Architecture Overview Web is a collection of web pages Each page contains links to other pages Hypertext – idea of having one page point to another Browser – program to view pages Hyperlinks – strings of text that are links to other pages Example: • Typical web page • The page reached by clicking on Department of Animal Psychology. Architectural Overview • Browser displays a page on the client machine – Click on a link, the browser sends a message to the abcd.com web server asking it for the page – When page arrives, it is displayed; if it contains a hyperlink on a page on xyz.com, that is clicked, then the browser will send a message to xyz.com server and the process continues Client side • Pages are named using URL (Uniform Resource Locators) (i.e. http://www.abcd.com/products.html) – Name of protocol (http) – DNS name of the machine where the page is located (www.abcd.com) – The name of the file containing the page (products.html) • When the link is selected: – – – – – – – – Browser detects the URL (by reading the input) Browser asks DNS server for IP address of www.abcd.com DNS replies with IP address: 156.106.192.32 Browser makes a TCP connection on port 80 to 156.106.192.32 It sends a request asking for file /products.html Web server www.abcd.com sends file /products.html TCP connection is released Browser displays all the text in /products.html • Web pages are written in standard language called HTML – A page may consist of a formatted document in PDF format, an icon in GIF format, a video in MPEF format, a song in MP3 format, or any other format Client side • The browser may have problems interpreting all of these formats … rather than making the browsers larger and larger, a more general solution is adopted. • When a server returns a page, it usually returns some information about the page – MIME type of the page • Pages of type text/html are just displayed directly • If MIME type is not of a built in type, then the browser consults an internal table with associations between MIME types and viewers • Two possibilities – Plug-ins – special modules that the browser loads in its memory space – Helper applications – separate process that takes as parameter the name of the file to display Server side • Typical web server operations: – Accept TCP connection – Get the name of the file requested – Get the file (from disk) – this can be a lengthy operation, since every disk access takes in average 5ms (access time) + time to read the file (up the file length); – Return the file to the client – Release the TCP connection • Improvements – Maintain a cache with last “n” most accessed files – Multithreaded server Server side • If too many requests come into one second, the CPU will not be able to manage the load, no matter how many disks are used in parallel – The solution is to add more nodes (computers), possible with replicated disks (server farms) – A front end still accepts incoming requests and “sprays” them over multiple CPUs rather than multiple threads – Individual machines may be multithreaded and pipelined as before Server side • Problems with server farms: – Cache shared between threads is lost (because each process has its own memory) • One solution is to have the front-end smart enough to keep track where it sends each request and send subsequent requests to the same page to the same node – Clients TCP connection terminates at the front-end, so the reply must go through the front-end (a) • One solution is to perform a trick known as TCP handoff, where the TCP endpoint is passed to the processing node, so it can reply directly to the client (b) URL – Uniform Resource Locator • Some common URLs Stateless and cookies • Web is stateless, no concept of login session; the browser sends a request to the server, and gets back a file; server forgets it seen that particular client – A quick solution would be to observe client’s IP addresses (not good since they could be NAT-ed) • Cookies (Netscape) solve this problem, by having the server supply additional information when a client requests a page; this info may include a cookie, that is a small (4KB) file or string – Contains up to five fields – When the browser sends a request to a page, it first checks to see if it has an associated cookie with the domain the request is going to. If yes, then it appends this cookie to the request; the server gets it and interpret it any way it wants HTML – HyperText Markup Language • (a) HTML source code – Markup language containing explicit commands for formatting • (b) Formatted page Typical HTML tags HTML versions • HTML 1.0 was one way • Large demand for two-way traffic – HTML 2.0 included forms; contain boxes or buttons that allow users to fill in information or make choices and send the information back to the server Forms • • • (a) HTML source code (containing forms) (b) Formatted page Possible response from the user (request from the browser to the server, with filled in values) Dynamic Web documents • So far, the model we have used is a static one – A client requests a file, the server sends it back • Content has become more and more dynamic (generated on demand) – Content generation can take place at both server side and client side. Server side dynamic web page • Steps in processing a HTML form PHP form processing example (a) A Web page containing a form. (b) A PHP script for handling the output of the form. (c) Output from the PHP script when the inputs are "Barbara" and 24 respectively. Client-Side Web Page Generation • CGI, PHP, JSP and ASP solve the problem of handling forms and interaction from forms – They can accept incoming information from forms, look up info in one or more databases, generate HTML pages with the results – They can’t respond to mouse movements or interact directly with users • Client side web page generation – It is required to have scripts embedded into HTML pages, scripts that will be executed on client’s machine; starting with HTML 4.0, such scripts are permitted starting with tag <script> • JavaScript is one popular client side scripting language – Use of applets (small java programs compiled for JVM) • They can be embedded into the HTML code between <applet> … </applet> tags and can be interpreted by JVM capable browsers – Use of ActiveX controls (compiled for the native processor, very fast) JavaScript used for processing a form • Same output as the PHP example Dynamic web page generation (a) Server-side scripting with PHP. (b) Client-side scripting with JavaScript. HTTP • HyperText Transfer Protocol is the transfer protocol used throughout the World Wide Web • It specifies what messages clients may send to servers and what responses they get back in return • Each interaction consists of one ASCII request followed by one RFC 822 MIME-like response – All clients and all servers should obey this protocol – It is defined in RFC 2616 • Connections – A browser contacts a server by making a TCP connection on port 80 • In HTTP 1.0 after a connection was setup, one request was sent over, one response was sent back; then TCP connection was released • In HTTP 1.1 persistent connections are supported HTTP Methods TRACE The POST DELETE GET HEAD PUT is method similar method does method what to isis requests PUT. used just the youreverse asks for It expect: the too, debugging. for server bears ofthe deletes GET. message to anIt send URL, Instead a instructs page the header, but from of page instead reading theawithout .server remote Theofapage page, replacing to the web send is actual itserver; suitably writes back the page. existing the authentication aencoded page. This request. data, method This inThis it MIME. can method appends and is useful be permissions used The makes itwhen toto vast any get requests it play possible majority existing a page’s a major areto data. oftime not build the role processed Posting of requests ahere. last collection modification, aThere message on correctly theis ofweb no web toand guarantee are atopages newsgroup the collect GETs. client onthat information a remote wants or a DELETE adding toweb know for aserver. file operation indexing it to sent The a the is purposes, body bulletin successful proper ofrequest. board theor since request just are even toexample test contains if an theURL of remote the thefor page. usage. web validity It server may beisencoded willing to using delete MIME, a page, in the which filecase, the GET filename HTTP/1.1 lines permissions following may the bePUT set inmight such include amoment way that Content-Type the server process and authentication doesn’t have headers rights to to do prove it. CONNECT – is not used at the Where is thehas resource nametotoperform be fetched 1.1 is the protocol version being that thefilename caller indeed permission theand operation used OPTIONS – provides a way for the client to query the server about its properties or those of a specified file HTTP methods • Every request gets back an response consisting of – A status line • A three digits status code telling whether the request was satisfied and if not, why not – Possible additional information References • Andrew S. Tanenbaum – Computer Networks, ISBN: 0-13066102-3