Question? Leave a message!




Network Services and Applications

Network Services and Applications
Network Services and Applications EECS 489 Computer Networks http://www.eecs.umich.edu/courses/eecs489/w07 Z. Morley Mao Wednesday Jan 17, 2007 1 Mao W07 Acknowledgement: Some slides taken from KuroseRoss and KatzStoicaƒ Adminstrivia Homework 1 was assigned, due 1/23 To be completed individually 2 Mao W07ƒƒ ƒ Principles of network applications Our goals: learn about protocols by examining popular conceptual, applicationlevel implementation protocols aspects of network application protocols HTTP FTP transportlayer SMTP / POP3 / IMAP service models DNS clientserver paradigm programming network applications peertopeer paradigm socket API 3 Mao W07ƒƒƒ ƒƒƒƒƒƒƒ Some network apps Email Internet telephone Web Realtime video conference Instant messaging Massive parallel Remote login computing P2P file sharing Multiuser network games Streaming stored video clips What’s your favorite network application 4 Mao W07ƒƒ Creating a network application application Write programs that transport network run on different end data link physical systems and communicate over a network. e.g., Web: Web server software communicates with browser software No software written application application for devices in network transport transport network core network data link data link physical physical Network core devices do not function at app layer This design allows for rapid app development 5 Mao W07ƒƒƒ Application architectures Clientserver Peertopeer (P2P) Hybrid of clientserver and P2P What is the key difference 6 Mao W07Clientserver architecture server: alwayson host permanent IP address server farms for scaling • Question: how do server farms still maintain a single IP address externally clients: communicate with server may be intermittently connected may have dynamic IP addresses do not communicate directly with each other 7 Mao W07ƒƒƒƒ Pure P2P architecture no always on server arbitrary end systems directly communicate peers are intermittently connected and change IP addresses example: Gnutella Highly scalable Why But difficult to manage 8 Mao W07Hybrid of clientserver and P2P Napster File transfer P2P File search centralized: • Peers register content at central server • Peers query same central server to locate content Instant messaging Chatting between two users is P2P Presence detection/location centralized: • User registers its IP address with central server when it comes online • User contacts central server to find IP addresses of buddies 9 Mao W07ƒ ƒƒ Processes communicating Client process: process Process: program running that initiates within a host. communication within same host, two Server process: process processes communicate that waits to be using interprocess contacted communication (defined Q: does it have to have a fixed by OS). port processes in different Note: applications with hosts communicate by P2P architectures have exchanging messages client processes server processes 10 Mao W07ƒƒ ƒ Sockets host or host or process sends/receives server server messages to/from its socket controlled by app developer process socket analogous to door process sending process shoves socket socket message out of door TCP with TCP with Internet buffers, sending process relies on buffers, variables variables transport infrastructure on other side of door which brings message to socket at controlled receiving process by OS API: (1) choice of transport protocol; (2) ability to fix a few parameters 11 Mao W07ƒƒƒ ƒƒ Addressing processes For a process to Identifier includes receive messages, it both the IP address must have an and port numbers identifier associated with the A host has a unique process on the host. 32bit IP address Example port Q: does the IP numbers: address of the host on HTTP server: 80 which the process Mail server: 25 runs suffice for identifying the process Have you heard of “port knocking” 12 Mao W07ƒƒƒƒ ƒƒƒƒ Applicationlayer protocol defines Types of messages Publicdomain exchanged, e.g., protocols: request response defined in RFCs messages allows for Syntax of message types: what fields in interoperability messages how eg, HTTP, SMTP fields are delineated Proprietary protocols: Semantics of the eg, KaZaA fields, i.e., meaning of information in fields Rules for when and how processes send respond to messages What’s the advantage/disadvantage of proprietary protocols 13 Mao W07ƒƒ ƒ ƒƒ What transport service does an app need Data loss Bandwidth some apps (e.g., audio) can some apps (e.g., tolerate some loss multimedia) require minimum amount of other apps (e.g., file bandwidth to be transfer, telnet) require “effective” 100 reliable data transfer other apps (“elastic apps”) make use of Timing whatever bandwidth they some apps (e.g., get Internet telephony, interactive games) require low delay to be “effective” 14 Mao W07Transport service requirements of common apps Time Sensitive Application Bandwidth Data loss no file transfer elastic no loss no email no Web documents yes, 100’s msec realtime audio/video audio: 5kbps1Mbps losstolerant video:10kbps5Mbps yes, few secs stored audio/video same as above yes, 100’s msec interactive games few kbps up yes and no instant messaging elastic 15 Mao W07ƒƒ ƒƒƒƒƒ Internet transport protocol services UDP service: TCP service: unreliable data transfer connectionoriented: setup between sending and required between client and server processes receiving process reliable transport between does not provide: sending and receiving connection setup, reliability, process flow control, congestion flow control: sender won’t control, timing, or overwhelm receiver bandwidth guarantee congestion control: throttle sender when network Q: why bother Why is there a overloaded UDP does not provide: timing, minimum bandwidth guarantees What other properties are desirable What combination of properties are desirable 16 Mao W07Internet apps: application, transport protocols Application Underlying Application layer protocol transport protocol email SMTP RFC 2821 TCP remote terminal access Telnet RFC 854 Web HTTP RFC 2616 file transfer FTP RFC 959 TCP streaming multimedia proprietary (e.g. RealNetworks) Internet telephony proprietary (e.g., Dialpad) 17 Mao W07ƒƒƒƒƒ Web and HTTP First some jargon Web page consists of objects Object can be HTML file, JPEG image, Java applet, audio file,… Web page consists of base HTMLfile which includes several referenced objects Each object is addressable by a URL Example URL: www.someschool.edu/someDept/pic.gif host name path name Have you heard of “PageRank” 18 Mao W07ƒƒƒƒ HTTP request HTTP response HTTP overview HTTP: hypertext transfer protocol Web’s application layer PC running protocol Explorer client/server model client: browser that requests, receives, “displays” Web objects Server running server: Web server Apache Web sends objects in server response to requests HTTP 1.0: RFC 1945 Mac running HTTP 1.1: RFC 2068 Navigator 19 Mao W07 HTTP request H TP r s o s T e p n eƒ ƒƒ ƒƒƒƒ HTTP overview (continued) HTTP is “stateless” Uses TCP: server maintains no client initiates TCP information about connection (creates socket) past client requests to server, port 80 server accepts TCP aside connection from client Protocols that maintain “state” are HTTP messages complex (applicationlayer protocol past history (state) must be messages) exchanged maintained between browser (HTTP if server/client crashes, their client) and Web server views of “state” may be (HTTP server) inconsistent, must be reconciled TCP connection closed Is it better to have a stateful protocol 20 Mao W07ƒƒ ƒƒ HTTP connections Nonpersistent HTTP Persistent HTTP At most one object is Multiple objects can sent over a TCP be sent over a single connection. TCP connection between client and HTTP/1.0 uses server. nonpersistent HTTP HTTP/1.1 uses persistent connections in default mode 21 Mao W07Nonpersistent HTTP Suppose user enters URL (contains text, www.someSchool.edu/someDepartment/home.index references to 10 jpeg images) 1a. HTTP client initiates a TCP 1b. HTTP server at host connection to HTTP server www.someSchool.edu waiting (process) at www.someSchool.edu for TCP connection at port 80. on port 80 “accepts” connection, notifying client 2. HTTP client sends HTTP 3. HTTP server receives request request message (containing message, forms response URL) into TCP connection message containing requested socket. Message indicates that object, and sends message into client wants object its socket someDepartment/home.index time 22 Mao W07Nonpersistent HTTP (cont.) 4. HTTP server closes TCP connection. 5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects time 6. Steps 15 repeated for each of 10 jpeg objects 23 Mao W07ƒƒƒ Response time modeling Definition of RTT: time to send a small packet to travel from client to server initiate TCP and back. connection RTT Response time: request one RTT to initiate TCP file time to connection RTT transmit one RTT for HTTP request file file and first few bytes of received HTTP response to return file transmission time time time total = 2RTT+transmit time 24 Mao W07ƒƒƒƒƒ ƒƒƒƒƒ Persistent HTTP Persistent without pipelining: Nonpersistent HTTP issues: client issues new request requires 2 RTTs per object only when previous OS must work and allocate response has been received host resources for each TCP one RTT for each referenced connection object but browsers often open Persistent with pipelining: parallel TCP connections to fetch referenced objects default in HTTP/1.1 Persistent HTTP client sends requests as soon as it encounters a server leaves connection referenced object open after sending responses as little as one RTT for all subsequent HTTP messages the referenced objects between same client/server are sent over connection Several dimensions to help speed up: Persistent connections, pipelining, parallel connections 25 Mao W07ƒƒ HTTP request message two types of HTTP messages: request, response HTTP request message: ASCII (humanreadable format) request line (GET, POST, GET /somedir/page.html HTTP/1.1 HEAD commands) Host: www.someschool.edu Useragent: Mozilla/4.0 header Connection: close lines Acceptlanguage:fr Carriage return, (extra carriage return, line feed) line feed indicates end of message 26 Mao W07HTTP request message: general format 27 Mao W07ƒƒ ƒƒ Uploading form input Post method: Web page often URL method: includes form input Uses GET method Input is uploaded to server in entity body Input is uploaded in URL field of request line: www.somesite.com/animalsearchmonkeysbanana 28 Mao W07ƒƒƒ ƒƒƒ Method types HTTP/1.0 HTTP/1.1 GET GET, POST, HEAD POST PUT uploads file in entity body HEAD to path specified in URL asks server to leave field requested object out of response DELETE deletes file specified in the URL field 29 Mao W07HTTP response message status line (protocol HTTP/1.1 200 OK status code Connection close status phrase) Date: Thu, 06 Aug 1998 12:00:15 GMT Server: Apache/1.3.0 (Unix) header LastModified: Mon, 22 Jun 1998 …... lines ContentLength: 6821 ContentType: text/html data, e.g., data data data data data ... requested HTML file 30 Mao W07HTTP response status codes In first line in serverclient response message. A few sample codes: 200 OK request succeeded, requested object later in this message 301 Moved Permanently requested object moved, new location specified later in this message (Location:) 400 Bad Request request message not understood by server 404 Not Found requested document not found on this server 505 HTTP Version Not Supported 31 Mao W07Userserver state: cookies Many major Web sites Example: use cookies Susan access Internet always from same PC Four components: She visits a specific e 1) cookie header line in commerce site for first the HTTP response time message When initial HTTP 2) cookie header line in requests arrives at site, HTTP request message site creates a unique 3) cookie file kept on ID and creates an entry user’s host and in backend database managed by user’s for ID browser 4) backend database at Web site 32 Mao W07entry in backend database Cookies: keeping “state” (cont.) client server usual http request msg Cookie file server usual http response + creates ID ebay: 8734 Setcookie: 1678 1678 for user Cookie file usual http request msg cookie amazon: 1678 cookie: 1678 ebay: 8734 specific usual http response msg action one week later: usual http request msg Cookie file cookie cookie: 1678 amazon: 1678 spectific ebay: 8734 usual http response msg action 33 Mao W07 access accessƒƒƒƒ ƒƒƒƒ Cookies (continued) aside Cookies and privacy: What cookies can bring: cookies permit sites to authorization learn a lot about you shopping carts you may supply name recommendations and email to sites user session state (Web search engines use email) redirection cookies to learn yet more advertising companies obtain info across sites Do cookies compromise security Can it be used for authentication 34 Mao W07ƒƒ HTTP request HTTP response Web caches (proxy server) Goal: satisfy client request without involving origin server user sets browser: origin Web accesses via server cache Proxy browser sends all server HTTP requests to client cache object in cache: cache returns object else cache requests object from origin server, then returns object to client client origin server 35 Mao W07 HTTP request HTTP response HTTP request HTTP responseƒƒƒ ƒƒ More about Web caching Cache acts as both Why Web caching client and server Reduce response time for client request. Typically cache is Reduce traffic on an installed by ISP institution’s access link. (university, company, Internet dense with caches residential ISP) enables “poor” content providers to effectively deliver content (but so does P2P file sharing) 36 Mao W07ƒƒƒƒƒƒ Caching example Assumptions origin average object size = 100,000 bits servers avg. request rate from institution’s public browsers to origin servers = 15/sec Internet delay from institutional router to any origin server and back to router = 2 sec 1.5 Mbps Consequences access link utilization on LAN = 15 institutional utilization on access link = 100 network 10 Mbps LAN total delay = Internet delay + access delay + LAN delay = 2 sec + minutes + milliseconds institutional cache 37 Mao W07ƒƒƒƒƒ Caching example (cont) origin Possible solution servers increase bandwidth of public access link to, say, 10 Internet Mbps Consequences 10 Mbps utilization on LAN = 15 access link utilization on access link = 15 institutional Total delay = Internet delay + network 10 Mbps LAN access delay + LAN delay = 2 sec + msecs + msecs often a costly upgrade institutional cache 38 Mao W07ƒƒƒƒƒ Caching example (cont) origin servers Install cache suppose hit rate is 0.4 public Internet Consequence 40 requests will be satisfied almost immediately 60 requests satisfied by origin 1.5 Mbps server access link utilization of access link reduced institutional to 60, resulting in negligible network 10 Mbps LAN delays (say 10 msec) total avg delay = Internet delay + access delay + LAN delay = .6(2.01) secs + milliseconds 1.4 secs institutional cache 39 Mao W07ƒƒƒ Conditional GET server cache Goal: don’t send object if cache has uptodate cached HTTP request msg version Ifmodifiedsince: object date cache: specify date of cached not copy in HTTP request modified Ifmodifiedsince: HTTP response HTTP/1.0 date 304 Not Modified server: response contains no object if cached copy is upto date: HTTP request msg HTTP/1.0 304 Not Ifmodifiedsince: date object Modified modified HTTP response HTTP/1.0 200 OK data 40 Mao W07ƒƒƒƒ FTP: the file transfer protocol file transfer FTP FTP FTP user client server interface user remote file local file at host system system transfer file to/from remote host client/server model client: side that initiates transfer (either to/from remote) server: remote host ftp: RFC 959 ftp server: port 21 41 Mao W07ƒƒƒ ƒƒƒƒƒ FTP: separate control, data connections TCP control connection port 21 FTP client contacts FTP server at port 21, specifying TCP as transport protocol TCP data connection Client obtains authorization FTP FTP port 20 over control connection client server Client browses remote directory by sending Server opens a second TCP data commands over control connection to transfer another file. connection. Control connection: “out of band” When server receives a FTP server maintains “state”: command for a file transfer, current directory, earlier the server opens a TCP authentication data connection to client After transferring one file, server closes connection. What’s the advantage of an outofband control channel 42 Mao W07ƒƒƒƒƒ ƒƒƒƒƒƒ FTP commands, responses Sample commands: Sample return codes sent as ASCII text over status code and phrase control channel (as in HTTP) USER username 331 Username OK, password required PASS password 125 data connection LIST return list of file in already open; current directory transfer starting RETR filename 425 Can’t open data retrieves (gets) file connection 452 Error writing STOR filename stores file (puts) file onto remote host 43 Mao W07ƒƒƒƒƒƒƒ outgoing Electronic Mail message queue user mailbox user Three major components: agent user agents mail user server mail servers agent simple mail transfer protocol: SMTP mail SMTP server user agent User Agent SMTP a.k.a. “mail reader” SMTP composing, editing, reading user mail mail messages agent server e.g., Eudora, Outlook, elm, Netscape Messenger user outgoing, incoming messages agent stored on server user agent 44 Mao W07ƒƒƒ Electronic Mail: mail servers user Mail Servers agent mailbox contains incoming mail user messages for user server agent message queue of SMTP mail outgoing (to be sent) mail server user messages agent SMTP SMTP protocol between mail servers to send email SMTP messages user mail agent client: sending mail server server user “server”: receiving mail agent server user agent Where can we find out the mail servers for a domain 45 Mao W07ƒƒƒƒƒ Electronic Mail: SMTP RFC 2821 uses TCP to reliably transfer email message from client to server, port 25 direct transfer: sending server to receiving server three phases of transfer handshaking (greeting) transfer of messages closure command/response interaction commands: ASCII text response: status code and phrase messages must be in 7bit ASCII 46 Mao W07Scenario: Alice sends message to Bob 4) SMTP client sends Alice’s 1) Alice uses UA to compose message over the TCP message and “to” connection bobsomeschool.edu 5) Bob’s mail server places the 2) Alice’s UA sends message to message in Bob’s mailbox her mail server; message 6) Bob invokes his user agent to placed in message queue read message 3) Client side of SMTP opens TCP connection with Bob’s mail server 1 mail mail user server user server agent 2 agent 6 3 4 5 47 Mao W07Sample SMTP interaction S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: alicecrepes.fr S: 250 alicecrepes.fr... Sender ok C: RCPT TO: bobhamburger.edu S: 250 bobhamburger.edu ... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup C: How about pickles C: . S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection 48 Mao W07ƒƒƒ Try SMTP interaction for yourself: telnet servername 25 see 220 reply from server enter HELO, MAIL FROM, RCPT TO, DATA, QUIT commands above lets you send email without using email client (reader) 49 Mao W07ƒƒƒƒƒ ƒƒƒ SMTP: final words SMTP uses persistent Comparison with HTTP: connections HTTP: pull SMTP requires SMTP: push message (header both have ASCII body) to be in 7bit command/response ASCII interaction, status codes SMTP server uses HTTP: each object CRLF.CRLF to encapsulated in its own determine end of response msg message SMTP: multiple objects sent in multipart msg 50 Mao W07ƒƒ Mail message format SMTP: protocol for exchanging email msgs header blank RFC 822: standard for text line message format: header lines, e.g., To: From: body Subject: different from SMTP commands body the “message”, ASCII characters only 51 Mao W07ƒƒ Message format: multimedia extensions MIME: multimedia mail extension, RFC 2045, 2056 additional lines in msg header declare MIME content type From: alicecrepes.fr MIME version To: bobhamburger.edu Subject: Picture of yummy crepe. method used MIMEVersion: 1.0 to encode data ContentTransferEncoding: base64 ContentType: image/jpeg multimedia data type, subtype, base64 encoded data ..... parameter declaration ......................... ......base64 encoded data encoded data 52 Mao W07ƒƒ Mail access protocols SMTP SMTP access user user agent protocol agent receiver’s mail sender’s mail server server SMTP: delivery/storage to receiver’s server Mail access protocol: retrieval from server POP: Post Office Protocol RFC 1939 • authorization (agent server) and download IMAP: Internet Mail Access Protocol RFC 1730 • more features (more complex) • manipulation of stored msgs on server HTTP: Hotmail , Yahoo Mail, etc. 53 Mao W07ƒƒƒƒƒƒ POP3 protocol S: +OK POP3 server ready authorization phase C: user bob client commands: S: +OK user: declare username C: pass hungry S: +OK user successfully logged on pass: password server responses C: list S: 1 498 +OK S: 2 912 ERR S: . transaction phase, client: C: retr 1 S: message 1 contents list: list message numbers S: . retr: retrieve message by C: dele 1 number C: retr 2 dele: delete S: message 1 contents quit S: . C: dele 2 C: quit S: +OK POP3 server signing off 54 Mao W07ƒƒƒ ƒƒƒƒ POP3 (more) and IMAP More about POP3 IMAP Previous example uses Keep all messages in “download and delete” one place: the server mode. Allows user to organize Bob cannot reread e messages in folders mail if he changes client IMAP keeps user state “Downloadandkeep”: across sessions: copies of messages on names of folders and different clients mappings between message IDs and folder POP3 is stateless name across sessions 55 Mao W07ƒƒ DNS: Domain Name System Domain Name System: People: many identifiers: distributed database SSN, name, passport implemented in hierarchy of Internet hosts, routers: many name servers IP address (32 bit) used applicationlayer protocol host, for addressing routers, name servers to datagrams communicate to resolve names (address/name translation) “name”, e.g., ww.yahoo.com used by note: core Internet function, humans implemented as application layer protocol complexity at network’s “edge” 56 Mao W07ƒƒƒƒ ƒƒƒƒ DNS Why not centralize DNS DNS services single point of failure Hostname to IP address translation traffic volume Host aliasing distant centralized database Canonical and alias names maintenance Mail server aliasing Load distribution doesn’t scale Replicated Web servers: set of IP addresses for one canonical name 57 Mao W07ƒƒƒ Distributed, Hierarchical Database Root DNS Servers org DNS servers edu DNS servers com DNS servers poly.edu umass.edu pbs.org yahoo.com amazon.com DNS serversDNS servers DNS servers DNS servers DNS servers st Client wants IP for www.amazon.com; 1 approx: Client queries a root server to find com DNS server Client queries com DNS server to get amazon.com DNS server Client queries amazon.com DNS server to get IP address for www.amazon.com 58 Mao W07ƒƒ DNS: Root name servers contacted by local name server that can not resolve name root name server: contacts authoritative name server if name mapping not known gets mapping returns mapping to local name server a Verisign, Dulles, VA c Cogent, Herndon, VA (also Los Angeles) d U Maryland College Park, MD k RIPE London (also Amsterdam, g US DoD Vienna, VA Frankfurt) i Autonomica, Stockholm (plus 3 h ARL Aberdeen, MD other locations) j Verisign, ( 11 locations) m WIDE Tokyo e NASA Mt View, CA f Internet Software C. Palo Alto, 13 root name servers CA (and 17 other locations) worldwide b USCISI Marina del Rey, CA l ICANN Los Angeles, CA 59 Mao W07ƒƒ TLD and Authoritative Servers Toplevel domain (TLD) servers: responsible for com, org, net, edu, etc, and all toplevel country domains uk, fr, ca, jp. Network solutions maintains servers for com TLD Authoritative DNS servers: organization’s DNS servers, providing authoritative hostname to IP mappings for organization’s servers (e.g., Web and mail). Can be maintained by organization or service provider 60 Mao W07ƒƒƒ Local Name Server Does not strictly belong to hierarchy Each ISP (residential ISP, company, university) has one. Also called “default name server” When a host makes a DNS query, query is sent to its local DNS server Acts as a proxy, forwards query into hierarchy. 61 Mao W07ƒ root DNS server Example 2 Host at cis.poly.edu wants 3 TLD DNS server IP address for 4 gaia.cs.umass.edu 5 local DNS server dns.poly.edu 6 7 1 8 authoritative DNS server dns.cs.umass.edu requesting host cis.poly.edu gaia.cs.umass.edu 62 Mao W07ƒƒƒƒ Recursive queries root DNS server recursive query: puts burden of name 2 3 resolution on contacted name server 6 7 heavy load TLD DNS server iterated query: contacted server replies local DNS server with name of server to 4 dns.poly.edu 5 contact “I don’t know this name, 1 8 but ask this server” authoritative DNS server dns.cs.umass.edu requesting host cis.poly.edu gaia.cs.umass.edu 63 Mao W07ƒƒ DNS: caching and updating records once (any) name server learns mapping, it caches mapping cache entries timeout (disappear) after some time TLD servers typically cached in local name servers • Thus root name servers not often visited update/notify mechanisms under design by IETF RFC 2136 http://www.ietf.org/html.charters/dnsindcharter.html 64 Mao W07ƒ ƒ ƒ ƒ DNS records DNS: distributed db storing resource records (RR) RR format: (name, value, type, ttl) Type=A Type=CNAME name is hostname name is alias name for some “cannonical” (the real) name value is IP address www.ibm.com is really Type=NS servereast.backup2.ibm.com name is domain (e.g. value is cannonical name foo.com) value is IP address of authoritative name server Type=MX for this domain value is name of mailserver associated with name 65 Mao W07ƒƒ DNS protocol, messages DNS protocol : query and reply messages, both with same message format msg header identification: 16 bit for query, reply to query uses same flags: query or reply recursion desired recursion available reply is authoritative 66 Mao W07DNS protocol, messages Name, type fields for a query RRs in reponse to query records for authoritative servers additional “helpful” info that may be used 67 Mao W07ƒƒƒƒ Inserting records into DNS Example: just created startup “Network Utopia” Register name networkuptopia.com at a registrar (e.g., Network Solutions) Need to provide registrar with names and IP addresses of your authoritative name server (primary and secondary) Registrar inserts two RRs into the com TLD server: (networkutopia.com, dns1.networkutopia.com, NS) (dns1.networkutopia.com, 212.212.212.1, A) Put in authoritative server Type A record for www.networkuptopia.com and Type MX record for networkutopia.com How do people get the IP address of your Web site 68 Mao W07