Transcript Interconnection networks 2, clusters

ENGS 116 Lecture 20
1
Interconnection Networks Cont’d
Vincent H. Berk
November 16, 2005
Reading for Friday: 8.1-8.7
Reading for Monday: 8.8-8.13
Homework for Friday 18th: 5.4, 5.17,
6.4, 6.10, 7.3, 7.21, 8.9/8.10, 8.17
ENGS 116 Lecture 20
2
Connection-Based vs. Connectionless
• Telephone: operator sets up connection between the caller and the
receiver
– Once the connection is established, conversation can continue for
hours
• Share transmission lines over long distances by using switches to
multiplex several conversations on the same lines
– “Time division multiplexing” divide B/W transmission line into a
fixed number of slots, with each slot assigned to a conversation
• Problem: lines busy based on number of conversations, not amount of
information sent
• Advantage: reserved bandwidth
ENGS 116 Lecture 20
Connection-Based vs. Connectionless
• Connectionless: every package of information must have an
address  packets
– Each package is routed to its destination by looking at its
address
– Analogy, the postal system (sending a letter)
– Also called “Statistical multiplexing”
• Each packet requires a new/separate routing decision
• Depending on implementation the switching stations may
also be called routers.
3
ENGS 116 Lecture 20
4
Routing Messages
• Within a network:
– Shared media:
• Broadcast to everyone
• Internetwork routing. Options:
– Source-based routing: message specifies path to the destination
(changes of direction)
– Virtual circuit: circuit established from source to destination,
message picks the circuit to follow
– Destination-based routing: message specifies destination, switch
must pick the path: deterministic vs. non-deterministic
• deterministic: always follow same path
• adaptive: pick different paths to avoid congestion, failures
• randomized routing: pick between several good paths to balance
network load
ENGS 116 Lecture 20
5
Deterministic Routing Examples
• mesh: dimension-order routing
– (x1, y1)  (x2, y2)
– first x = x2 – x1,
– then y = y2 – y1,
• hypercube: edge-cube routing
– X = xox1x2 . . .xn  Y = yoy1y2 . . .yn
– R = X xor Y
– Traverse dimensions of differing
address in order
• tree: common ancestor
• Deadlock free?
110
010
111
011
100
000
001
101
ENGS 116 Lecture 20
Store and Forward vs. Cut-Through
• Store-and-forward policy: each switch waits for the full packet to
arrive in switch before sending to the next switch (good for WAN)
• Cut-through routing or wormhole routing: switch examines the
header, decides where to send the message, and then starts
forwarding it immediately
– In wormhole routing, when head of message is blocked, message
stays strung out over the network, potentially blocking other
messages (needs only buffer the piece of the packet that is sent
between switches). CM-5 uses it, with each switch buffer being
4 bits per port.
– Cut-through routing lets the tail continue when head is blocked,
“accordioning” the whole message into a single switch.
(Requires a buffer large enough to hold the largest packet).
6
ENGS 116 Lecture 20
7
Congestion Control
• Packet switched networks do not reserve bandwidth; this leads to
contention (connection-based limits input)
• Solution: prevent packets from entering until contention is reduced
(e.g., freeway on-ramp metering lights)
• Options:
– Packet discarding: If packet arrives at switch and no room in buffer,
packet is discarded (e.g., UDP)
– Flow control: between pairs of receivers and senders;
use feedback to tell sender when allowed to send next packet
• Back-pressure: separate wires to tell to stop
• Window: give original sender right to send N packets before getting
permission to send more; overlaps latency of interconnection with overhead
to send & receive packet (e.g., TCP), adjustable window
– Choke packets: aka “rate-based”; each packet received by busy
switch in warning state sent back to the source via choke packet.
Source reduces traffic to that destination by a fixed % (e.g., ATM,
ICMP source quench)
ENGS 116 Lecture 20
Practical Issues for Interconnection Networks
• Standardization advantages:
– low cost (components used repeatedly)
– stability (many suppliers to chose from)
• Standardization disadvantages:
– Time for committees to agree
– When to standardize?
• Before anything built?  Committee does design?
• Too early suppresses innovation
• Perfect interconnect vs. Fault Tolerant?
– Will SW crash on single node prevent communication?
(MPP typically assumes perfect)
• Reliability (vs. availability) of interconnect
• Most successful system is not always the best design.
8
ENGS 116 Lecture 20
9
Practical Issues
Interconnection
Example
Standard
Fault Tolerance?
Hot Insert?
MPP
CM-5
No
No
No
LAN
Ethernet
Yes
Yes
Yes
WAN
ATM
Yes
Yes
Yes
• Standards: required for WAN, LAN!
• Fault Tolerance: Can nodes fail and still deliver messages to other
nodes? Required for WAN, LAN!
• Hot Insert: If the interconnection can survive a failure, can it also
continue operation while a new node is added to the interconnection?
Required for WAN, LAN!
ENGS 116 Lecture 20
10
Inter-Network-Routing
• Connecting >2 networks together.
• Requires:
– Addressing Hierarchy
– Common Protocols
– Courtesy and Security
• Each step in a route (hop) decides:
– What first?
– Where next?
• Transparent or explicit
ENGS 116 Lecture 20
11
Bridging (transparent routing)
ENGS 116 Lecture 20
12
OSI model
• This one has to be in every network presentation
7. Application
Web browser
6. Presentation
Network library interface
5. Session
TCP
4. Transport
IP
3. Network
Packet
2. Data Link
Ethernet Frame
1. Physical
Electrical signals
ENGS 116 Lecture 20
Networking Protocols: HW/SW Interface
• Internetworking: allows computers on independent and incompatible
networks to communicate reliably and efficiently;
– Enabling technologies: SW standards that allow reliable
communications without reliable networks
– Hierarchy of SW layers, giving each layer responsibility for
portion of overall communications task, called
protocol families or protocol suites
• Transmission Control Protocol/Internet Protocol (TCP/IP)
– This protocol family is the basis of the Internet
– IP makes best effort to deliver; TCP “guarantees” delivery
– TCP/IP used even when communicating locally: NFS uses IP
even though communicating across homogeneous LAN
13
ENGS 116 Lecture 20
14
Protocol
Message
Logical
Message
Actual
H
T
H
T
Actual
H
T
Logical
H
T
H
Actual
T T
H H
T T
T
H
T
Actual
H H
T T
H H
T T
H H
T T
H H
T T
Actual
• Key to protocol families is that communication occurs logically at the
same level of the protocol, called peer-to-peer, but is implemented via
services at the lower level
• Danger is each level increases latency if implemented as hierarchy
(e.g., multiple check sums)
ENGS 116 Lecture 20
15
IP, TCP, and UDP
• IP = internet protocol, used at network layer
– IP routes datagrams to destination machine, makes best effort to deliver
packets but does not guarantee delivery or order of datagrams
– For IP, every host and router must have unique IP address
• IPv4 uses 32-bit addresses
• IPv6 uses 16-byte addresses (not that straight forward, though!!!)
• TCP = transmission control protocol, used at transport layer
– TCP is connection-oriented, makes guarantee of reliable, in-order delivery
– Up to 4 retries on failure to deliver (or acknowledge!)
• UDP = user data protocol, used at transport layer
– Connectionless protocol, makes no guarantees of delivery
ENGS 116 Lecture 20
16
Packet Formats
CM-5
Route
T L
Data (4 - 20)
C
32 bits
Ethernet
ATM
Preamble
Preamble
Destination
Destination
Source
Source
Length
Destination
C
Data (48)
Data (0 - 1500)
Pad (0 -46)
Checksum
32 bits
32 bits
• Fields: Destination, Checksum (C), Length (L), Type (T)
• Data/Header Sizes in bytes: (4 to 20)/4, (0 to 1500)/26, 48/5
T
ENGS 116 Lecture 20
17
Networking Summary
• Protocols allow heterogeneous networking
• Protocols allow operation in the presence of failures
• Routing issues: store and forward vs. cut-through, congestion, ...
• Standardization key for LAN, WAN
• Internetworking protocols used as LAN protocols  large overhead
for LAN
• Integrated circuit revolutionizing networks as well as processors
• Switch is a specialized computer
ENGS 116 Lecture 20
18
Cluster (Multicomputer)
• A collection of low-cost nodes connected by a fast network.
• Applications:
–
–
–
–
Less synchronization required than for MP applications
Less need for communication
No need for one large homogeneous memory
Many copies of one application run in parallel
• Each node:
– cheap
– redundant
• Easily expandable
– Scales if the software application scales
ENGS 116 Lecture 20
19
Possible Applications
• Distributed Database
– Each node works as the query engine for data on local disk(s)
– All nodes together implement redundancy:
• Failure of 1 or more nodes doesn’t damage the database
• Scientific applications:
– Nuclear or Oceanographic simulations
– Diskless nodes. Each node uses NFS (of similar SAN-based
system) to access central data repository.
– Applications are started over the network.
– Think SETI@home
ENGS 116 Lecture 20
20
Book Example: google cluster
•
•
•
•
•
1000’s of cheap PC’s with a very fast network connection.
Most of bandwidth is used in keeping database updated.
How are queries handled? Distributed?
How is their database constructed?
What is the algorithm used?
• Paper: “The Anatomy of a Large-Scale Hypertextual Web
Search Engine” by Sergey Brin and Lawrence Page