Transcript Lectures for 2nd Edition
Chapters 8 Storage, Networks, and Other Peripherals
2004 Morgan Kaufmann Publishers
1
2004 Morgan Kaufmann Publishers
2
Outline
• • • • • • • • • • •
8.1
8.2
8.3
8.4
Introduction Disk Storage and Dependability Networks Buses and Other Connections between Processors, Memory, and I/O Devices 8.5
8.6
8.7
8.8
Interfacing I/O Devices to the Processor, Memory, and Operating System I/O Performance Measures: Examples from Disk and File Systems Designing an I/O System Real Stuff: A Digital Camera 8.9
Fallacies and Pitfalls 8.10 Concluding Remarks 8.11 Historical Perspective and Further Reading
2004 Morgan Kaufmann Publishers
3
8.1
Introduction
2004 Morgan Kaufmann Publishers
4
Keywords
•
I/O requests Reads or writes to I/O devices
2004 Morgan Kaufmann Publishers
5
Interfacing Processors and Peripherals
• • •
I/O Design affected by many factors (expandability, resilience) Performance: — access latency — throughput — connection between devices and the system — the memory hierarchy — the operating system A variety of different users (e.g., banks, supercomputers, engineers)
Interrupts Processor Cache Memory- I/O bus Main memory I/O controller Disk Disk I/O controller Graphics output I/O controller Network 2004 Morgan Kaufmann Publishers
6
I/O
•
Important but neglected “The difficulties in assessing and designing I/O systems have often relegated I/O to second class status” “courses in every aspect of computing, from programming to computer architecture often ignore I/O or give it scanty coverage” “textbooks leave the subject to near the end, making it easier for students and instructors to skip it!”
•
GUILTY!
— we won’t be looking at I/O in much detail — be sure and read Chapter 8 in its entirety.
— you should probably take a networking class!
2004 Morgan Kaufmann Publishers
7
I/O Devices
•
Very diverse devices — behavior (i.e., input vs. output) — partner (who is at the other end?) — data rate
2004 Morgan Kaufmann Publishers
8
Figure 8.2
The diversity of I/O devices.
Device Keyboard Mouse Voice input Sound input Scanner Voice output Sound output Laser printer Graphics display Modem Network/LAN Network/wireless LAN Optical disk Magnetic tape Magnetic disk Behavior Input Input Input Input Input Output Output Output Output Input or output Input or output Input or output Storage Storage storage Partner Human Human Human machine Human Human Human Human Human Machine Machine Machine Machine Machine machine Data rate (Mbit/sec) 0.0001
0.0038
0.2640
3.0000
3.2000
0.2640
8.0000
3.2000
800.0000-8000.0000
0.0160-0.0640
100.0000-1000.0000
11.0000-54.0000
80.0000
32.0000
240.0000-2560.0000
2004 Morgan Kaufmann Publishers
9
8.2
Disk Storage and Dependability
2004 Morgan Kaufmann Publishers
10
Keywords
• • • • • •
Nonvolatile
Storage device where data retains its value even when power is removed.
Track
One of thousands of concentric circles that makes up the surface of a magnetic disk.
Sector
One of the segments that make up a track on a magnetic disk; a sector is the smallest amount of information that is read or written on a disk.
Seek
The process of positioning a read/write head over the proper track on a disk.
Rotation latency
Also called delay. The time required for the desired sector of a disk to rotate under the read/write head; usually assumed to be half the rotation time.
Small computer systems interface (SCSI)
for I/O devices.
A bus used as a standard 2004 Morgan Kaufmann Publishers
11
Keywords
• • • • • •
Redundant arrays of inexpensive disks (RAID)
An organization of disks that uses an array of small and inexpensive disks so as to increase both performance and reliability.
Striping
Allocation of logically sequential blocks to separate disk to allow higher performance than a single disk can deliver.
Mirroring
availability.
Writing the identical data to multiple disks to increase data
Protection group
The group of data disks or blocks that share a common check disk or block.
Hot swapping
running.
Replacing a hardware component while the system is
Standby spares
Reserve hardware resources that can immediately take the place of a failed component.
2004 Morgan Kaufmann Publishers
12
I/O Example: Disk Drives
Platters Tracks Platter Track Sectors •
To access data: — seek: position head over the proper track (3 to 14 ms. avg.) — rotational latency: wait for desired sector (.5 / RPM) — transfer: grab the data (one or more sectors) 30 to 80 MB/sec
2004 Morgan Kaufmann Publishers
13
Disk Read Time
•
Q
：
What is the average time to read or write a 512-byte sector for a typical disk rotating at 10,000 RPM? The advertised average seek time is 6 ms, the transfer rate is 50 MB/sec, and the controller overhead is 0.2 ms. Assume that the disk is idle so that there is no waiting time.
•
A
： 6 .
0 ms 0 .
5 rotation 10 , 000 RPM 0 .
5 KB 50 MB/sec 0 .
2 ms 6 .
0 3 .
0 0 .
01 0 .
2 9 .
2 ms If the measured average seek time is 25% of the advertised average time. The answer is 1.5ms + 3.0ms + 0.01ms + 0.2ms = 4.7ms
2004 Morgan Kaufmann Publishers
14
Figure 8.3
Six magnetic disks, varying in diameter from 14 inches down to 1.8 inches.
2004 Morgan Kaufmann Publishers
15
Figure 8.4
Characteristics of three magnetic disks by a single manufacturer in 2004.
Characteristics
Disk diameter (inches) Formatted data capacity (GB) Number of disk surfaces (heads) Rotation speed (PRM) Internal disk cache size (MB) 15,000 8 External interface, bandwidth (MB/sec) Ultra320 SCSI, 320 Sustained transfer rate (MB/sec) 57-86 Minimum seek (read/write) (ms) Average seek read/write (ms)
Seagate ST373453
3.50
73.4
8 0.2/0.4
3.6/3.9
Seagate ST3200822
3.50
200.0
4 7200 8 Serial ATA, 150 32-58 1.0/1.2
8.5/9.5
Seagate ST94811A
2.50
40.0
2 5400 8 ATA, 100 34 1.5/2.0
12.0/14.0
2004 Morgan Kaufmann Publishers
16
Mean time to failure (MTTF) (hours) Warranty (years) Nonrecoverable read errors per bits read Temperature, vibration limits (operating) Size: dimensions (in.), weight (pounds) Power: operating/idle/standby (watts) GB/cu. In., GB/watt 1,200,000@25 ℃ 5 <1 per 10 15 5-55 ℃ , [email protected]
1.0”X4.0”X5.8”, 1.9 lbs 20?/12/ — 3GB/cu.in., 4GB/W 600,000@25 ℃ 3 <1 per 10 14 0-60 ℃ , [email protected]
1.0”X4.0”X5.8”, 1.4 lbs 12/8/1 9GB/cu.in., 16GB/W Price in 2004, $/GB $400, $5/GB $100, $0.5/GB 330,000@25 ℃ — <1 per 10 14 5-55 ℃ , 400Hz@1G 0.4”X2.7”X3.9”, 0.2 lbs 2.4/1.0/0.4
10GB/cu.in., 17GB/W $100, $2.50/GB 2004 Morgan Kaufmann Publishers
17
Figure 8.5 Summary of studies of reasons for failures.
Operator 42% 15% 18% 50% 50% 54% 60% Software 25% 55% 44% 20% 14% 7% 25% Hardware 18% 14% 39% 30% 19% 30% 15% System Data center (Tandem) Data center (Tandem) Data center (DEC VAX) Data center (DEC VAX) U.S. public telephone network U.S. public telephone network Internet services Year data collected 1985 1989 1985 1993 1996 2000 2002
2004 Morgan Kaufmann Publishers
18
Figure 8.6
RAID for an example of four data disks showing extra check disks per RAID level and companies that use each level.
2004 Morgan Kaufmann Publishers
19
Figure 8.7
Small write update on RAID 3 versus RAID4
2004 Morgan Kaufmann Publishers
20
Figure 8.8
Block-interleaved parity (RAID 4) versus distributed block-interleaved parity (RAID 5)
2004 Morgan Kaufmann Publishers
21
8.3
Networks
2004 Morgan Kaufmann Publishers
22
Networks are growing in popularity over time, and unlike other I/O devices, there are many books and courses on them. For readers who have not taken courses or read books on networking, Section 8.3 on the CD gives a quick overview of the topics and terminology, including internetworking, the OSI model, protocol families such as TCP/IP, long-haul networks such as ATM, local area networks such as Ethernet, and wireless networks such as IEEE 802.11.
2004 Morgan Kaufmann Publishers
23
8.4
Buses and Other Connections between Processors, Memory, and I/O Devices
2004 Morgan Kaufmann Publishers
24
Keywords
• • • • •
Bus transaction
A sequence of bus operations that includes a request and may includes a request and may include a response, either of which may carry data. A transaction is initiated by a single request and may take many individual bus operations.
Processor-memory bus
A bus that connects processor and memory and that is short, generally high speed, and matched to the memory system so as to maximize memory-processor bandwidth.
Backplane bus
A bus that is designed to allow processors, memory. And I/O devices to coexist on a single bus.
Synchronous bus
A bus that includes a clock in the control lines and a fixed protocol for communicating that is relative to the clock.
Asynchronous bus
A bus that uses a handshaking protocol for coordinating usage rather than a clock; can accommodate a wide variety of devices of differing speeds.
2004 Morgan Kaufmann Publishers
25
Keywords
• •
Handshaking protocol
A series of steps used to coordinate asynchronous bus transfers in which the sender and receiver proceed to the next step only when both parties agree that the current step has been completed.
Split transaction protocol
A protocol in which the bus is released during a bus transaction while the requester is waiting for the data to be transmitted, which frees the bus for access by another requester. 2004 Morgan Kaufmann Publishers
26
I/O Example: Buses
• • • •
Shared communication link (one or more wires) Difficult design: — may be bottleneck — length of the bus — number of devices — tradeoffs (buffers for higher bandwidth increases latency) — support for many different devices — cost Types of buses: — processor-memory (short high speed, custom design) — backplane (high speed, often standardized, e.g., PCI) — I/O (lengthy, different devices, e.g., USB, Firewire) Synchronous vs. Asynchronous — use a clock and a synchronous protocol, fast and small but every device must operate at same rate and clock skew requires the bus to be short — don’t use a clock and instead use handshaking
2004 Morgan Kaufmann Publishers
27
I/O Bus Standards
•
Today we have two dominant bus standards:
2004 Morgan Kaufmann Publishers
28
Figure 8.10
The asynchronous handshaking protocol consists of seven steps to read a word from memory and receive it in an I/O device.
2004 Morgan Kaufmann Publishers
29
2004 Morgan Kaufmann Publishers
30
Figure 8.11
Organization of the I/O system on a Pentium 4 PC using the intel 875 chip
Main memory DIMMs Disk DDR 400 (3.2 GB/sec) DDR 400 (3.2 GB/sec) Serial ATA (150 MB/sec) Pentium 4 processor System bus (800 MHz, 604 GB/sec) Memory controller hub (north bridge) 82875P AGP 8X (2.1 GB/sec) CSA (0.266 GB/sec) Graphics output 1 Gbit Ethernet (266 MB/sec) Parallel ATA (100 MB/sec) CD/DVD Serial ATA (150 MB/sec) Disk AC/97 (1 MB/sec) Stereo (surround sound) USB 2.0
(60 MB/sec) . . .
I/O controller hub (south bridge) 82801EB Parallel ATA (100 MB/sec) (20 MB/sec) PCI bus (132 MB/sec) Tape 10/100 Mbit Ethernet 2004 Morgan Kaufmann Publishers
31
Figure 8.12
Two Pentium 4 I/O chip sets from Intel.
2004 Morgan Kaufmann Publishers
32
8.5
Interfacing I/O Devices to the Processor, Memory, and Operating System
2004 Morgan Kaufmann Publishers
33
Keywords
• • • • • •
Memory-mapped I/O
An I/O scheme in which portions of address space are assigned to I/O devices and reads and writes to those addresses are interpreted as commands to the I/O device.
I/O instructions
A dedicated instruction that is used to give a command to an I/O device and that specifies both the device number and the command word (or the location of the command word in memory).
Polling
The process of periodically checking the status of an I/O device to determine the need to service the device.
Interrupted-driven I/O
An I/O scheme that employs interrupts to indicate to the processor that an I/O device needs attention.
Direct memory access (DMA)
A mechanism that provides a device controller the ability to transfer data directly to or from the memory without involving the processor.
Bus master
A unit on the bus that can initiate bus requests.
2004 Morgan Kaufmann Publishers
34
Other important issues
• • • •
Bus Arbitration: — daisy chain arbitration (not very fair) — centralized arbitration (requires an arbiter), e.g., PCI — collision detection, e.g., Ethernet Operating system: — polling — interrupts — direct memory access (DMA) Performance Analysis techniques: — queuing theory — simulation — analysis, i.e., find the weakest link (see “I/O System Design”) Many new developments
2004 Morgan Kaufmann Publishers
35
Figure 8.13
The Cause and Status registers.
2004 Morgan Kaufmann Publishers
36
8.6
I/O Performance Measures
：
Examples from Disk and File Systems
2004 Morgan Kaufmann Publishers
37
Keywords
• • •
Transaction processing
A type of application that involves handling small short operations (called transactions) that typically require both I/O and computation. Transaction processing applications typically have both response time requirements and a performance measurement based on the throughput of transactions.
I/O rate
second.
Performance measure of I/Os per unit time. Such as reads per
Data rate
GB/second.
Performance measure of bytes per unit time, such as 2004 Morgan Kaufmann Publishers
38
Impact of I/O on system performance
•
Q
：
Suppose we have a benchmark that executes in 100 seconds of elapsed time, where 90 seconds is CPU time and the rest is I/O time. If CPU time improves by 50% per year for the next five years but I/O time doesn’t improve, how much faster will our program run at the end of five years?
•
A
： We know that Elapsed time = CPU time + I/O time 100 = 90 + I/O time I/O time = 10 seconds The new CPU times and the resulting elapsed times are computed in the following table ：
39
After n years CPU time I/O time Elapsed time % I/O time 0 90 seconds 10 seconds 100 seconds 10 % 1 2 3 4 5
90
60 seconds 10 seconds
1 .
5 60
40 seconds 10 seconds
1 .
5 40
27 seconds 10 seconds
1 .
5 27
18 seconds 10 seconds
1 .
5 18 1 .
5
12 seconds 10 seconds 70 seconds 50 seconds 37 seconds 28 seconds 22 seconds
90 The improvement in CPU performance over five years is However, the improvement in elapsed time is only 100 22 12 4 .
5
14 % 20 % 27 % 36 % 45 %
7 .
5 and the I/O time has increased from 10% to 45% of the elapsed time.
2004 Morgan Kaufmann Publishers
40
8.7
Designing an I/O System
2004 Morgan Kaufmann Publishers
41
I/O System Design
•
Q
：
Consider the following computer system
： – A CPU that sustain 3 billion instructions per second and averages 100,000 instruction in the operating system per I/O operation – A memory backplane bus capable of sustaining a transfer rate of 1000 MB/sec – SCSI Ultra320 controllers with a transfer rate of 320 MB/sec and accommodating up to 7 disks – Disk drives with a read/write bandwidth of 75 MB/sec and an average seek plus rotational latency of 6 ms.
If the workload consists of 64 KB reads (where the block is sequential on a track) and the user program needs 200,000 instructions per I/O operation, find the maximum sustainable I/O rate and the number of disks and SCSI controllers required. Assume that the reads can always be done on an idle disk if one exists (i.e., ignore disk conflicts).
2004 Morgan Kaufmann Publishers
42
•
A
： Maximum I/O rate of CPU Instrucito n Instructio execution ns per I/O rate 3 10 9 ( 200 100 ) 10 3 I/Os 10 , 000 second Each I/O transfer 64 KB, so Maximum I/O rate of bus Bus bandwidth Bytes per I/O 1000 10 6 64 10 3 15 , 625 I/Os second Time per I/O at disk Seek rotation t ime Transfer t ime 6 ms 64 KB 75 MB/sec Transfer rate Transfer size Transfer t ime 6 .
9 ms 6 .
9 ms 64 KB 9 .
56 MB/sec The maximum number of disks per SCSI bus is 7, which won’t saturate this bus. This means we will need 69/7, or 10 SCSI buses and controllers.
2004 Morgan Kaufmann Publishers
43
8.8
Real Stuff
：
A Digital Camera
2004 Morgan Kaufmann Publishers
44
Figure 8.14
The Sanyo VPC-SX500 with Flash memory card and IBM Microdrive.
2004 Morgan Kaufmann Publishers
45
Figure 8.15
Characteristics of three storage alternatives for digital cameras.
Characteristics Formatted data capacity (MB) Bytes per sector Data transfer rate (MB/sec) Link speed to buffer (MB/sec) Power standby/operating (W) Size: height X width X depth (inches) Weight in grams (454 grams/pound) Write cycles before sector wear-out Mean time between failures (hours) Best price (2004) Sandisk Type I compactFlash SDCFB-128-768
128 512 4 (burst) 6 0.15/0.66
1.43 X 1.68 X 0.13
11.4
300,000 > 1,000,000 $40
Sandisk Type II compactFlash SDCFB-1000-768
1000 512 4 (burst) 6 0.15/0.66
1.43 X 1.68 X 0.13
13.5
300,000 > 1,000,000 $200
Hitachi 4 GB Microdrive DSCM-10340
4000 512 4 – 7 33 0.07/0.83
1.43 X 1.68 X 0.16
16 Not applicable (see caption) $480 2004 Morgan Kaufmann Publishers
46
Figure 8.16
The system on a chip (SOC) found in Sanyo digital cameras.
2004 Morgan Kaufmann Publishers
47
8.9
Fallacies and Pitfalls
2004 Morgan Kaufmann Publishers
48
Fallacies and Pitfalls
• • • •
Fallacy
: the rated mean time to failure of disks is 1,200,000 hours, so disks practically never fail.
Fallacy
: magnetic disk storage is on its last legs, will be replaced.
Fallacy
: A 100 MB/sec bus can transfer 100 MB/sec.
Pitfall
: Moving functions from the CPU to the I/O processor, expecting to improve performance without analysis.
2004 Morgan Kaufmann Publishers
49
Multiprocessors
•
Idea: create powerful computers by connecting many smaller ones good news: works for timesharing (better than supercomputer) bad news: its really hard to write good concurrent programs many commercial failures
Processor Processor Cache Cache Single bus Memory Processor Cache I/O Processor Processor Cache Cache Memory Memory Network Processor Cache Memory 2004 Morgan Kaufmann Publishers
50
Questions
•
How do parallel processors share data?
— single address space (SMP vs. NUMA) — message passing
•
How do parallel processors coordinate? — synchronization (locks, semaphores) — built into send / receive primitives — operating system protocols
•
How are they implemented?
— connected by a single bus — connected by a network
2004 Morgan Kaufmann Publishers
51
Supercomputers
Plot of top 500 supercomputer sites over a decade:
500 Single Instruction multiple data (SIMD) 400 Cluster (network of workstations) Cluster (network of SMPs) 300 200 100 0 93 93 94 94 95 95 96 96 97 97 98 98 99 99 00 Uniprocessors Massively parallel processors (MPPs) Shared memory multiprocessors (SMPs) 2004 Morgan Kaufmann Publishers
52
Using multiple processors an old idea
•
Some SIMD designs:
•
Costs for the the Illiac IV escalated from $8 million in 1966 to $32 million in 1972 despite completion of only ¼ of the machine. It took three more years before it was operational! “For better or worse, computer architects are not easily discouraged” Lots of interesting designs and ideas, lots of failures, few successes
2004 Morgan Kaufmann Publishers
53
Topologies
a. 2-D grid or mesh of 16 nodes b. n-cube tree of 8 nodes (8 = 23 so n = 3) P0 P1 P2 P3 P4 P5 P6 P7 a. Crossbar P0 P1 P2 P3 P4 P5 P6 P7 b. Omega network 2004 Morgan Kaufmann Publishers
54
Clusters
• • • • • •
Constructed from whole computers Independent, scalable networks Strengths:
–
Many applications amenable to loosely coupled machines
– –
Exploit local area networks Cost effective / Easy to expand Weaknesses:
–
Administration costs not necessarily lower
–
Connected using I/O bus Highly available due to separation of memories In theory, we should be able to do better
2004 Morgan Kaufmann Publishers
55
• • • • •
Serve an average of 1000 queries per second Google uses 6,000 processors and 12,000 disks Two sites in silicon valley, two in Virginia Each site connected to internet using OC48 (2488 Mbit/sec) Reliability:
–
On an average day, 20 machines need rebooted (software error)
–
2% of the machines replaced each year In some sense, simple ideas well executed. Better (and cheaper) than other approaches involving increased complexity
2004 Morgan Kaufmann Publishers
56
Concluding Remarks
•
Evolution vs. Revolution “More often the expense of innovation comes from being too disruptive to computer users” “Acceptance of hardware ideas requires acceptance by software people; therefore hardware people should learn about software. And if software people want good machines, they must learn more about hardware to be able to communicate with and thereby influence hardware engineers.”
2004 Morgan Kaufmann Publishers