Transcript Lecture 1: Course Introduction and Overview

CS194-3/CS16x
Introduction to Systems
Lecture 11
Address Translation, Caches
October 3, 2007
Prof. Anthony D. Joseph
http://www.cs.berkeley.edu/~adj/cs16x
Review: Important Aspects of Memory Multiplexing
• Controlled overlap:
– Separate state of threads should not collide in physical
memory. Obviously, unexpected overlap causes chaos!
– Conversely, would like the ability to overlap when
desired (for communication)
• Translation:
– Ability to translate accesses from one address space
(virtual) to a different one (physical)
– When translation exists, processor uses virtual
addresses, physical memory uses physical addresses
– Side effects:
» Can be used to avoid overlap
» Can be used to give uniform view of memory to programs
• Protection:
– Prevent access to private memory of other processes
10/3/07
» Different pages of memory can be given special behavior
(Read Only, Invisible to user programs, etc).
» Kernel data protected from User programs
» Programs protected from themselves
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.2
Goals for Today
• Address Translation Schemes
–
–
–
–
–
Segmentation
Paging
Multi-level translation
Paged page tables
Inverted page tables
• Comparison among options
• Caching
Note: Some slides and/or pictures in the following are
adapted from slides ©2005 Silberschatz, Galvin, and Gagne.
Slides courtesy of Kubiatowicz, AJ Shankar, George Necula,
Alex Aiken, Eric Brewer, Ras Bodik, Ion Stoica, Doug Tygar,
and David Wagner.
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.3
Simple Segmentation: Base and Bounds (CRAY-1)
CPU
Base
Virtual
Address
Limit
>?
+
Physical
Address
DRAM
Yes: Error!
• Can use base & bounds/limit for dynamic address
translation (Simple form of “segmentation”):
– Alter every address by adding “base”
– Generate error if address bigger than limit
• This gives program the illusion that it is running on its
own dedicated machine, with memory starting at 0
– Program gets continuous region of memory
– Addresses within program do not have to be relocated
when program placed in different region of DRAM
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.4
Base and Limit segmentation discussion
• Provides level of indirection
– OS can move bits around behind program’s back
– Can be used to correct if program needs to grow
beyond its bounds or coalesce fragments of memory
• Only OS gets to change the base and limit!
– Would defeat protection
• What gets saved/restored on a context switch?
– Everything from before + base/limit values
– Or: How about complete contents of memory (out to
disk)?
» Called “Swapping”
• Hardware cost
– 2 registers/Adder/Comparator
– Slows down hardware because need to take time to do
add/compare on every access
• Base and Limit Pros: Simple, relatively fast
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.5
Cons for Simple Segmentation Method
• Fragmentation problem (complex memory allocation)
– Not every process is the same size
– Over time, memory space becomes fragmented
– Really bad if want space to grow dynamically (e.g. heap)
process 6
process 6
process 6
process 6
process 5
process 5
process 5
process 5
process 9
process 9
process 2
OS
process 10
OS
OS
OS
• Other problems for process maintenance
– Doesn’t allow heap and stack to grow independently
– Want to put these as far apart in virtual memory space
as possible so that they can grow as needed
• Hard to do inter-process sharing
– Want to share code segments when possible
– Want to share memory between processes
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.6
More Flexible Segmentation
1 1
4
1
2
3
2 2
4
3
user view of
memory space
• Logical View: multiple separate segments
physical
memory space
– Typical: Code, Data, Stack
– Others: memory sharing, etc
• Each segment is given region of contiguous memory
– Has a base and limit
– Can reside anywhere
in physical memory
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.7
Implementation of Multi-Segment Model
Virtual
Address
Seg # Offset
Base0
Base1
Base2
Base3
Base4
Base5
Base6
Base7
Limit0
Limit1
Limit2
Limit3
Limit4
Limit5
Limit6
Limit7
V
V
V
N
V
N
N
V
>
Error
+
Physical
Address
• Segment map resides in processor
– Segment number mapped into base/limit pair
– Base added to offset to generate physical address
– Error check catches offset out of range
• As many chunks of physical memory as entries
– Segment addressed by portion of virtual address
– However, could be included in instruction instead:
» x86 Example: mov [es:bx],ax.
• What is “V/N”?
– Can mark segments as invalid; requires check as well
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.8
Example: Four Segments (16 bit addresses)
Seg Offset
15 14 13
0
Virtual Address Format
Seg ID #
0 (code)
1 (data)
2 (shared)
3 (stack)
0x0000
0x0000
0x4000
0x4000
0x4800
0x5C00
Base
0x4000
0x4800
0xF000
0x0000
Limit
0x0800
0x1400
0x1000
0x3000
Might
be shared
0x8000
Space for
Other Apps
0xC000
0xF000
10/3/07
Virtual
Address Space
Physical
Address Space
Joseph CS194-3/16x ©UCB Fall 2007
Shared with
Other Apps
Lec 11.9
Example of segment translation
0x240
0x244
…
0x360
0x364
0x368
…
0x4050
main:
strlen:
loop:
varx
la $a0, varx
jal strlen
…
li
$v0, 0 ;count
lb
$t0, ($a0)
beq $r0,$t1, done
…
dw
0x314159
Seg ID #
0 (code)
1 (data)
2 (shared)
3 (stack)
Base
0x4000
0x4800
0xF000
0x0000
Limit
0x0800
0x1400
0x1000
0x3000
Let’s simulate a bit of this code to see what happens (PC=0x240):
•
Fetch 0x240. Virtual segment #? 0; Offset? 0x240
Physical address? Base=0x4000, so physical addr=0x4240
Fetch instruction at 0x4240. Get “la $a0, varx”
Move 0x4050  $a0, Move PC+4PC
2. Fetch 0x244. Translated to Physical=0x4244. Get “jal strlen”
Move 0x0248  $ra (return address!), Move 0x0360  PC
3. Fetch 0x360. Translated to Physical=0x4360. Get “li $v0,0”
Move 0x0000  $v0, Move PC+4PC
4. Fetch 0x364. Translated to Physical=0x4364. Get “lb $t0,($a0)”
Since $a0 is 0x4050, try to load byte from 0x4050
Translate 0x4050. Virtual segment #? 1; Offset? 0x50
Physical address? Base=0x4800, Physical addr = 0x4850,
Load Byte from 0x4850$t0, Move PC+4PC
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.10
Observations about Segmentation
• Virtual address space has holes
– Segmentation efficient for sparse address spaces
– A correct program should never address gaps (except
as mentioned in moment)
» If it does, trap to kernel and dump core
• When it is OK to address outside valid range:
– This is how the stack and heap are allowed to grow
– For instance, stack takes fault, system automatically
increases size of stack
• Need protection mode in segment table
– For example, code segment would be read-only
– Data and stack would be read-write (stores allowed)
– Shared segment could be read-only or read-write
• What must be saved/restored on context switch?
– Segment table stored in CPU, not in memory (small)
– Might store all of processes memory onto disk when
switched (called “swapping”)
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.11
Administrivia
• Office hours this week
– 413 Soda: Wed 12:30-1:30pm and Thu 1-2pm
• Midterm #1 is Thursday 10/4 in 405 Soda (5:30-7pm)
–
–
–
–
–
Food and drinks will be provided (pizza)
Old CS 162 exams posted
No notes, book, calculator, etc.
Covers up to Monday’s lecture (10/1)
History, Concurrency, Multithreading, Synchronization,
ACID, Intro to DBs, Protection/Address Spaces
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.12
Schematic View of Swapping
• Extreme form of Context Switch: Swapping
– In order to make room for next process, some or all
of the previous process is moved to disk
» Likely need to send out complete segments
– This greatly increases the cost of context-switching
• Desirable alternative?
– Some way to keep only active portions of a process in
memory at any one time
– Need finer granularity control over physical memory
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.13
Paging: Physical Memory in Fixed Size Chunks
• Problems with segmentation?
– Must fit variable-sized chunks into physical memory
– May move processes multiple times to fit everything
– Limited options for swapping to disk
• Fragmentation: wasted space
– External: free gaps between allocated chunks
– Internal: don’t need all memory within allocated chunks
• Solution to fragmentation from segments?
– Allocate physical memory in fixed size chunks (“pages”)
– Every chunk of physical memory is equivalent
» Can use simple vector of bits to handle allocation:
00110001110001101 … 110010
» Each bit represents page of physical memory
1allocated, 0free
• Should pages be as big as our previous segments?
– No: Can lead to lots of internal fragmentation
» Typically have small pages (1K-16K)
– Consequently: need multiple pages/segment
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.14
How to Implement Paging?
Virtual Address:
Virtual
Page #
Offset
>
page
page
page
page
page
page
PageTablePtr
PageTableSize
Access
Error
#0
#1
#2
#3
#4
#5
V,R
V,R
Physical
Page # Offset
V,R,W
Physical Address
V,R,W
Check Perm
N
V,R,W
• Page Table (One per process)
Access
Error
– Resides in physical memory
– Contains physical page and permission for each virtual page
» Permissions include: Valid bits, Read, Write, etc
• Virtual address mapping
– Offset from Virtual address copied to Physical Address
» Example: 10 bit offset  1024-byte pages
– Virtual page # is all remaining bits
» Example for 32-bits: 32-10 = 22 bits, i.e. 4 million entries
» Physical page # copied from table into physical address
– Check Page Table bounds and permissions
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.15
What about Sharing?
Virtual Address
(Process A):
Virtual
Page #
PageTablePtrA
PageTablePtrB
Virtual Address:
Process B
10/3/07
Virtual
Page #
Offset
page
page
page
page
page
page
#0
#1
#2
#3
#4
#5
V,R
V,R
V,R,W
V,R,W
page
page
page
page
page
page
#0
#1
#2
#3
#4
#5
V,R
N
V,R,W
Shared
Page
N
V,R,W
V,R,W
N
V,R
This physical page
appears in address
space of both processes
V,R,W
Offset
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.16
Simple Page Table Discussion
• What needs to be switched on
a context switch?
0x00 a
b
c
0x04 d
e
f
g
0x08 h
i
j
k
l
Virtual
Memory
– Page table pointer and limit
0x00
4
0x04 i
3
1
0x08
j
k
l
Page 0x0C e
Table
f
g
h
0x10 a
b
c
d
Physical
Memory
Example (4 byte pages)
10/3/07
• Analysis
– Pros
» Simple memory allocation
» Easy to Share
– Con: What if address space is
sparse?
» E.g. on UNIX, code starts at
0, stack starts at (231-1).
» With 1K pages, need 2 million
page table entries!
– Con: What if table really big?
» Not all pages used all the
time  would be nice to have
working set of page table in
memory
• How about combining paging
and segmentation?
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.17
Multi-level Translation
• What about a tree of tables?
– Lowest level page tablememory still allocated with bitmap
– Higher levels often segmented
• Could have any number of levels. Example (top segment):
Virtual
Address:
Virtual
Seg #
Base0
Base1
Base2
Base3
Base4
Base5
Base6
Base7
Virtual
Page #
Limit0
Limit1
Limit2
Limit3
Limit4
Limit5
Limit6
Limit7
V
V
V
N
V
N
N
V
Offset
page
page
page
page
page
page
>
#0
#1
#2
#3
#4
#5
V,R
V,R
V,R,W
V,R,W
V,R,W
Physical
Page # Offset
Physical Address
N
V,R,W
Access
Error
Check Perm
Access
Error
• What must be saved/restored on context switch?
– Contents of top-level segment registers (for this example)
– Pointer to top-level table (page table)
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.18
What about Sharing (Complete Segment)?
Process
A
Virtual
Seg #
Base0
Base1
Base2
Base3
Base4
Base5
Base6
Base7
Process
B
10/3/07
Virtual
Page #
Limit0
Limit1
Limit2
Limit3
Limit4
Limit5
Limit6
Limit7
Virtual
Seg #
V
V
V
N
V
N
N
V
Virtual
Page #
Offset
page
page
page
page
page
page
#0
#1
#2
#3
#4
#5
V,R
V,R
V,R,W
V,R,W
N
V,R,W
Shared Segment
Base0
Base1
Base2
Base3
Base4
Base5
Base6
Base7
Limit0
Limit1
Limit2
Limit3
Limit4
Limit5
Limit6
Limit7
V
V
V
N
V
N
N
V
Offset
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.19
Another common example: two-level page table
Virtual
Address:
10 bits
10 bits
12 bits
Virtual
Virtual
P1 index P2 index Offset
Physical
Address:
Physical
Page #
Offset
4KB
PageTablePtr
4 bytes
• Tree of Page Tables
• Tables fixed size (1024 entries)
– On context-switch: save single
PageTablePtr register
• Valid bits on Page Table Entries
– Don’t need every 2nd-level table
– Even when exist, 2nd-level tables
can reside on disk if not in use
10/3/07
4 bytes
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.20
Multi-level Translation Analysis
• Pros:
– Only need to allocate as many page table entries as we
need for application
» In other words, sparse address spaces are easy
– Easy memory allocation
– Easy Sharing
» Share at segment or page level (need additional reference
counting)
• Cons:
– One pointer per page (typically 4K – 16K pages today)
– Page tables need to be contiguous
» However, previous example keeps tables to exactly one
page in size
– Two (or more, if >2 levels) lookups per reference
» Seems very expensive!
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.21
Inverted Page Table
• With all previous examples (“Forward Page Tables”)
– Size of page table is at least as large as amount of
virtual memory allocated to processes
– Physical memory may be much less
» Much of process space may be out on disk or not in use
Virtual
Page #
Offset
Hash
Table
Physical
Page # Offset
• Answer: use a hash table
–
–
–
–
Called an “Inverted Page Table”
Size is independent of virtual address space
Directly related to amount of physical memory
Very attractive option for 64-bit address spaces
• Cons: Complexity of managing hash changes
– Often in hardware!
Joseph CS194-3/16x ©UCB Fall 2007
10/3/07
Lec 11.22
BREAK
What is in a Page Table Entry?
• What is in a Page Table Entry (or PTE)?
– Pointer to next-level page table or to actual page
– Permission bits: valid, read-only, read-write, execute-only
• Example: Intel x86 architecture PTE:
– Address same format previous slide (10, 10, 12-bit offset)
– Intermediate page tables called “Directories”
P:
W:
U:
PWT:
PCD:
A:
D:
L:
10/3/07
31-12
Free
(OS)
11-9
0 L D A
PWT
PCD
Page Frame Number
(Physical Page Number)
U WP
8 7 6 5 4 3 2 1 0
Present (same as “valid” bit in other architectures)
Writeable
User accessible
Page write transparent: external cache write-through
Page cache disabled (page cannot be cached)
Accessed: page has been accessed recently
Dirty (PTE only): page has been modified recently
L=14MB page (directory only).
Bottom 22 bits of virtual address serve as offset
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.24
Examples of how to use a PTE
• How do we use the PTE?
– Invalid PTE can imply different things:
» Region of address space is actually invalid or
» Page/directory is just somewhere else than memory
– Validity checked first
» OS can use other (say) 31 bits for location info
• Usage Example: Demand Paging
– Keep only active pages in memory
– Place others on disk and mark their PTEs invalid
• Usage Example: Copy on Write
– UNIX fork gives copy of parent address space to child
» Address spaces disconnected after child created
– How to do this cheaply?
» Make copy of parent’s page tables (point at same memory)
» Mark entries in both sets of page tables as read-only
» Page fault on write creates two copies
• Usage Example: Zero Fill On Demand
– New data pages must carry no information (say be zeroed)
– Mark PTEs as invalid; page fault on use gets zeroed page
– Often, OS creates zeroed pages in background
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.25
How is the translation accomplished?
CPU
Virtual
Addresses
MMU
Physical
Addresses
• What, exactly happens inside MMU?
• One possibility: Hardware Tree Traversal
– For each virtual address, takes page table base pointer
and traverses the page table in hardware
– Generates a “Page Fault” if it encounters invalid PTE
» Fault handler will decide what to do
» More on this next lecture
– Pros: Relatively fast (but still many memory accesses!)
– Cons: Inflexible, Complex hardware
• Another possibility: Software
– Each traversal done in software
– Pros: Very flexible
– Cons: Every translation must invoke Fault!
• In fact, need way to cache translations for either case!
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.26
Caching Concept
• Cache: a repository for copies that can be accessed
more quickly than the original
– Make frequent case fast and infrequent case less dominant
• Caching underlies many of the techniques that are used
today to make computers fast
– Can cache: memory locations, address translations, pages,
file blocks, file names, network routes, etc…
• Only good if:
– Frequent case frequent enough and
– Infrequent case not too expensive
• Important measure: Average Access time =
(Hit Rate x Hit Time) + (Miss Rate x Miss Time)
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.27
Why Bother with Caching?
Processor-DRAM Memory Gap (latency)
“Moore’s Law”
(really Joy’s Law)
100
10
1
“Less’ Law?”
µProc
60%/yr.
(2X/1.5yr)
Processor-Memory
Performance Gap:
(grows 50% / year)
DRAM
DRAM
9%/yr.
(2X/10
yrs)
CPU
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
Performance
1000
Time
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.28
Another Major Reason to Deal with Caching
Virtual
Address:
Virtual
Seg #
Base0
Base1
Base2
Base3
Base4
Base5
Base6
Base7
Virtual
Page #
Limit0
Limit1
Limit2
Limit3
Limit4
Limit5
Limit6
Limit7
V
V
V
N
V
N
N
V
Offset
page
page
page
page
page
page
>
#0
#1
#2
#3
#4
#5
V,R
V,R
V,R,W
V,R,W
Physical
Page # Offset
Physical Address
N
V,R,W
Access
Error
Check Perm
Access
Error
• Too expensive to translate on every access
– At least two DRAM accesses per actual DRAM access
– Or: perhaps I/O if page table partially on disk!
• Even worse problem: What if we are using caching to
make memory access faster than DRAM access???
• Solution? Cache translations!
– Translation Cache: TLB (“Translation Lookaside Buffer”)
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.29
Why Does Caching Help? Locality!
Probability
of reference
0
2n - 1
Address Space
• Temporal Locality (Locality in Time):
– Keep recently accessed data items closer to processor
• Spatial Locality (Locality in Space):
– Move contiguous blocks to the upper levels
To Processor
Upper Level
Memory
Lower Level
Memory
Blk X
From Processor
10/3/07
Blk Y
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.30
Memory Hierarchy of a Modern Computer System
• Take advantage of the principle of locality to:
– Present as much memory as in the cheapest technology
– Provide access at speed offered by the fastest technology
Processor
Control
1s
Size (bytes): 100s
10/3/07
On-Chip
Cache
Speed (ns):
Registers
Datapath
Second
Level
Cache
(SRAM)
Main
Memory
(DRAM)
10s-100s
100s
Ks-Ms
Ms
Secondary
Storage
(Disk)
Tertiary
Storage
(Tape)
10,000,000s 10,000,000,000s
(10s ms)
(10s sec)
Joseph CS194-3/16x ©UCB Fall 2007
Gs
Ts
Lec 11.31
Jim Gray’s Storage Latency Analogy:
How Far Away is the Data?
109
Andromeda
Tape /Optical
Robot
10 6 Disk
100
10
2
1
10/3/07
2,000 Years
Pluto
Sacramento
Memory
On Board Cache
On Chip Cache
Registers
2 Years
1.5 hr
This Lecture Hall
10 min
This Room
My Head 1 min
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.32
Where does a Block Get Placed in a Cache?
• Example: Block 12 placed in 8 block cache
32-Block Address Space:
Block
no.
Block
no.
10/3/07
1111111111222222222233
01234567890123456789012345678901
Direct mapped:
Set associative:
Fully associative:
block 12 can go
only into block 4
(12 mod 8)
block 12 can go
anywhere in set 0
(12 mod 4)
block 12 can go
anywhere
01234567
Block
no.
01234567
Block
no.
Set Set Set Set
0 1 2 3
Joseph CS194-3/16x ©UCB Fall 2007
01234567
Lec 11.33
A Summary on Sources of Cache Misses
• Compulsory (cold start): first reference to a block
– “Cold” fact of life: not a whole lot you can do about it
– Note: When running “billions” of instruction, Compulsory
Misses are insignificant
• Capacity:
– Cache cannot contain all blocks access by the program
– Solution: increase cache size
• Conflict (collision):
– Multiple memory locations mapped to same cache location
– Solutions: increase cache size, or increase associativity
• Two others:
– Coherence (Invalidation): other process (e.g., I/O)
updates memory
– Policy: Due to non-optimal replacement policy
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.34
How is a Block found in a Cache?
Block Address
Tag
Index
Block
offset
Set Select
Data Select
• Index Used to Lookup Candidates in Cache
– Index identifies the set
• Tag used to identify actual copy
– If no candidates match, then declare cache miss
• Block is minimum quantum of caching
– Data select field used to select data within block
– Many caching applications don’t have data select field
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.35
Summary (1/2)
• Segment Mapping
– Segment registers within processor
– Segment ID associated with each access
» Often comes from portion of virtual address
» Can come from bits in instruction instead (x86)
– Each segment contains base and limit information
» Offset (rest of address) adjusted by adding base
• Page Tables
– Memory divided into fixed-sized chunks of memory
– Virtual page number from virtual address mapped
through page table to physical page number
– Offset of virtual address same as physical address
– Large page tables can be placed into virtual memory
• Multi-Level Tables
– Virtual address mapped to series of tables
– Permit sparse population of address space
• Inverted page table
– Size of page table related to physical memory size
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.36
Summary (2/2)
• The Principle of Locality:
– Program likely to access a relatively small portion of the
address space at any instant of time.
» Temporal Locality: Locality in Time
» Spatial Locality: Locality in Space
• Three (+1) Major Categories of Cache Misses:
– Compulsory Misses: sad facts of life. Example: cold start
misses.
– Conflict Misses: increase cache size and/or associativity
– Capacity Misses: increase cache size
– Coherence Misses: Caused by external processors or I/O
devices
• Cache Organizations:
– Direct Mapped: single block per set
– Set associative: more than one block per set
– Fully associative: all entries equivalent
10/3/07
Joseph CS194-3/16x ©UCB Fall 2007
Lec 11.37