CPE 626 CPU Resources: ARM Cache Memories Aleksandar Milenkovic

E-mail: Web: [email protected]

http://www.ece.uah.edu/~milenka

On-chip RAM

  On-chip memory is essential if a processor is to deliver its best performance    zero wait state access speed power efficiency reduced electromagnetic interference In many embedded systems simple on-chip RAM is preferred to a cache   Advantages o o simpler, cheaper, less power more deterministic behavior Disadvantages o require explicit control by the programmer 2

Unified instruction and data cache

FF..FF

16 registers

processor

address copies of instructions copies of data

cache

instructions and data address instructions and data instructions data

memory

00..00

16 3

Separate data and instruction caches

copies of instructions address instructions

cache

address instructions instructions registers

processor

address data address data copies of data

cache

data

memory

FF..FF

16 00..00

16 4

Direct-mapped cache organization

address tag index tag RAM data RAM compare hit mux data 5

Two-way set-associative cache organization

address tag tag RAM index data RAM compare compare hit mux mux data tag RAM data RAM 6

Fully associative cache

address tag CAM data RAM hit mux data 7

An Example

  ARM3 designed in 1989 was the first to incorporate an on-chip cache Design steps    analysis using ARM2: collect hardware traces running typical benchmarks exploring the upper-bound performance benefit considering a perfect cache (always contains the requested data) o Assuming 20MHz cache and 8MHz main memory performance of various systems is: No cache – 1, Instr. only – 1.95, Data only – 1.13, Instr. + data cache – 2.5

investigate different cache organizations and sizes o write-back, write-through, write-allocate, write-no-allocate, replacement policies, associativity, power 8

Summary of cache organizational options

Org an i z at i o n al f e at ure Cac h e - MMU re l at i o n s h i p Cac h e c o n t e n t s As s o c i at i v i t y Re p l ac e me n t s t rat e g y Wri t e s t rat e g y

Physical cache Unified instruction and data cache Direct-mapped RAM-RAM Cyclic Write-through

Op t i o n s

Virtual cache Separate instruction and data caches Set-associative RAM-RAM Random Write-through with write buffer Fully associative CAM-RAM LRU Copy-back 9

Unified cache performance as a function of size and organization

rela tive pe rforma nce 2.5

2 1.5

1 0.5

0 16 4 cache s ize (Kbyte s) 1 1/4 2-way asso cia tive direct-ma pped 10

The effect of associativity on performance and bandwidth requirement

2.5

2 1.5

1 0.5

0 1 2 4 associativity (ways) 8 16 32 64 128 256 perform ance bandwid th 11

ARM3 cache organization

31 us er/su perviso r tag CAM tag CAM 64-way, 4KB cache 6 5 4 3 2 1 0 virtual address decode en able tag CAM tag CAM [9 :0] [1 :0] [3 :2] byte addres ses data RAM 64 entry 64 entry 64 entry 64 entry [9 :4] 1024 x 32-bit word hit data 12

ARM600 cache control state machine

     After initialization, the processor enters state If address is non-sequential, does not fault in MMU, and is either read hit of a buffered write, the state machine remains in the 

data value is read or written every clock cycle

When the next address is sequential read in the same cache line or a sequential buffered write, the state moves to where the data may be accessed without checking the tag and without activating the MMU 

data value is read or written every clock cycle

If the address is not in the cache or is an unbuffered write an external access is needed; this begins in the state. Reads from uncacheable memory and unbuffered writes are completed as single memory transactions. Cacheable reads perform a quad-word line fetch, after fetching the necessary translation information if this was not already in the MMU Cycles where the CPU does not use memory are executed in the state 13

ARM600 cache control state machine

14

Memory Management

    Today computer systems typically run multiple processes, each with its own address space It would be too expensive to dedicate a full- address-space worth of memory for each process (many use only a small part of their address spac.) If Principle of Locality allows caches to offer speed of cache memory with size of DRAM memory, then  Virtual Memory recursively DRAM can act as a “cache” for secondary storage (disk) Virtual memory – divides physical memory into blocks and allocate them to different processes 15

Virtual Memory Motivation

     Historically virtual memory was invented when programs became too large for physical memory Allows OS to share memory and protect programs from each other (main reason today) Provides illusion of very large memory   sum of the memory of many jobs greater than physical memory allows each job to exceed the size of physical mem.

Allows available physical memory to be very well utilized Exploits memory hierarchy to keep average access time low 16

Virtual Memory Terminology

    Virtual Address  address used by the programmer; CPU produces virtual addresses Virtual Address Space  collection of such addresses Memory (Physical or Real) Address  address of word in physical memory Memory mapping or address translation  process of virtual to physical address translation 17

Paging vs. Segmentation

   Fixed size blocks, called pages  (4KB – 64KB) Both logical and physical memory are divided into fixed-size components called pages (typically a few KBs)  Relationship between the logical and physical pages is stored in page tables (PTs) which are held in main memory Variable size blocks, called segments (1B – 64KB/4GB); each segment contains a particular sort of information  e.g., code segment, data segment, stack segment Paged segments – a segment is an integral number of pages Code Data Paging Segmentation 18

Segmented memory management

  Segmentation allows a program to have its own private view of memory Segments are of variable size => free memory becomes fragmented over time  it is possible that a new program is unable to start when the memory is fragmented in small pieces, none of which is big enough to hold a segment, even if there is enough free memory => OS is responsible to coalesce the free memory into one large piece segment selector logical address base limit segment descriptor table + physic al address >?

ac cess fault 19

Paging memory management

Virtual Address 31 ...

10 9 ...

Virtual Page No. Offset 0 translation 29 ...

10 9 ... 0 Physical Address Phys. Page No. Offset   Use table lookup (“ Page Table ”) for mappings: Virtual Page number is index Virtual Memory Mapping Function   Physical Offset = Virtual Offset Physical Page Number (P.P.N. or “Page frame”) = PageTable[Virtual Page Number] 20

Paging memory management (cont’d)

Virtual Address: virtual page no.

offset Valid Page Table Access Rights Physical Page Number Page Table Base Reg index into Page Table ...

physical page no.

offset Physical Address 21

Paging memory management (cont’d)

   Size of a PT?

 4KB pages, 32-bit VA => 2 20 x 20 (2.5MB) Use two or more levels of page table Example   10 MSBs are used to identify appropriate second level table page table in the first-level page table directory second ten bits of the address then identify the page table entry which contains the physical page number 31 22 21 12 11 logical address page directory page table 0 data page frame 22

Mapping Virtual to Physical Memory

  Program with 4 pages (A, B, C, D) Any chunk of Virtual Memory assigned to any chuck of Physical Memory (“page”) 0 4 KB Virtual Memory A B 8 KB C 12 KB D Disk D Physical Memory 0 4 KB B 8 KB 12 KB 16 KB A 20 KB 24 KB 28 KB C 23

Fast Address Translation

    PTs are stored in main memory  Every memory access logically takes at least twice as long, one access to obtain physical address and second access to get the data Observation: locality in pages of data, must be locality in virtual addresses of those pages  Remember the last translation(s) Address translations are kept in a special cache called Translation Look-Aside Buffer or TLB TLB must be on chip; its access time is comparable to cache 24

Typical TLB Format

Virtual Addr.

Physical Addr.

Dirty Ref Valid Access Rights       Tag: Portion of virtual address Data: Physical Page number Dirty: since use write back, need to know whether or not to write page to disk when replaced Ref: Used to help calculate LRU on replacement Valid: Entry is valid Access rights: R (read permission), W (write perm.) 25

Translation Look-Aside Buffers

  TLBs usually small, typically 128 - 256 entries Like any other cache, the TLB can be fully associative, set associative, or direct mapped Processor VA TLB Lookup hit PA Cache miss miss hit Data Translation Main Memory 26

TLB Translation Steps

     Assume 32 entries, fully-associative TLB (Alpha AXP 21064) 1: Processor sends the virtual address to all tags 2: If there is a hit (there is an entry in TLB with that Virtual Page number and valid bit is 1) and there is no access violation, then 3: Matching tag sends the corresponding Physical Page number 4: Combine Physical Page number and Page Offset to get full physical address 27

What if not in TLB?

  Option 1: Hardware checks page table and loads new Page Table Entry into TLB Option 2: Hardware traps to OS, up to OS to decide what to do     When in the operating system, we don't do translation (turn off virtual memory) The operating system knows which program caused the TLB fault, page fault, and knows what the virtual address desired was requested So it looks the data up in the page table If the data is in memory, simply add the entry to the TLB, evicting an old entry from the TLB 28

What if the data is on disk?

  We load the page off the disk into a free block of memory, using a DMA transfer  Meantime we switch to some other process waiting to be run When the DMA is complete, we get an interrupt and update the process's page table  So when we switch back to the task, the desired data will be in memory 29

What if we don't have enough memory?

  We chose some other page belonging to a program and transfer it onto the disk if it is dirty   If clean (other copy is up-to-date), just overwrite that data in memory We chose the page to evict based on replacement policy (e.g., LRU) And update that program's page table to reflect the fact that its memory moved somewhere else 30

Page Replacement Algorithms

  First-In/First Out     in response to page fault, replace the page that has been in memory for the longest period of time does not make use of the principle of locality: an old but frequently used page could be replaced easy to implement (OS maintains history thread through page table entries) usually exhibits the worst behavior Least Recently Used    selects the least recently used page for replacement requires knowledge of past references more difficult to implement, good performance 31

Page Replacement Algorithms

 Not Recently Used (an estimation of LRU)     A reference bit flag is associated to each page table entry such that Ref flag = 1 - if page has been referenced in recent past Ref flag = 0 - otherwise If replacement is necessary, choose any page frame such that its reference bit is 0 OS periodically clears the reference bits Reference bit is set whenever a page is accessed 32

Virtual and physical caches

  When system incorporates both MMU and a cache. the cache may operate either with virtual or physical address Virtual cache   + : cache access may start immediately; there is no need to activate the MMU if the data is found in the cache => save the power, eliminates address translation from a cache hit - : drawbacks o every time a process is switched VAs refer to different physical addresses => cache to flushed on each process switch • increase the width of the cache address tag with a PID o OS and user programs may use two different VAs for the same physical location (synonyms, aliases) => may result in two copies of the same data, • if we modify one, the other will have wrong value 33

Virtual and physical caches (cont’d)

 Paging MMU only affects the high-order address bits, while the cache is accessed by the low-order address bits    if these two sets do not overlap, the cache and MMU may proceed in parallel the physical address from MMU arrives at the right time to be compared with the physical address tags from the cache, hiding the address translation time behind the cache tag access Limits: if we have 4KB page =>max cache is 4KB direct-mapped, or 8KB 2-way set associative, or 16KN 4-way, etc ...

34

The ARM710T cache organization

31 4-way, 4KB, 16B blocks, random replacement policy, write-through, virtual cache, 11 10 4 3 2 1 0 virtual address tag RAM 128 entry =?

tag RAM 128 entry =?

tag RAM 128 entry =?

byte addresses tag RAM [10:0] [1:0] [8:2] 128 entry [10:9] =?

data RAM 2048 x 32-bit word hit encode data 35