Transcript VMs: Supporting Changing Technology and New Applications

Virtual Machines:
Supporting Changing Technology
and New Applications
Georgia Tech
November 2006
J. E. Smith
Introduction
Why virtual machines?
They allow transcending of standardized interfaces
(which sometimes are an obstacle to innovation)
They enable innovation in flexible, adaptive software &
hardware, security, network computing (and others)
They involve computer architecture in a pure sense
Virtualization technologies will be a key part of most future
computer systems
VMs (c) 2006, J. E. Smith
2
Outline




Virtualization
Virtual Machine Architecture
Virtual Machine Implementation
Computer Architecture Applications
•
•
Co-Designed VMs
Private Virtual Machines
VMs (c) 2006, J. E. Smith
3
Abstraction
Software



Computer systems are built
on levels of abstraction
Higher level of abstraction
hide details at lower levels
Example: files are an
abstraction of a disk
Application
Programs
fileLibraries
file
Operating System
abstraction
Memory
Manager
Drivers
Scheduler
Execution Hardware
System Interconnect
(bus)
Memory
Translation
Controllers
Controllers
I/O devices
and
Networking
Main
Memory
Hardware
VMs (c) 2006, J. E. Smith
4
Virtualization

Similar to abstraction
Except
• Same level of detail

Construct Virtual Disks
•
•
•


virtualization
file
file
As files on a larger disk
Map state
Map operations
VMs: do the same thing with the whole
“machine”
Key concepts: Map state; Map Operations
VMs (c) 2006, J. E. Smith
5
The Family of Virtual Machines
There are things
Including
lots of not
“virtual
called
machines”
“virtual machines”
IBM VM/370
IA-32
EL
Java
HP
Dynamo
VMware products
Transmeta
Crusoe
“The subjects of virtual machines and
emulators have been treated as entirely
separate. … they have much in common. Not
only do the usual implementations have many
shared characteristics, but this commonality
extends to the theoretical concepts on which
they are based”
-- Efrem G. Wallach, 1973
VMs (c) 2006, J. E. Smith
6
“Machines”




Different perspectives on
what the Machine is:
OS developer
Compiler developer
Application programmer
Application
Programs
Libraries
Operating System
Execution Hardware
Instruction Set
Architecture
Application
Binary
Program
Interface
Interface
••
•
•
ISA
API
ABI
Major
division
User
User ISA
ISA
++ library
OSbetween
calls
calls hardware
and software
System Interconnect
(bus)
I/O devices
and
Networking
VMs (c) 2006, J. E. Smith
Memory
Translation
Main
Memory
7
System Virtual Machines





ISA level
Provide a system
environment
VMM manages
guest OS + apps
Persistent
Examples: IBM
VM/360, VMware,
Transmeta Crusoe
guest
guest
process process
guest
process
guest
guest
guest
process process process
Guest OS
Guest OS2
VMM
VMM
HOST PLATFORM
virtual
network communication
VMs (c) 2006, J. E. Smith
8
Process Virtual Machines







ABI level
Runtime manages guest
process
Guest processes may
intermingle with host
processes
Not persistent
Guest and host OSes are
often the same
Dynamic optimizers are
a special case
Examples: IA-32 EL,
FX!32, Dynamo
guest
process
host
process
runtime
guest
process
guest
process
runtime
host
process
runtime
create
HOST OS
file sharing
Disk
network communication
VMs (c) 2006, J. E. Smith
9
High Level Language Virtual Machines


Constructed at API level
• User higher level virtual ISA
• OS abstracted as standard libraries
A form of process VM
HLL Program
HLL Program
Compiler front-end
Intermediate Code
Compiler
Portable Code
( Virtual ISA )
Compiler back-end
VM loader
Object Code
(ISA)
Virt. Mem. Image
VM Interpreter/Translator
Loader
Memory Image
Host Instructions
Traditional
HLL VM
VMs (c) 2006, J. E. Smith
10
Virtual Machine Architectures
Process VMs
same ISA
Multi
programmed
Systems
Dynamic
Binary
Optimizers
System VMs
different
ISA
same ISA
Dynamic
Translators
Classic
OS VMs
Whole
System VMs
HLL VMs
Hosted
VMs
Co-Designed
VMs
VMs (c) 2006, J. E. Smith
different
ISA
11
VM Technology – State Mapping

VM SW re-maps virtual
state to real state
•
•
•
•
•
Recall virtual disk
Registers to registers
Registers to memory
Memory to memory
Memory to disk
Host Registers
Host
Register
Space
Guest Registers
Runtime
Data
Runtime
Code
Guest Data
Host ABI
Address
Space
Guest Code
VMs (c) 2006, J. E. Smith
12
VM Technology – Operation Mapping


VM SW re-maps operations on state
Instruction-level state changes
•

Protected state changes
•
•

Emulation
OS operations
Done under VMM control
Key concepts: Emulation and Control
VMs (c) 2006, J. E. Smith
13
VM Technology – Emulation

Interpretation
•
Software loop
decodes and
dispatches each
instruction
interpreter
routines
source code
"data"
accesses
dispatch
loop
VMs (c) 2006, J. E. Smith
14
VM Technology – Emulation

Binary translation and
code caching
•
•
•
binary translated
target code
source code
Translate blocks of
instructions at a time
Hold translated blocks in
code cache
This was a key enabler
for VMware success
VMs (c) 2006, J. E. Smith
binary
translator
15
VM Technology – Emulation

Staged Emulation
•
•
Emulation techniques invoked in staged manner
Based on performance tradeoffs
Interpreter
Binary Memory
Image
Emulation
manager
Code Cache
Profile Data
Translator/
Optimizer
VMs (c) 2006, J. E. Smith
16
Code Caches

Contain
•
•
•

Basic blocks
Superblocks (one entrance, multiple exits)
Optimized Superblocks
Used in many VMs
•
•
•
•
•
•
Dynamic binary translators: Intel IA-32 EL, Compaq FX!32
Dynamic binary optimizers: Dynamo family
Co-designed virtual machines: Transmeta, IBM DAISY
High performance Java virtual machines
System VMs with “inefficiently virtualizable” ISAs
“Sandboxing” secure VMs (x86 DynamoRIO)
VMs (c) 2006, J. E. Smith
17
Code Caching with Chaining

Chaining of blocks in code cache minimizes VM
overhead
Code Cache
Super
block
Dispatch
table
lookup
code
Super
block
Super
block
Super
block
VMs (c) 2006, J. E. Smith
18
VM Technology – Control

Interpretation
•
•

Binary translation and code caching
•
•
•

Fine grain control
Every dynamic instruction “inspected” before execution
Coarser grain control
Every static instruction inspected before execution
Jumps to VM SW can be inserted anywhere
Protection levels
•
•
•
Very coarse grain control
Every resource-related instruction trapped by protection system
Otherwise, use interpretation/translation techniques
Used in system VMs
VMs (c) 2006, J. E. Smith
19
Resource Control in System VMs
Application

Traps and interrupts (& sys calls)
•
•
•

Guest OS “return” to user app.
•
•

Transfer to VMM
VMM determines appropriate Guest OS
VMM transfers to Guest OS
Transfer to VMM
VMM bounces return back to Guest app.
system call/trap
Guest OS
privileged operation
next instruction
virtual vector location:
system return
Resource sensitive instructions
•
•
•
•
Trap to VMM
VMM checks correctness
VMM reads/modifies guest resource
Returns to Guest
VMM
check privileges
perform operation
return
vector location:
VMs (c) 2006, J. E. Smith
20
VMs and Computer Architecture

Use virtualization to give computer architects a layer
of software
•
•

Performance optimizations via Co-designed VMs
•

Beneath all conventional software
Maintains vision of hardware as seen by conventional software
VM SW can alter/enhance architecture via emulation
Resource management – Private Virtual Machines
•
VM SW can manage microarchitecture resources
VMs (c) 2006, J. E. Smith
21
Co-Designed Virtual Machines


Separate the hardware/software interface from the
ISA level of abstraction
Restore the ISA to its “natural” place
 as an Implementation ISA that reflects actual hardware

Support existing ISAs
 as a Virtual ISA


Let processor designers use both
hardware and software
A form of system VM
User Applications
Applications
User
libs.
libs.
OS
V-ISA
ISA
Software
I-ISA
Hardware
Hardware
VMs (c) 2006, J. E. Smith
22
VM Technology -- Concealed Memory


VM software resides in memory concealed from
all conventional software
This software is available to hardware designer
concealed memory
Code
Cache
VM Code
ICache
Hierarchy
Processor
Core
VM Data
Source ISA Code
conventional
memory
Source ISA Data
DCache
Hierarchy
VMs (c) 2006, J. E. Smith
23
Co-Designed VMs

Of interest to both architects and microarchitects
•
•
•
Offers opportunities for performance, power saving,
fault tolerance and other implementation-dependent
features
Allows transcending conventional ISAs
Don’t confuse them with VLIW!
Early examples: IBM Daisy and Transmeta Crusoe
“pioneers are the ones with arrows in their backs”
VMs (c) 2006, J. E. Smith
24
Another Way of Doing Things
conventional
Main Memory
Translation
Unit
(form uops)
Cache
Hierarchy
Processor
Pipeline
Func.
Unit
Func.
Unit
. ..
Func.
Unit
Main Memory
Software
dynamic translation
Func.
Unit
Translator
Code Cache
Cache
Hierarchy
Processor
Pipeline
. ..
Translation
Unit
(form uops)
Func.
Unit
VMs (c) 2006, J. E. Smith
25
Fused Microarchitecture

Fuse dependent pairs of micro-ops to macro-ops
•



Current Intel approach
Use co-designed SW to achieve wider-scale fusing
Process & execute fused macro-ops as single
Instructions throughout the entire pipeline
Allows pipelined wake-up/select issue logic
3-1 ALUs
cache
ports
Fuse
bit
Fetch
Align
Fuse
Decode
Rename
Dispatch
WakeSelect
up
Payload
RAM
RF
VMs (c) 2006, J. E. Smith
EXE
MEM
WB
Retire
26
Fusible Instruction Set

Core 32 - bit instruction formats
F
10 b opcode
21 bit Immediate / Displacement
F
10 b opcode
16 bit immediate / Displacement 5b Rds
F
10 b opcode
11b Immediate/ Disp 5 b Rsrc 5b Rds
F
RISC-ops with unique
features:
16 bit opcode
5b Rsrc 5b Rsrc 5b Rds
Add-on16-bit instruction formats for code density
F
5 b op
10b Immd/Disp
F
5 b op
5b Rsrc 5 b Rds
F
5b op
5b Rsrc 5 b Rds

•
Fuse bit per instruction fuses
two dependent instructions
•
Dense instruction encoding,
16/32-bit ISA design
Special Features to Support
the x86 ISA
•
Condition codes
•
Addressing modes
•
Aware of long immediate &
displacement values
Fusible ISA Instruction Formats
VMs (c) 2006, J. E. Smith
27
Fusing Profile

About 50% of operations are fused
Only 5-10% of non-fused are single-cycle ALU ops
100%
90%
Percentage of Dynamic Instructions

80%
70%
60%
50%
ALU
40%
FP or NOPs
30%
BR
20%
ST
LD
10%
Fused
0%
VMs (c) 2006, J. E. Smith
28
Performance
Nomarlized IPC speedup (%)
Base + Code Cache
+ fusing
+ shorter pipe
+ 3-1 ALU
70
60
50
40
30
20
10
0
-10
VMs (c) 2006, J. E. Smith
29
Virtual Private Machines

Multi-core systems will have many hardwarelevel shared resources
•
•
•
•

Multi-threaded processors
Multi-level shared caches
Shared memory ports
Spares for fault tolerance
And a number of important implementation
dependences
•
•
•
Non-uniform memory delays
Power optimization features
Fault tolerance features
VMs (c) 2006, J. E. Smith
30
Virtual Private Machines

Co-design a MicroVisor to provide software with
Virtual Private Machines
•

Insulates conventional software from complicated
implementation-dependent features
Provides performance virtualization
•
•
•
Unlike classic VMs
Quality of Service (QoS)
Performance Isolation Functional
Virtualization
System
VM
System
VM
Conventional
OS
Hypervisor (VMM)
VPM
Performance
Virtualization
VPM
MicroVisor
Multi-Core Hardware
VMs (c) 2006, J. E. Smith
31
MicroVisor


Virtualizes performance, not functionality
Co-Designed software to support hardware resource
management
•
•
•
Concealed/isolated from all conventional software
Much larger than microcode, does not consume processor chip
real estate
Uses conventional instructions (extended) so there is more
likelihood of some cross-system portability
concealed memory
μV Code
μV Data
ICache
Hierarchy
Processor
Core
Code
conventional
memory
Data
DCache
Hierarchy
VMs (c) 2006, J. E. Smith
32
“Real-izing” Processors/Memory

Separate Real Processors
from Physical Processors
•
•

OS assigns processes to
Real Processors
MicroVisor maps real
processors to physical
processors
MicroVisor also maps
real memory to physical
memory
•
Extend to cache memories
VMs (c) 2006, J. E. Smith
Processes
OS Maps
Real
Processors
MicroVisor Maps
Physical
Processors
33
Virtual Private Machines


MicroVisor maps high level requirements to hardware
configuration
Requires mechanisms to provide microarchitecture level QoS
Main Memory
Main Memory
Memory Controller
L2 Cache
Mem
Controller
Mem
Controller
Mem
Controller
Mem
Controller
L2 Cache
L2 Cache
L2 Cache
L2 Cache
L1 Cache
L1 Cache
L1 Cache
L1 Cache
Proc. 0
Thread 0
Proc. 0
Thread 0
Proc. 0
Thread 0
Proc. 0
Thread 0
VPM 0
VPM 1
VPM 2
VPM 3
Interconnection Net
L1 Cache
L1 Cache
Proc. 0
Threads
0&1
Proc. 1
Threads
2&3
VMs (c) 2006, J. E. Smith
34
Applications

Performance optimization
•
•

Power management
•
•

Deal with NUMA
Provide QoS and performance isolation in multi-threaded systems
Adjust resources to match power constraints
Requires inferring demand for resources
in contrast to conventional OS
Fault tolerance
•
•
Detected fault triggers MicroVisor
Diagnose, reconfigure, re-map memory/processors
VMs (c) 2006, J. E. Smith
35
Summary

Many types of VMs
•

An important system component
•

But common virtualization technologies
Should be studied/taught as a discipline on its own
Alongside OS, Application SW, HW
Many avenues for computer architecture
research
•
•
•
•
•
Co-designed VMs
Virtual Private Machines
Adaptive microarchitecture
Fault-tolerant implementations
Primitives for supporting efficient VMs
VMs (c) 2006, J. E. Smith
36