Transcript Parallelizing Security Checks on Commodity Hardware Ed Nightingale Microsoft Research Dan Peek, Peter Chen Jason Flinn University of Michigan.

Parallelizing Security Checks
on Commodity Hardware
Ed Nightingale
Microsoft Research
Dan Peek, Peter Chen
Jason Flinn
University of Michigan
Run-time security checks

Provide run-time intrusion detection/prevention


Many powerful checks have been proposed
Examples: Taint-analysis, on-access virus scanning,
system call graph modeling

Often impose high performance penalties

What makes run-time checks slow?
ASPLOS 2008
2
Problem
Application
& security check
Application
A
B
C
D
PAUSE
RESUME
PAUSE
RESUME
PAUSE
RESUME
PAUSE

A
B
1)
2)
3)
4)
Begin application
Pause execution
Run security check
Resume application
C
D
Checks may slow performance by 10x
ASPLOS 2008
3
Solution
Speck (Speculative parallel check)




Parallelize checks across many cores
Provide security equivalent to sequential check
Use three techniques



Transparent kernel replay system
Operating system support for speculative execution
Buffer dependent output until checks complete
ASPLOS 2008
4
Accelerating checks with Speck
cpu0
Speculate!


cpu1
cpu2
cpu3
cpu4
A
B
C
D
1) Run process speculatively
2) Execute check in parallel
Speed of parallelization without sacrificing safety!
 3) Start later checks in parallel to earlier ones

ASPLOS 2008
5
Outline

Introduction

Implementing Speck

Parallelizing security checks
ASPLOS 2008
6
Example: Executing sequentially
PAUSE

A
All state ‘frozen’ and available to security check

Examples – heap, address space, file name,
system call, value of a single address in memory
ASPLOS 2008
7
Example: Executing in parallel
cpu0
A
cpu1
fork()

How does security check get app state?

Speck uses fork() to copy state to other core
ASPLOS 2008
8
Example:
Executing
in
parallel
cpu0
cpu1
cpu2
epoch 0
epoch 1
A
B
C
D
E
F
fork()
fork()
B
C
E
F


Amortize fork by grouping checks into epochs
What happens when app interacts with OS?
ASPLOS 2008
9
Safely executing system calls

Some system calls affect state of calling process


Other system calls may affect other processes


Introduces non-determinism in execution
Other processes may be compromised as well
Some system calls generate output

Output to screen or network cannot be undone
ASPLOS 2008
10
Replaying non-determinism
Uninst Process
data
Inst Clone
Execute read()
Execute read()
OS

Speck replays system calls that affect process state



Some system calls (mmap) re-executed
Signal delivery limited to exit from system call
Prevents instrumented clone from diverging
ASPLOS 2008
11
Tracking causal dependencies

Speculator [sosp05] tracks causal dependencies




Supports FIFOs, pipes, UNIX sockets etc.
Undo log associated with each object
Do not handle multi-threading (MP replay hard)
All dependent objects rolled back on failure

Uninstrumented process terminated on failure

Uninstrumented process can safely run ahead
ASPLOS 2008
12
Handling output commits

Problem: Some system calls create output

Output to network or screen buffered

When all checks within epoch complete
dependent output released
ASPLOS 2008
13
Equivalence/Safety

Assume attacker cannot compromise



Replay system ensures code equivalence


Speck in-kernel replay system
Speculator in-kernel causal dependency tracking
Instrumented clone does not diverge
Speculator prevents permanent damage

Speculator rolls back all dependent state on failure
ASPLOS 2008
14
Outline

Introduction

Implementing Speck

Parallelizing security checks
ASPLOS 2008
15
Choosing security checks

All checks depend upon uninstrumented process


Amount of state determines Speck strategy
Some checks require little state or run infrequently


No need for fork and replay…just pause app and ship state
Some checks depend upon result of earlier check



Independent checks easy to parallelize
Many dependencies make parallelization harder
Example: taint analysis
ASPLOS 2008
16
Using Speck to parallelize checks

Process memory analysis


System call analysis


Sensitive data leaks
On-access virus scanner
Data flow analysis

Taint analysis
ASPLOS 2008
17
Process Memory Analysis

Checking for transient leaks of sensitive data

Our check examines every memory store




Looks for signature of sensitive data
Examines all 16 byte windows around address
Use Pin dynamic binary rewriting tool
Later checks do not depend on earlier checks
ASPLOS 2008
18
MPlayer video decoding
Speck
Sequential
Frames per second
40
35
30
25
20
15
10
5
0
1
2
3
4
5
6
7
8
Number of cores

Speck 7.5x faster with 8 cores

Video plays in real time
ASPLOS 2008
19
On-access virus scanner

Scan files on-access by a process


Our check scans each file on-close



Many different policies -- on-read on-write etc.
Emulates news/email server
Implemented using ClamAV libraries (180K sigs)
Little state is required and checks independent

Speck does not use fork and replay for this check
ASPLOS 2008
20
PostMark benchmark
Transactions per second
Speck
Sequential
7000
6000
5000
4000
3000
2000
1000
0
1
2
3
4
5
6
7
8
Number of cores

2.8x more TPS
ASPLOS 2008
21
Taint analysis

Trace flow of data from untrusted sources




Instrument application at instruction granularity
Update map of tainted addresses at run-time
Ensure certain addresses are not tainted
Later checks depend on result of earlier checks

Running in parallel cannot tell whether address was
tainted during prior check
ASPLOS 2008
22
Parallelizing taint analysis
cpu0
A
B
C
D
cpu1
fork()
…cpu n
cpu2
A
fork()
B
C
D
log
log

New algorithm minimizes sequential processing
ASPLOS 2008
23
Dynamic taint compression

Dependency logs were very large


Time to process longer than to run sequential check
Only care about dependencies before check



Addresses and registers often overwritten
Eliminate unimportant dependencies
Reduce log size 6x
ASPLOS 2008
24
Taint analysis: MPlayer
Frames per second
Speck
Sequential
Speck no opt
10
9
8
7
6
5
4
3
2
1
0
1
2
3
4
5
6
7
8
Number of cores

2x speedup on 8-cores
ASPLOS 2008
25
Choosing a strategy
Check
State required/
When required
Checks
independent?
Fork
& replay?
Work to
make ||?
Memory
analysis
Memory
On store
Yes
Yes
None
Virus
check
File name
On file close
Yes
No
A little
Taint
analysis
Rep. of addr space
On data flow inst.
No
Yes
A lot


Fork & replay – need to attach check at fork
No fork & replay – need to accept concurrent
requests
ASPLOS 2008
26
Conclusion

Run-time security checks can be slow

Speck accelerates run-time checks


Parallelizing across many cores
Provides safety of executing checks sequentially
ASPLOS 2008
27