Transcript PowerPoint 簡報 - Stony Brook University

Scalable Network-based Buffer Overflow
Attack Detection
Fu-Hau Hsu
Department of Computer
Science and Information
Engineering
National Central University
Taoyuan, Taiwan, R.O.C.
[email protected]
Fanglu Guo
Symantec Research
Laboratory
Cupertino, CA, U.S.A.
[email protected]
ANCS 2006
Tzi-cker Chiueh
Computer Science
Department
Stony Brook University
Stony Brook, NY, U.S.A.
[email protected]
Virulence of Buffer Overflow
Attacks
Buffer overflow attack is arguably the most
widely used and thus most dangerous attack
method used today.
Most Internet Worms use it to proliferate
themselves.
It accounts for more than 50% of all the
security vulnerabilities recorded by CERT.
ANCS 2006
Proposed Solutions
Compiler Transformation
Stack Guard, RAD, Address Obfuscation
Library Rewriting
OS
Non-executable Stack
Instruction Set
Hardware
AMD Athlon-64
ANCS 2006
Discrepancy between Theory and
Practice
In theory, these efforts have largely solved
the buffer overflow attack problem.
In practice, however, new buffer overflow
vulnerabilities are still discovered and
reported on a routine basis.
ANCS 2006
substantial modification
substantial resistance
Why?
ANCS 2006
A Solution to the above Dilemma -Nebula
Nebula
A network-based buffer overflow attack
detection mechanism
Observe the network traffic only to detect
BOAs
Currently version is developed for Linux
paltforms.
ANCS 2006
Existing Network-based Intrusion
Detection System (NIDS)
Misuse intrusion detection
Zero-day BOAs
Labor-Intensive
• Solution: automatically signature-generating
approaches
Anomaly intrusion detection
False Positive
ANCS 2006
Generalized Signature
ANCS 2006
Two Factors for a Successful
Buffer Overflow-style Attack
A successful buffer overflow-style attack
should be able to overflow the right place
(e.g. the place to hold a return address with
the correct value (e.g. the address of
injected code entry point)).
ANCS 2006
Non-predicable Offset and Entry
Point Address
return address
buffer where the
overflow start
injected code
address of injected code
entry point.
offset between the beginning of the
overflowed buffer and the overflow
target.
The offset and the entry point address are non-predicable. They can
not decided by just looking the source code or local binary code.
ANCS 2006
Non-predicable Offset
For performance concerns, most compilers don’t
allocate memory for local variables in the order
they appear in the source code, sometimes some
space may be inserted between them. (Source
Code doesn’t help)
Different compiler/OS uses different allocation
strategy. (Local binaries don’t help)
Address obfuscation insert random number of
space between local variables and return address.
(Super good luck may help)
ANCS 2006
Non-predicable Entry Point
Address
[fhsu@ecsl]#
0xbfffffff
webserver –a –b security
system data
environment variables
argument strings
env pointers
argv pointers
argc
Function main()’s
stack frame
ANCS 2006
command line arguments
and environment variables
Strategies Used by Attackers to
Increase Their Success Chance
Repeat address patterns.
Insert NOP (0x90) operations before the
entry point of injected code.
ANCS 2006
Indispensable Elements of BOstyle Attacks
‘ The Address ’
For buffer overflow attacks, it is the address
of the entry point of injected code.
ANCS 2006
Linux Process Memory Layout
0xffffffff
0xc0000000
kernel address space
user stack
8M
address space of
addresses of
%esp
injected code
and frame pointers
(Stack Address Zone)
for Shared libraries,
0x40000000 including libc functions
run-time heap
data and code
ANCS 2006
brk
Size of Stack Address Zone
The default maximum size of a process’s
user space stack is 8 Mbytes.
However, according to Ditzel et al., the
average function frame size is 28 bytes.
Therefore, the majority of program are not
supposed to use a 2Mbyte stack.
In our test, a 8k stack is enough to identify
all 10 remote exploit strings.
ANCS 2006
Repeating Times and Values of
Return Addresses
2k stack --- 0xbffffffff ~ 0xbfffe000
ANCS 2006
A Property of Stack Addresses
The leading byte of any words that contain
a stack address corresponds to a nonprintable ASCII character.
ANCS 2006
Generalized Signature
Signature of a stack smashing buffer
overflow attack :
If a sub-string of a traffic string could be
interpreted as a stack address that repeats 3
or more times, it is alarmed as a buffer
overflow attack string.
ANCS 2006
Contextual Analysis
ANCS 2006
Bypassing Detection
Patient attackers could bypass detection based on
repeating address signature by repeating addresses
no more than 2 times.
PS: All the 10 remote exploit code we tested repeat at
least 4 times.
Attackers repeat the addresses to increase their
chance to success. In other words, it is very likely
that without the repeat, attackers will fail many
times before getting a successful one.
ANCS 2006
Unsuccessful Attacks
Buffer overflow-style attacks will destroy targeted
process’s address space which in turn usually will
crash the attacked process.
In order to recycle valuable system resources, OS
will close the sockets opened by crash processes
automatically.
On both Linux and Windows, when a program is
crashed, the OS will terminate all the program’s
pending socket connections by sending out an
RST packet to the communicating hosts on its
behalf.
ANCS 2006
Server Termination Signature
After forwarding a sub-string which could
be interpreted as a single stack address,
Nebula detects that the server closes the
TCP connection without sending any data,
then the traffic string is deemed as a buffer
overflow attack string.
Future traffic coming from the same hosts
will be blocked or examined thoroughly.
ANCS 2006
Will Normal Traffic Behavior the
Same Way?
HTTP Protocol (RFC 2616) works in the
request-reply way. (After the request, there
will be a reply before the server close the
connection)
SMTP protocol (RFC 2821), for e-mail, and
FTP protocol(RFC 959) use QUIT
command to close a connection. (QUIT can
not be interpreted as a stack address.)
ANCS 2006
Payload Bypassing
ANCS 2006
Payload Bypassing
Payload bypassing tries to avoid packet analysis
for as much traffic as possible.
Because most buffer overflow attacks take place
during the exchange of control messages, it is safe
to ignore the bulk of data that is downloaded as
uninterpreted bytes.
For example, in an FTP session, data transferred over
the data connection can never be used to mount a buffer
overflow attack against the FTP program because the
FTP program does not interpret them.
ANCS 2006
Internet Traffic Statistic
From CacheLogic’s measurement on USA,
Europe, and Asia backbone in June 2004,
HTTP and P2P packets accounted for more
than 70% of the total traffic.
ANCS 2006
Percentage of Payload
Percentage of payload in the traffic when each of
the four protocols that Nebula can recognize is
used to transfer files of a total size of 1.22 Gbytes.
ANCS 2006
Number of False Positives without
Payload Bypassing
Number of false positives under the our sample as reported by Nebula.
The minimal number of times the attack pattern is repeated is assumed
to be 1, 2, 3 or 10, and the stack size tested is 2Mbytes, 16Kbytes, or
8Kbytes. In each entry the left is the number of false positives for RTL
attacks, whereas the right is the number of false positives for CI attacks.
The sample includes 134966 TCP connections and about 1.582 Gbytes
of data.
ANCS 2006
Number of False Positives with
Payload Bypassing
The number of false positives in the test traffic
associated with different protocols after applying
payload bypassing is negligible even when the
attack pattern repetition count is 1.
ANCS 2006
Throughput Comparison
The throughput of Nebula under a test HTTP connection when
different options are turned on. With payload bypassing, Nebula can
perform buffer overflow attack detection and still achieve a throughput
higher than a generic Linux router.
ANCS 2006