Transcript Intrusion Detection & Network Forensics Marcus J. Ranum [email protected] Chief Technology Officer Network Flight Recorder, Inc.

Intrusion Detection
&
Network Forensics
Marcus J. Ranum
[email protected]
Chief Technology Officer
Network Flight Recorder, Inc.
1
An ounce of
prevention is worth a
pound of detection
2
Why Talk about IDS?
• Emerging new technology
– Very interesting
...but...
– About to be over-hyped
• Being informed is the best weapon in
the security analyst’s arsenal
– It also helps keep vendors honest!
3
What is an Intrusion?!
• Difficult to define
– Not everyone agrees
– This is a big problem
• How about someone telnetting your system?
– And trying to log in as “root”?
• What about a ping sweep?
• What about them running an ISS scan?
• What about them trying phf on your webserver?
– What about succeeding with phf and logging in?
4
What is IDS?
• The ideal Intrusion Detection System
will notify the system/network manager
of a successful attack in progress:
– With 100% accuracy
– Promptly (in under a minute)
– With complete diagnosis of the attack
– With recommendations on how to block it
…Too bad it doesn’t exist!!
5
Objectives: 100% Accuracy
and 0% False Positives
• A False Positive is when a system
raises an incorrect alert
– “The boy who cried ‘wolf!’” syndrome
• 0% false positives is the goal
– It’s easy to achieve this: simply detect
nothing
• 0% false negatives is another goal:
don’t let an attack pass undetected
6
Objectives: Prompt
Notification
• To be maximally accurate the system
may need to “sit on” information for a
while until all the details come in
– e.g.: Slow-scan attacks may not be
detected for hours
– This has important implications for how
“real-time” IDS can be!
– IDS should notify user as to detection lag
7
Objectives: Prompt
Notification
(cont)
• Notification channel must be protected
– What if attacker is able to sever/block
notification mechanism?
– An IDS that uses E-mail to notify you is
going to have problems notifying you that
your E-mail server is under a denial of
service attack!
8
Objectives: Diagnosis
• Ideally, an IDS will categorize/identify
the attack
– Few network managers have the time to
know intimately how many network attacks
are performed
• This is a difficult thing to do
– Especially with things that “look weird” and
don’t match well-known attacks
9
Objectives: Recommendation
• The ultimate IDS would not only identify
an attack, it would:
– Assess the target’s vulnerability
– If the target is vulnerable it would notify the
administrator
– If the vulnerability has a known “fix” it
would include directions for applying the fix
• This requires huge, detailed knowledge
10
IDS: Pros
• A reasonably effective IDS can identify
– Internal hacking
– External hacking attempts
• Allows the system administrator to
quantify the level of attack the site is
under
• May act as a backstop if a firewall or
other security measures fail
11
IDS: Cons
• IDS’ don’t typically act to prevent or
block attacks
– They don’t replace firewalls, routers, etc.
• If the IDS detects trouble on your
interior network what are you going to
do?
– By definition it is already too late
12
Paradigms for Deploying IDS
• Attack Detection
• Intrusion Detection
13
Attack Detection
DMZ
Network
Desktop
WWW
Server
Internet
Router
w/some
screening
Internal
Network
Firewall
IDS detects (and counts) attacks against
the Web Server and firewall
IDS
14
Attack Detection
• Placing an IDS outside of the security
perimeter records attack level
– Presumably if the perimeter is well
designed the attacks should not affect it!
– Still useful information for management
(“we have been attacked 3,201 times this
month…)
– Prediction: AD Will generate a lot of noise
and be ignored quickly
15
Intrusion Detection
DMZ
Network
Desktop
WWW
Server
Internet
Router
w/some
screening
Internal
Network
Firewall
IDS detects hacking activity WITHIN
the protected network, incoming or outgoing
IDS
16
Intrusion Detection
• Placing an IDS within the perimeter will
detect instances of clearly improper
behavior
– Hacks via backdoors
– Hacks from staff against other sites
– Hacks that got through the firewall
• When the IDS alarm goes off, it’s a red
alert
17
Attack vs Intrusion Detection
• Ideally do both
• Realistically, do ID first then AD
– Or, deploy AD to justify security effort to
management, then deploy ID (more of a
political problem than a technical one)
• The real question here is one of staffing
costs to deal with alerts generated by
AD systems
18
IDS Data Source Paradigms
• Host Based
• Network Based
19
Host Based IDS
• Collect data usually from within the
operating system
– C2 audit logs
– System logs
– Application logs
• Data collected in very compact form
– But application / system specific
20
Host Based: Pro
• Quality of information is very high
– Software can “tune” what information it
needs (e.g.: C2 logs are configurable)
– Kernel logs “know” who user is
• Density of information is very high
– Often logs contain pre-processed
information (e.g.: “badsu” in syslog)
21
Host Based: Con
• Capture is often highly system specific
– Usually only 1, 2 or 3 platforms are
supported (“you can detect intrusions on
any platform you like as long as it’s Solaris
or NT!”)
• Performance is a wild-card
– To unload computation from host logs are
usually sent to an external processor
system
22
Host Based: Con
(cont)
• Hosts are often the target of attack
– If they are compromised their logs may be
subverted
– Data sent to the IDS may be corrupted
– If the IDS runs on the host itself it may be
subverted
23
Network Based IDS
• Collect data from the network or a hub /
switch
– Reassemble packets
– Look at headers
• Try to determine what is happening from
the contents of the network traffic
– User identities, etc inferred from actions
24
Network Based: Pro
•
•
•
•
•
No performance impact
More tamper resistant
No management impact on platforms
Works across O/S’
Can derive information that host based
logs might not provide (packet
fragmenting, port scanning, etc.)
25
Network Based: Con
• May lose packets on flooded networks
• May mis-reassemble packets
• May not understand O/S specific
application protocols (e.g.: SMB)
• May not understand obsolete network
protocols (e.g.: anything non-IP)
• Does not handle encrypted data
26
IDS Paradigms
•
•
•
•
•
Anomaly Detection - the AI approach
Misuse Detection - simple and easy
Burglar Alarms - policy based detection
Honey Pots - lure the hackers in
Hybrids - a bit of this and that
27
Anomaly Detection
• Goals:
– Analyse the network or system and infer
what is normal
– Apply statistical or heuristic measures to
subsequent events and determine if they
match the model/statistic of “normal”
– If events are outside of a probability
window of “normal” generate an alert
(tuneable control of false positives)
28
Anomaly Detection
(cont)
• Typical anomaly detection approaches:
– Neural networks - probability-based pattern
recognition
– Statistical analysis - modelling behavior of
users and looking for deviations from the
norm
– State change analysis - modelling system’s
state and looking for deviations from the
norm
29
Anomaly Detection: Pro
• If it works it could conceivably catch any
possible attack
• If it works it could conceivably catch
attacks that we haven’t seen before
– Or close variants to previously-known
attacks
• Best of all it won’t require constantly
keeping up on hacking technique
30
Anomaly Detection: Con
• Current implementations don’t work
very well
– Too many false positives/negatives
• Cannot categorize attacks very well
– “Something looks abnormal”
– Requires expertise to figure out what
triggered the alert
– Ex: Neural nets can’t say why they trigger
31
Anomaly Detection: Examples
• Most of the research is in anomaly
detection
– Because it’s a harder problem
– Because it’s a more interesting problem
• There are many examples, these are
just a few
– Most are at the proof of concept stage
32
Misuse Detection
• Goals:
– Know what constitutes an attack
– Detect it
33
Misuse Detection
(cont)
• Typical misuse detection approaches:
– “Network grep” - look for strings in network
connections which might indicate an attack
in progress
– Pattern matching - encode series of states
that are passed through during the course
of an attack
• e.g.: “change ownership of /etc/passwd” ->
“open /etc/passwd for write” -> alert
34
Misuse Detection: Pro
•
•
•
•
•
•
Easy to implement
Easy to deploy
Easy to update
Easy to understand
Low false positives
Fast
35
Misuse Detection: Con
• Cannot detect something previously
unknown
• Constantly needs to be updated with
new rules
• Easier to fool
36
Burglar Alarms
• A burglar alarm is a misuse detection
system that is carefully targeted
– You may not care about people portscanning your firewall from the outside
– You may care profoundly about people
port-scanning your mainframe from the
inside
– Set up a misuse detector to watch for
misuses violating site policy
37
Burglar Alarms
(cont)
• Goals:
– Based on site policy alert administrator to
policy violations
– Detect events that may not be “security”
events which may indicate a policy
violation
• New routers
• New subnets
• New web servers
38
Burglar Alarms
(cont)
• Trivial burglar alarms can be built with
tcpdump and perl
• Netlog and NFR are useful event
recorders which may be used to trigger
alarms
http://www.nswc.navy.mil/ISSEC/Docs/loggingproject.html
ftp://coast.cs.purdue.edu/pub/tools/unix/netlog/
http://www.nfr.net/download
39
Burglar Alarms
(cont)
• The ideal burglar alarm will be situated
so that it fires when an attacker
performs an action that they normally
would try once they have successfully
broken in
– Adding a userid
– Zapping a log file
– Making a program setuid root
40
Burglar Alarms
(cont)
• Burglar alarms are a big win for the
network manager:
– Leverage local knowledge of the local
network layout
– Leverage knowledge of commonly used
hacker tricks
41
Burglar Alarms: Pro
•
•
•
•
•
•
Reliable
Predictable
Easy to implement
Easy to understand
Generate next to no false positives
Can (sometimes) detect previously
unknown attacks
42
Burglar Alarms: Con
• Policy-directed
– Requires knowledge about your network
– Requires a certain amount of stability
within your network
• Requires care not to trigger them
yourself
43
Honey Pots
• A honey pot is a system that is
deliberately named and configured so
as to invite attack
– swift-terminal.bigbank.com
– www-transact.site.com
– source-r-us.company.com
– admincenter.noc.company.net
44
Honey Pots
(cont)
• Goals:
– Make it look inviting
– Make it look weak and easy to crack
– Instrument every piece of the system
– Monitor all traffic going in or out
– Alert administrator whenever someone
accesses the system
45
Honey Pots
(cont)
• Trivial honey pots can be built using
tools like:
– tcpwrapper
– Burglar alarm tools (see “burglar alarms”)
– restricted/logging shells (sudo, adminshell)
– C2 security features (ugh!)
• See Cheswick’s paper “An evening with
Berferd” for examples
46
Honey Pots: Pro
•
•
•
•
Easy to implement
Easy to understand
Reliable
No performance cost
47
Honey Pots: Con
• Assumes hackers are really stupid
– They aren’t
48
Hybrid IDS
• The current crop of commercial IDS are
mostly hybrids
– Misuse detection (signatures or simple
patterns)
– Expert logic (network-based inference of
common attacks)
– Statistical anomaly detection (values that
are out of bounds)
49
Hybrid IDS
(cont)
• At present, the hybrids’ main strength
appears to be the misuse detection
capability
– Statistical anomaly detection is useful more
as backfill information in the case of
something going wrong
– Too many false positives - many sites turn
anomaly detection off
50
Hybrid IDS
(cont)
• The ultimate hybrid IDS would
incorporate logic from vulnerability
scanners*
– Build maps of existing vulnerabilities into
its logic of where to watch for attacks
• Backfeed statistical information into
misuse detection via a user interface
* Presumably, a clueful network
51
admin would just fix the vulnerabilty
Books
• Internet Security and Firewalls:
Repelling the Wily Hacker, by Bill
Cheswick and Steve Bellovin, from
Addison Wesley
• Internet Firewalls, by Brent Chapman
and Elizabeth Zwicky
52
URLs
• Spaf’s Security Page
– http://www.cs.purdue.edu/people/spaf
• Mjr’s home page
– http://www.clark.net/pub/mjr
• Hacker sites: the fringe
– http://www.lopht.com
– http://www.digicrime.com
53
Addresses
• CERT
– [email protected]
• Firewalls mailing list
– [email protected]: subscribe
firewalls
• Web security mailing list
– [email protected]: subscribe
www-security
54
Addresses
• Firewalls Wizards mailing list
– [email protected]: subscribe firewallwizards
• http://www.nfr.net/forum/firewall-wizards.html
– Searchable online archive on
• http://www.nfr.net/firewall-wizards/
55