Transcript No Slide Title

DERBI: Diagnosis, Explanation and
Recovery from Break-Ins
Mabry Tyson
Pauline Berry
Nate Williams
Doug Moran
David Blei
Artificial Intelligence Center
SRI International
333 Ravenswood Avenue
Menlo Park CA 94025
http://www.ai.sri.com/~derbi
[email protected]
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
1
DERBI Objective
• Assist SysAdmin after an attack
– No special security expertise required
– Detailed system analysis as though by a OS/security expert
– For sites that didn’t think they needed a real-time ID system
• Require nothing beyond off-the-shelf OS
– No special logging or monitoring
• Provide guidance on what happened and how to
recover
• How much info can be detected after-the-fact?
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
2
System Description
• Rules specify bits of evidence and associated
exploit
• Rule Graph embodies relationships of evidence
and attack goals
– Beliefs of evidence combined to generate overall belief of attack
• Anthropomorphic characterization of system
– Head - High level control
– Body - Passes messages between Head and Feet
– Feet - Runs around and does the work
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
3
Head
• Uses PRS (Procedural Reasoning System)
• Operates on rule graph
– Goal is to determine whether attack happened
– Goal is achieved by acquiring evidence
• Handles user interaction
–
–
–
–
User can add evidence
Rules can query user
Results presented to user
User can drill down
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
4
Body
• Allows Head to deal with abstract queries
• Allows Feet to deal with O/S specific queries
• Deals with multiple hosts
– Network communications
– Time differences
– File system differences
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
5
Feet
• O/S specific
– Knows how to traverse file system
• Careful to collect file info before altering it
– Understands special file locations
– Parses log files
• ID Evaluation primarily exercises the Feet
• Solaris & Linux
– Only Solaris used in ID Evaluation
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
6
Example Evidence Rule:
EJECT buffer overflow
EVIDENCE-TYPE
(exploit (setuid root) buffer-overflow)
UNIQUE-NAME
eject-1
EVALUATION-NAME
eject
PATHS
(follow-links '("/usr/bin/eject"))
EVIDENCE
( ((not (and (command-version-vulnerable-p DIR FILE)
;; not vulnerable command or
(window-of-opportunity (TimeAccessed PATH)))) ;; not used in interval of interest
0 0)
;;; assign 0% probability to command being used and 0% believe that it was
((greater-than (TimeAccessed PATH)
;;; use is later than
(max (TimeModified "/cdrom") (TimeModified "/floppy"))) ;;; expected effects
40 100)) ;;; 40% probability of exploit, no change in believe about whether it was exploited
POSIT
((posit ((TIME (TimeAccessed PATH))) (compromised-shell "root" TIME *unknown-time*)))
EXPLANATION (next slide)
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
8
Evidence Rule:
EJECT buffer overflow (cont)
UNIQUE-NAME
eject-1
PATHS
(follow-links '("/usr/bin/eject"))
EXPLANATION
(explain-evidence
(
PATH
;;; variable declarations
(TIME (print-unix-time (TimeAccessed PATH)))
(TIME2 (print-unix-time (TimeModified "/cdrom")))
(TIME3 (print-unix-time (TimeModified "/floppy"))) )
(TimeAccessed PATH)
;;; “as-of” time
"The command ~S is version vulnerable to a buffer overflow attack
and appears to have been used at time ~A
which is more recent than two associated files:
/cdrom (~A) and /floppy (~A)."
PATH TIME TIME2 TIME3)
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
9
Example Output for an Attack
Time: 08-Apr-1999 13:11:57 EDT
Exploit: Suspicious-login (Suspicious-login)
Login was found for user "doireano"
from host 194.27.251.21. This user not seen
before.
-----------------------------------------------------------+00:12:05 later
Time: 08-Apr-1999 13:24:02 EDT
Exploit: FORMAT (FORMAT-1)
The command "/usr/bin/fdformat" is a version
vulnerable to a buffer overflow attack
and appears to have been used at time 08-Apr1999 13:24:02 EDT
which is more recent than the associated device:
"/devices/sbus@1f,0/SUNW,fdtwo@f,1400000:c,r
aw" (04-Mar-1999 11:52:23 EST).
DERBI
13 December 1999
+00:02:17 later
Time: 08-Apr-1999 13:26:19 EDT
Exploit: Unauthorized/nonstandard file activity
(FILEACT)
1 files were created with no obvious legitimate
user having access.
Root users currently are *None*.
Normal users are (erink doireano ulandusm
grzegors).
Groups with a member logged in are *None*.
Ignored logins are *None*.
Groups with an ignored login are *None*.
Files' owner: root Files's group: staff Protection:
-rw------/.sh_history
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
10
Checking a Suspect System
Ultra
Ultra
DERBI
Ultra
DERBI
DERBI
Ultra
Ultra
Ultra
DERBI
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
11
Data Sources for ID Evaluation
• File system is only source of information
– System files
– Log files
– File system
• DERBI has capability to query operator
– For example, compare file to backup version
– Allow operator to indicate remote login normal or
suspicious
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
12
Target System Configuration Files
• Passwd
– Notes crackable passwords
• Hosts.equiv, .rhosts
– Notes capability for passwordless logins
– Notes world-writable system directories
• Crontab files
– Notes programs run from crontab
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
13
Log Files
• utmpx, wtmpx, utmp, wtmp, lastlog
–
–
–
–
–
–
All compared for inconsistencies
Note logins without logouts
Note inconsistencies in tty usage
Note currently unknown users
Note remote logins from a new host for that user
Note failed logins
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
14
Log File Information Relationships
• Partial redundancy of info
utmp
wtmp
syslog
utmpx
wtmpx
messages
lastlog
authlog
• Redundancy a common result
of the evolution & growth of
systems
•Use to check for tampering
• Also exposes changes to system
clock
sulog
cronlog
crontabs
Shell Init Files
File system
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
15
Log Files (2)
• Syslog, messages, authlog
–
–
–
–
–
sendmail messages (mailbomb, locally sent mail)
su times
sshd messages (failures, successful logins/logouts)
ntp anomalies
Verify time of log messages monotonic
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
16
File System Info
• Executables
– Access times usually means execution
– Comparison of suid execute-time vs data file access time
– Checksums checked for vulnerable or replaced versions
• Normal files
– File access/creation, owner and protection recorded for every file
– Files that indicate login/logout are specially noted (dot files, pty
and window system files)
• Special files
– Known cracker file names (included deleted files)
– Rarely used files that crackers may use
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
17
Evidence Correlated by Time
• File access/creation and log information sorted by time
• Unauthorized access detected when no authorized user
known to be logged in at time files accessed or created
– Complications:
• Background processes, servers and scheduled jobs
• Suid executables
• Attacks usually evident by clustering of evidence
– Often see evidence of an exploit
– Followed by evidence of unauthorized access to files
– However, attack can be inferred from a single anomaly
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
18
Detection of New Attacks
• “New attack” means new exploit
• DERBI spots the intentional and secondary effects of the
cracker on the system, after the (new) exploit
• Crackers often leave a large trail of evidence
–
–
–
–
Exploit files touched
Camouflage attempts often leave footprints
Data collectors & back doors often detectable
However, ID Evaluation attacks often are hit-and-run
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
19
Detectable Attacks
• Detects R2L, U2R, Data attacks on Solaris (and Linux)
• Can detect some DoS attacks when logged (mailbomb, ssh,
or telnet attempts)
• Generally can only detect latest use of executables (i.e.,
only the last eject attack could be detected)
• Cracker or normal activity can destroy evidence of attack
• Can’t detect network traffic but not blinded by encryption
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
20
ID Evaluation Results
• Test procedure artifacts complicated evaluation
– Evaluation team affected file system (apparently including running
attacks) outside of simulation runs but with clock set to times
within simulation periods
• Dot files accessed and files written in a user’s directory but simulation
contained no login
• Executables such as eject accessed without device accessed as though
an attack was done, but no attack at that time during simulation
– Also overwrote access times of all files on some days
• Simulated “attacks” were often just exercise exploit and
leave
– DERBI picks up evidence of usage of privileges
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
21
ID Evaluation Results
• 25 attacks in detectable classes
• 17 attacks detected
– score of 16.98 (68%)
• 47 false alarms
– score of 25
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
22
ID Evaluation Results - Misses
• 8 misses
– 1 attack missed due to test procedure overwriting
access times
• ffbconfig
– 5 attacks left no evidence
• guessftp, xsnoop, xlock, httptunnel usage (x2)
– 2 attacks indistinguishable from normal activity
• httptunnel setup - no recognizable suspicious indications
• ps - telnet from a new host, but otherwise nothing suspicious
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
23
ID Evaluation Results - False Alarms
• 47 total false alarms (total score of 25)
• 29 probably due to test procedure (total score 15.2)
– 18 definite test procedure artifacts (score 4.55)
– 11 probable test procedure artifacts (score 10.65)
• 18 other false alarms (total score 9.8)
–
–
–
–
–
7 pseudo-tty errors (looked like log file truncation) (score 5.1)
5 login/logout record problems (score 3.6)
3 dot files accessed when user not logged in (score 0.03)
2 root accessed secret files in a sweep of file system (score 1)
1 secret access while logged in locally and remotely (score 0.05)
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
24
ROC - Overall
100
90
80
70
P Detect
without art if acts
including art if acts
60
50
Total Attacks: 25
30 Hits: 17 (16.98)
Total FAs: 47 (25)
20
Hits: 18 (17.98)
10
Total FAs: 18 (9.8)
40
0
0.1
DERBI
13 December 1999
1
10
Fal se Al ar ms per Day
100
1000
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
25
ROC - Old vs Overall
100
90
80
70
60
P Detect
Ov erall w/ o artif act s
Ov erall including artif act s
Old w/o artif act s
Old including artif acts
50
40
30
20
Total Attacks: 23
Hits: 15 (15)
Total FAs: 47 (25)
Hits: 16 (16)
Total FAs: 18 (9.8)
10
0.1
1
10
100
0
1000
Fal se Al ar ms per Day
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
26
ROC - R2L
100
90
80
P Detect
70
60
50
40
30
20
without art if ac ts
inc luding art if ac ts
Total Attacks: 12
Hits: 6 (6)
Total FAs: 2 (1.7)
Hits: 6 (6)
Total FAs: 1 (0.7)
10
0.1
1
10
100
0
1000
Fal se Al ar ms per Day
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
27
ROC - U2R
100
90
80
P Detect
70
60
50
40
30
20
without art if ac ts
inc luding art if ac ts
Total Attacks: 11
Hits: 9 (9)
Total FAs: 21(18.45)
Hits: 10 (10)
Total FAs: 10 (7.5)
10
0.1
1
10
100
0
1000
Fal se Al ar ms per Day
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
28
ROC - Data
100
90
80
P Detect
70
60
50
40
30
20
without art if ac ts
inc luding art if ac ts
Total Attacks: 3
Hits: 3 (2.98)
Total FAs: 26 (6.53)
Hits: 3 (2.98)
Total FAs: 8 (2.28)
10
0.1
1
10
100
0
1000
Fal se Al ar ms per Day
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
29
DERBI Project Ends
• DERBI has come to its end -- for now
• Experience at analyzing intrusions as a sysadmin
led to the idea a system could be built to do this
and to make it easier for less experienced
sysadmins
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
30
DERBI is a Success
• Successful at detecting intrusions on a stock
system
– Original idea of a post-mortem analysis has been
proven
– Designed for real intrusions, it performs better the more
the cracker does
– Difficult to imagine how to further improve detection
without modifying O/S
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
31
DERBI is Different
• The DERBI concept is orthogonal to most other
ID systems
– This diversity could be useful as the systems have
different strengths and weaknesses
– Didn’t fit too well with the design of the ID evaluation
• Not a substitute for intrusion monitoring systems,
but can aid those sites that don’t want the
overhead of such systems
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
32
Parting Thoughts
• The problem of intrusions has a variety of
responses for a variety of consumers
–
–
–
–
Read-only systems or network computers
Brick-up-the-door approach
“We can’t let it happen” approach (most IDS)
“It happens” approach (DERBI)
• ID shouldn’t be an after-market add-on to an OS
– Watch for incoming and outgoing attacks
DERBI
13 December 1999
1999 Intrusion Detection Evaluation
Joint DARPA ID/SIA PI Meeting
33