Transcript Network Forensics Tracking Hackers Through Cyberspace.
STATISTICAL FLOW ANALYSIS
Section 4.1
Network Forensics TRACKING HACKERS THROUGH CYBERSPACE
• • •
PURPOSE
Identify compromised hosts • Send out more traffic • • Use usual ports Communicate with known malicious systems Confirm / Disprove data leakage • Volume of exported data Individual profiling • Reveal • Normal working hours • • • Periods of inactivity Sources of entertainment Correlate activity exchanges
PROCESS OVERVIEW
• Defined • “ Flow record—A subset of information about a flow. Typically, a flow record includes the source and destination IP address, source and destination port (where applicable), protocol, date, time, and the amount of data transmitted in each flow.” (Davidoff & Ham, 2012)
FLOW RECORD PROCESSING SYSTEM
• Flow record processing systems include the following components: • • Sensor—The device that is used to monitor the flows of traffic on any given segment and extract important bits of information to a flow record.
Collector—A server (or multiple servers) configured to listen on the network for flow record data and store it to a hard drive.
• • Aggregator—When multiple collectors are used, the data is typically aggregated on a central server for analysis.
Analysis—Once the flow record data has been exported and stored, it can be analyzed using a wide variety of commercial, open-source, and homegrown tools.
1 1. PG 161
SENSORS
• Sensor types • Network Equipment • • • Many switches support flow record creation and export • Cisco - NetFlow format • • Sonicwall – IPFIX and NetFlow Be cautious of “sampling” which is not comprehensive data Standalone appliances • Used if existing network software does not support flow data Software • • Argus – Audit Record Generation and Utilization System Softflowd • Yaf – Yet Another Flowmeter
• • •
SENSOR SOFTWARE
• • • • Yaf Argus • Two packages • • Argus Server • Argus Client Libpcap- based • • Supports BPF filtering Documentation specifically mentions forensic investigation • Argus’ compressed format over UDP Softflowd Passively monitor traffic Exports record data in NetFlow format Linux and OpenBSD Libpcap- based • • • Libpcap and live packet transfer IPFIX format over SCTP, TCP or UDP Supports BPF filters
SENSOR PLACEMENT
• • • Investigators often do not have much control over placement Infrastructures should be set up with flow monitoring in mind but usually are not Factors to consider • • Duplication is inefficient and must be minimized Time synchronization is crucial • • • Most flow records are collected on external devices such as firewalls but this ignores internal network traffic which can be valuable Resources are important when planning, prioritize Do not over load your network capacity
MODIFYING THE ENVIRONMENT
• • • Leverage existing equipment • Switches, routers, firewalls, NIDS / NIPS Upgrade network equipment • If existing equipment will not work deploy replacements Deploy additional sensors • • Use port mirroring to send packets to standalone sensor Network tap another option
FLOW RECORD EXPORT PROTOCOLS
• • • Proprietary – Cisco’s NetFlow Open source – IPFIX Relatively new and not yet matured – better tools on the horizon
NETFLOW
• • • • Maintains a cache that tracks the state of all active flows observed Completed flows marked as “expired” and exported as a “NetFlow Export” packet to a collector Newer versions (NetFlow v9) are transport-layer independent: UDP, TCP and SCTP Older versions only support UDP and IPv4
IPFIX
• • Extends NetFlow v9 • Handles bidirectional flow reporting • Reduces redundancy • Better interoperability Extensible flow record data using data templates • • Template defines data to be exported Sensor uses template to construct flow data export packets
SFLOW
• • • • • Supported by many devices – not Cisco Conduct statistical packet sampling Does not support recording and processing every packet Scales very well Generally not very good for forensic analysis
COLLECTION AND AGGREGATION
• Placement factors to consider • Congestion • Flow records generate network traffic and can intensify congestion • Choose location where this will cause low network impact • Security • Export flow records on separate VLAN if possible • Isolate physical cables • • • • Encrypt using IPSec or TLS Reliability • Consider using TCP or SCTP over UDP Capacity • One sensor or many?
Analysis strategy • Can affect all of the above, plan accordingly
COLLECTION SYSTEMS
• Commercial options • Cisco NetFlow Collector • • Manage Engine’s NetFlow Analyzer WatchPoint NetFlow Collector
COLLECTION SYSTEMS CONTINUED
• Open source options • SiLK – System for Internet Level Knowledge • • • • Command-line Most powerful – biggest learning curve • Collector specific tools – flowcap and rwflowpack Flow-tools Modular and easily extensible • • • Only accepts UDP input Nfdump / NfSen • Collector daemon – nfcapd • UDP network socket or pcap files Argus • Supports Argus format and NetFlow v 1-8 • NetFlow v9 and IPFIX not yet supported
ANALYSIS
• • Defined • “ Statistics—“The science which has to do with the collection, classification, and analysis of facts of a numerical nature regarding any topic.” (The Collaborative International Dictionary of English v.0.48).” (Davidoff & Ham, 2012) Purpose • Store a summary of information about the traffic flowing across the network • • Forensic data carving does not apply Still very useful
FLOW RECORD TECHNIQUES
• • Goals and resources • This should shape your analysis • Access available time, staff, equipment and tools Starting indicators – triggering event • Example evidence: • IP address of compromised or malicious system • • • Time frame of suspect activity Known ports of suspect activity Specific flows which indicate abnormal or unexplained activity
FLOW RECORD TECHNIQUES CONTINUED
• Analysis techniques • Filtering • • • Baselining “Dirty Values” Activity pattern matching
FILTERING
• • • Important to narrow down a large pool of evidence Remove extraneous data • • Start by isolating activity relating to specific IP address/es Filter for known patterns of behavior Use small percentages of data for detailed analysis
BASELINING
• • • • Advantage of flow record data vs full traffic capture • Dramatically smaller allowing for longer retention Build a profile of “normal” network activity Network baseline • General trends over a period of time Host baseline • • Historical baseline can identify anomalous behavior Most flow patterns will change dramatically if host is compromised or under attack
• • • • Suspicious keywords IP addresses Ports Protocols
“DIRTY VALUES”
•
ACTIVITY PATTERN MATCHING
Elements • IP address • Internal network or Internet-exposed network • • • • • • Country of origin Who are they registered too? Ports • Assigned / well-known ports link to specific applications • Is system scanning or being scanned?
Protocols and Flags • Layer 3 and 4 are often tracked in flow record data • Connection attempts Directionality • Data coming in (something downloaded) or going out (something uploaded) Volume of data transferred • Lots of small packets can indicate port scanning • • • Successful port scans Data transfers Large amounts of data usually cause for concern
• • • •
SIMPLE PATTERNS
Many-to-one IP addresses • DOS attack • • Email server (at destination) One-to-many IP addresses • Web server • • Syslog server “Drop box” data repository on destination IP Email server (at source) • SPAM bot • Warez server • Network port scanning Many-to-many IP addresses • Peer-to-peer file sharing • Widespread port scanning One-to-one IP addresses • Targeted attack • Routine Server communication
COMPLEX PATTERNS
• Fingerprinting • Matching complex flow record patterns to specific activities • Example: • TCP SYN port scan • One source IP address • • • • • One or more destination IP addresses Destination port numbers increase incrementally Volume of packets surpass a specified value within a given period of time TCP protocol Outbound protocol flags set to “SYN”
FLOW RECORD ANALYSIS TOOLS
• • • • • flowtools SiLK Argus FlowTraq Nfdump / NfSen
• • • • •
SiLK
Rwfilter • • Extracts flows of interest Filters by time and category • Partitions them by protocol attributes • Generally as functional as BPF Rwstats, rwcounts, rwcut, rwuniq • Basic manipulation utilities Rwidsquery • Can be fed a Snort rule or alert file and it will figure out which flow matches it and writes an rwfilter to match it Rwpmatch • Libpcap-based program that reads in SiLK-format flow metadata and an input source and save only the packets that match the metadata Advanced SiLK • Includes a Python interpreter “PySiLK”
•
FLOW-TOOLS
Variety • Flow export data collection • • Storage Processing • • • Sending tools • “flow-report” • ASCII text report based on stored flow data “flow-nfilter” • Filter based on primitives specific to flow-tools “flow-dscan” • Identifies suspicious traffic based on flow export data
• • • • •
ARGUS CLIENT TOOLS
Ra • • • Reads Filters Prints • Supports BPF filtering Racluster • Exports based on user-specified criteria Rasort • Sorts based on user-specified criteria Ragrep • Regular expression and pattern matching Rahisto • Generated frequency distribution table for user-selected metrics: flow duration, src and dst port numbers, byte transfer, packet counts, average duration, IP address, ports, etc
• • • • Commercial tool by ProQueSys Supports many formats and sniffs traffic directly Users can • • Filter Search • • Sort Produce reports Designed for forensics and incident response
FLOW TRAQ
• • •
NFDUMP
Part of the nfdump suite Includes • Aggregate flow record fields by specific fields • • Limit by time range Generate statistics • • • IP addresses Interfaces Ports • • • Anonymize IP addresses Customize output format BPF-style filters
NFSEN
Graphical, web-based interface for nfdump
ETHERAPE
• • • Libpcap-based graphical tool Visually displays activity in real time • Colors designate traffic protocol • • HTTP SMB • ICMP • IMAPS Does not take flow records as input
Works Cited