Transcript - TMA portal
Connect. Communicate. Collaborate
Experiences with tools for network anomaly detection in the GÉANT2 core
Maurizio Molina, DANTE COST TMA tech. Seminar Samos, 23
rd
Sep 2008
The GÉANT Network
Connect. Communicate. Collaborate • DANTE operates GÉANT2 • Backbone network for National Research and Education Networks in Europe • 30+ NRENs, 2 global connectivity providers (Telia and GCrossing), peerings with other research networks (Abilene, Canarie, Clara, TEIN2, SINET…)
The GÉANT Network (IP layer)
Pls see www.dante.net
Connect. Communicate. Collaborate • 20 Juniper routers • tenths of GBit/s of aggregated traffic • Main accesses and the backbone 10Gbit/s
The Services
Connect. Communicate. Collaborate • So…. Just a big pipe? No!
• Services – Dedicated L1-L2 circuits via multiple technologies – Performance Monitoring services (perfSONAR)
NEW!
– Support for federation of National AA Infrastructures (eduGAIN) and wireless roaming (eduROAM) – Security Service
Very NEW!
The vision: enhance NRENs security
Connect. Communicate. Collaborate • NRENs have their (+ - evolved…) CERTs to deal with security • and DANTE can filter traffic on GÉANT upon NRENs request….
! BUT !
• Can we be more
proactive
to NREN CERTs exploiting the
visibility
of the GN2 core?
The vision (cont.): enhance NRENs security
Connect. Communicate. Collaborate •
Approach: NetFlow (+ Routing data) & good processing tools
NetFlow v5 collector • Netflow collected on all peering interfaces • 1 / 1,000 Sampling • ~3k flows/s
Proof of concept: Can we identify anomalies in the core?
Connect. Communicate. Collaborate • Anomalies are often “hidden” Requirements:
NfSen
High detection rate Low false positives Anomaly classification Evidence collection
From “volume” to “IP feature entropies”
•“IP features entropies” •Simple linear filter
Drilling down on peaks
-Concentration of DST IPs and DST ports receiving flows -Dispersion of SRC IPs and SRC ports • IRC server in Slovenia, receiving a lot of 60 bytes syn pkts on port 6667, mainly from a /16 Subnetwork of an University in the Netherlands.
• Likely a “BotNet war”?
Drilling down on peaks (cont.)
- Concentration of SRC and DST IPs and SRC ports - Dispersion of DST ports • Portscan of host in CARNET, from 4 hosts, 29 bytes packets
Open source tools
Connect. Communicate. Collaborate • Results: – anomalies are observable in the GÉANT2 core – Novel methodologies (IP Features entropy) for their classifications are applicable • Limits: – NfSen does not fuse NetFlow and Routing data – Extensions would need to be run (and tuned) on all ingress/egress points – No support, no guaranteed development
Commercial tools
Connect. Communicate. Collaborate • Test started Jun 08 (3 tools) – Tool 1 • PCA, entropy – Tool 2 • Large scale DDoS and Worm spread – Tool 3 • Per host behaviour
Tool 1 (as a security tool…)
Connect. Communicate. Collaborate • Two main novel elements – Principal Component Analysis (
PCA
) – Both Volume and
IP features Entropy
anomaly detection • Address what makes anomaly detection a complex task –
PCA
: single parameter to control detection sensitivity, even if anomalies are
attributed
to specific OD pairs –
Entropy
: Detection of both low volume (scans) and high volume (DoS) anomalies
Demo….
• …. Or Screenshots….
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Connect. Communicate. Collaborate
Tool 2
Connect. Communicate. Collaborate • Well-established (and expensive!) solution for detecting “large” events • Originally based on large volume shifts only • Now enhanced to give alerts on “fingerprints” (e.g. communication with C&C servers) – Shared by (part) of the user community (50 out of 120) • No usage of routing data – though “zones” can be manually created via BGP prefixes lists • Traditional threshold based detection (although adaptive)
Tool 3
Connect. Communicate. Collaborate • Per host behavioural analysis • rather complex “scoring” system to distinguish normal from abnormal behaviour. Proprietary algorithms • Doesn’t use routing info – though “zones” can be manually created via BGP prefixes lists • Potentially attractive methodology • Concerns on scalability and accuracy with 1,000 sampling
lessons learnt and directions for research
Connect. Communicate. Collaborate • Manual validation is required to confirm/correct anomalies – More automatic intelligence to help this process – Fusion with other data sources (router logs? Honeynets?) • Detection space of 3 tools often disjoint – (Standard) anomaly injection • Operations need
supported tools
to
support services
• If choice is among published but “not a tool” or “secret but supported and (claiming to) work” => risk to stick to those!
– Fill the gap towards TOOLS!
Thank you!
[email protected]
Connect. Communicate. Collaborate