Connect. Communicate. Collaborate

Experiences with tools for network anomaly detection in the GÉANT2 core

Maurizio Molina, DANTE COST TMA tech. Seminar Samos, 23

rd

Sep 2008

The GÉANT Network

Connect. Communicate. Collaborate • DANTE operates GÉANT2 • Backbone network for National Research and Education Networks in Europe • 30+ NRENs, 2 global connectivity providers (Telia and GCrossing), peerings with other research networks (Abilene, Canarie, Clara, TEIN2, SINET…)

The GÉANT Network (IP layer)

Pls see www.dante.net

Connect. Communicate. Collaborate • 20 Juniper routers • tenths of GBit/s of aggregated traffic • Main accesses and the backbone 10Gbit/s

The Services

Connect. Communicate. Collaborate • So…. Just a big pipe? No!

• Services – Dedicated L1-L2 circuits via multiple technologies – Performance Monitoring services (perfSONAR)

NEW!

– Support for federation of National AA Infrastructures (eduGAIN) and wireless roaming (eduROAM) – Security Service

Very NEW!

The vision: enhance NRENs security

Connect. Communicate. Collaborate • NRENs have their (+ - evolved…) CERTs to deal with security • and DANTE can filter traffic on GÉANT upon NRENs request….

! BUT !

• Can we be more

proactive

to NREN CERTs exploiting the

visibility

of the GN2 core?

The vision (cont.): enhance NRENs security

Connect. Communicate. Collaborate •

Approach: NetFlow (+ Routing data) & good processing tools

NetFlow v5 collector • Netflow collected on all peering interfaces • 1 / 1,000 Sampling • ~3k flows/s

Proof of concept: Can we identify anomalies in the core?

Connect. Communicate. Collaborate • Anomalies are often “hidden” Requirements:

NfSen

 High detection rate    Low false positives Anomaly classification Evidence collection

From “volume” to “IP feature entropies”

•“IP features entropies” •Simple linear filter

Drilling down on peaks

-Concentration of DST IPs and DST ports receiving flows -Dispersion of SRC IPs and SRC ports • IRC server in Slovenia, receiving a lot of 60 bytes syn pkts on port 6667, mainly from a /16 Subnetwork of an University in the Netherlands.

• Likely a “BotNet war”?

Drilling down on peaks (cont.)

- Concentration of SRC and DST IPs and SRC ports - Dispersion of DST ports • Portscan of host in CARNET, from 4 hosts, 29 bytes packets

Open source tools

Connect. Communicate. Collaborate • Results: – anomalies are observable in the GÉANT2 core – Novel methodologies (IP Features entropy) for their classifications are applicable • Limits: – NfSen does not fuse NetFlow and Routing data – Extensions would need to be run (and tuned) on all ingress/egress points – No support, no guaranteed development

Commercial tools

Connect. Communicate. Collaborate • Test started Jun 08 (3 tools) – Tool 1 • PCA, entropy – Tool 2 • Large scale DDoS and Worm spread – Tool 3 • Per host behaviour

Tool 1 (as a security tool…)

Connect. Communicate. Collaborate • Two main novel elements – Principal Component Analysis (

PCA

) – Both Volume and

IP features Entropy

anomaly detection • Address what makes anomaly detection a complex task –

PCA

: single parameter to control detection sensitivity, even if anomalies are

attributed

to specific OD pairs –

Entropy

: Detection of both low volume (scans) and high volume (DoS) anomalies

Demo….

• …. Or Screenshots….

Connect. Communicate. Collaborate

Connect. Communicate. Collaborate

Connect. Communicate. Collaborate

Connect. Communicate. Collaborate

Connect. Communicate. Collaborate

Connect. Communicate. Collaborate

Connect. Communicate. Collaborate

Connect. Communicate. Collaborate

Connect. Communicate. Collaborate

Tool 2

Connect. Communicate. Collaborate • Well-established (and expensive!) solution for detecting “large” events • Originally based on large volume shifts only • Now enhanced to give alerts on “fingerprints” (e.g. communication with C&C servers) – Shared by (part) of the user community (50 out of 120) • No usage of routing data – though “zones” can be manually created via BGP prefixes lists • Traditional threshold based detection (although adaptive)

Tool 3

Connect. Communicate. Collaborate • Per host behavioural analysis • rather complex “scoring” system to distinguish normal from abnormal behaviour. Proprietary algorithms • Doesn’t use routing info – though “zones” can be manually created via BGP prefixes lists • Potentially attractive methodology • Concerns on scalability and accuracy with 1,000 sampling

lessons learnt and directions for research

Connect. Communicate. Collaborate • Manual validation is required to confirm/correct anomalies – More automatic intelligence to help this process – Fusion with other data sources (router logs? Honeynets?) • Detection space of 3 tools often disjoint – (Standard) anomaly injection • Operations need

supported tools

to

support services

• If choice is among published but “not a tool” or “secret but supported and (claiming to) work” => risk to stick to those!

– Fill the gap towards TOOLS!

Thank you!

[email protected]

Connect. Communicate. Collaborate