Transcript Event Correlation: Security’s Holy Grail? Matthew Caldwell, CSO, GuardedNet BlackHat Briefings Las Vegas, NV August 1st 2002 ©2002 GuardedNet.

Event Correlation:
Security’s Holy Grail?
Matthew Caldwell, CSO, GuardedNet
BlackHat Briefings
Las Vegas, NV
August 1st 2002
©2002 GuardedNet. All rights reserved.
Confidential – Do Not Copy or Distribute
Why turn to Correlation?
>
Organizations have multi-vendor security infrastructures and can not integrate
each vendor’s log data for complete threat assessment.
>
No complete view of the enterprise’s security environment.
>
Log data overload allows for intruders to bypass the watchman.
>
High rate of false positives with present intrusion detection systems, caused by
their orientation toward attack signatures or attack anomaly detection and not
qualified threats
>
Vendors that only manage their own products also fail to scale to even medium
and large infrastructures.
>
Limited security budgets.
Correlation and Security
>
Correlation is particularly valuable to security teams, because
information security is a many to one process, rather than a one
to one or exception-based process.
>
Network Management is a good example of an exception-based
process. Correlation is much less valuable in that environment.
View of correlation in relation to other products
Heterogeneous
collection & correlation
threat analysis
Security
Event
Management
(SEM)
Configuration &
Control Product
Vendor-specific
configuration & control
of point solutions
i.e. Provider-1, Site Protector
Routers
Applications
Security Hardware
Devices
Servers
Antivirus
Firewall
Network IDS
Host IDS
Vendor-specific
point solutions
Correlation Pre-requisites
1. Data Transport – Moving Log Data from point A to point B
•
Security of the data is important at this level, by
reducing risk exposure by encrypting and authenticating
the data, data transport can be accomplished with high
certainty.
• 2. Data Normalization – Taking the data from disparate log
formats and creating a flexible single log format.
• 3. Aggregation – Taking Data from multiple B level
devices and consolidating the data into one system.
Correlation Pre-requisites
• 4. Data Reduction – Reducing the data through deletion
of duplicates, combining similar events into a single
filtering, or using summary information. Compressing the
data also is helpful to reduce bandwidth usage.
•
• 5. Correlation Engine – The process of correlating field
variables into a unique consolidated event.
What is Correlation?
• Cor·re·la·tion n.
1. A causal, complementary, parallel, or reciprocal relationship,
especially a structural, functional, or qualitative correspondence
between two comparable entities: a correlation between drug abuse
and crime.
2. Statistics. The simultaneous change in value of two numerically
valued random variables: the positive correlation between cigarette
smoking and the incidence of lung cancer; the negative correlation
between age and normal vision.
Types of Correlation
Micro Level Correlation – Comparing fields within one data set. Most of
the vendors today are performing this type of correlation. Micro
correlation is typically done with just the fields in the normalized data.
Otherwise known as atomic correlations.
Macro Level Correlation - Comparing multiple data sets. A few vendors
are performing Macro correlation. Example: Correlating particular
events from countries or comparing event names to vulnerability
names.
Micro Correlation
Field Correlation - The ability to correlate specific events to a single field
or multiple fields in the normalized data. Can be as simple as a search
for all port 80 attacks.
Rule/Pattern Correlation – The ability via specific rules to label a certain
set of events as a correlated event. Often used to correlate events over
a long period of time.
Macro Correlation
Vulnerability Correlation – The process of mapping event IDS event
names to vulnerability names this allows for reduction in false
positives. This is particular useful with a good tuned vulnerability
scanner and IDS.
Profile (Finger Print) Correlation - Forensic network data such as remote
port scans, remote OS finger prints, finger information, and banner
snatching provides a series of data sets that can be compared to help
correlate attacks to attacker profiles.
Watch List Correlation – Using a set of learned inputs the watch list can
serve as a reminder of previous offenders in real-time. Correlating
previous attackers with current attacks.
Who Needs Correlation?
Anyone Managing and Monitoring over 2 Firewalls or Intrusion Detection
Systems:
•
•
•
•
•
Managed Security Providers
Corporate Enterprises
Federal/State/Local Governments
Critical Infrastructure
ISACS
So how does correlation reduce false positives?
Step through correlation – Packets are seen across a router ACL,
attackers packets pass through firewall, miss the IDS, get picked up by
the anomaly detection and the IIS-UNICODE exploit stops at our
application security product or the correlations system compares the
event type from the application security product with a vulnerability
database. Well, no vulnerabilities existed, and the Server was Apache
not IIS.
All of the data is seen and understood. The analyst and correlation
system can make judgments based on knowing all the facts of the
attempted intrusion. Will disregard, the attack as NIMDA.
Understanding Market Powwow
According to some vendors correlation = aggregation
Others would like you to believe that correlation is some fancy algorithm
that puts everything together for you. Usually aimed at impressing the
Boss with big words.
Some of them would like you to believe that correlation is a verb not a
noun.
Beware of the Snake Oil’s (Really just a glorified Syslog viewer with color)
and the Candle’s (Blow out Easily)
The Capital Players – Current Products Claiming
Correlation

CyberWolf

eSecurity

GuardedNet

Intellitactics

ISS SiteProtector

NetForensics

OPEN

OpenSystems
What to do with Correlated Data?
Examples of what to do with Correlated Data
Threat Analysis – This is a process of taking the correlated data and
determining what it means and then prioritizing the data. Includes the
reduction of False Positives.
Forensic Analysis – The ability to analyze correlated events historically
for trending and for prosecution.
Policy Analysis – Correlation provides a holistic view, providing a
complete picture of your security posture.
Threat Analysis Process
Threat Analysis and Trending
Forensic Analysis
Holistic View: The High Ground
Benefits to the Security Organization
>
Real time view of event data from all systems
>
Centralized repository for event data from all systems
>
Provides threat rankings based on severity, allowing the analyst to
focus on true threat vs. false alarms
>
Enhances ability to proactively respond to threat in the fastest time
possible, with most complete information
>
Increases efficiency of security operation, reducing costs
>
Scales to monitor global networks
Why is Correlation and Threat Analysis Important?
>
Reduce operating costs through efficiency and better resource
allocation:
•
•
•
>
Avoid the costs associated with a breach
•
>
Monitor existing environment and deploy additional security measures without increasing resources
Improve decision-making processes
Dramatically reduce response times
Downtime, theft, or damage to reputation
Maximize utilization of existing security infrastructure
•
Allows customers to manage their “Best of Breed” products and to truly get the full use those
resources
>
Increase security awareness at all organizational levels
>
Ensure better overall enterprise protection