Transcript Document

Network Monitoring &
Troubleshooting plus Log
Analysis
Faculty:
Scott Greene
of
Evidence Solutions, Inc.
[email protected]
www.EvidenceSolutions.com
10 Signs
you aren’t cut out for IT
► 10
signs that you aren't cut out for IT
 1: You lack patience
U of Nebraska Incident
► An
Undergrad suspected in Univ. of
Nebraska breach where more than 650K
personal records were compromised in
attack.
► The intrusion was into a university database
containing personal information on more
than 650,000 students, parents and
employees.
U of Nebraska Incident
► The
intrusion, which was described by
university officials as a "skilled attack,"
exposed the Social Security Numbers
(SSNs), names, addresses, course grades
financial aid and other information on
students who attended the university since
1985.
U of Nebraska Incident
► The
breach also exposed personal data and
financial information for parents of students
who applied for financial aid at UNL,
according to the university.
► A staff member in UNL's Computing
Services Network discovered the breach in
the Nebraska Student Information System
(NeSIS) on May 23.
U of Nebraska Incident
► An
Undergrad suspected in Univ. of
Nebraska breach where more than 650K
personal records were compromised in
attack.
► The intrusion was into a university database
containing personal information on more
than 650,000 students, parents and
employees.
U of Nebraska Incident
► The
system manages student admissions,
campus housing and course registration.
► It was built over a three-year period at a
cost of $29.9 million, has been operational
for the past two years and is based on
Oracle's PeopleSoft Enterprise Campus
Solution platform.
U of Nebraska Incident
► An
FAQ on the incident posted by the
university makes it clear that personal data
in the breached server was not encrypted.
"However, we are confident that the type of
attack we experienced would have bypassed
any encryption that was in place," the
university said.
U of Nebraska Incident
► The
vulnerability that enabled the intrusion
has since been closed and the university is
currently working with a third-party firm to
review and address remaining
vulnerabilities.
20 Critical Security Controls
► 1)
Inventory of Authorized & Unauthorized
Devices
► 2) Inventory of Authorized & Unauthorized
Software
► 3) Secure Configurations for Hardware &
Software on Laptops, Workstations, &
Servers
20 Critical Security Controls
► 4)
Continuous Vulnerability Assessment &
Remediation
► 5) Malware Defenses
20 Critical Security Controls
► 6)
Application Software Security
 Code Reviews
 Proper Logging
 Abnormal operation reporting
20 Critical Security Controls
► 7)
Wireless Device Control
► 8) Data Recovery Capability
► 9) Security Skills Assessment and
Appropriate Training to Fill Gaps
► 10) Secure Configurations for Network
Devices such as Firewalls, Routers, and
Switches
20 Critical Security Controls
► 11)
Limitation and Control of Network Ports,
Protocols, and Services
 Including custom applications
 Development departments need to
communicate with network departments
20 Critical Security Controls
► 12)
Controlled Use of Administrative
Privileges
► 13) Boundary Defense





See 10
Include penetration testing
Include review of firewall rules
Remote Users
Mobile Devices
20 Critical Security Controls
► 14)
Maintenance, Monitoring, & Analysis of
Security Audit Logs
► 15) Controlled Access Based on the Need to
Know
20 Critical Security Controls
► 16)
Account Monitoring and Control
 What do users have rights to
►Why?
 What do processes have rights to
►Why?
20 Critical Security Controls
► 17)
Data Loss Prevention
► 18) Incident Response Capability
 Who responds
 Test those responses
 Who gets notified
►Hr
►Legal
20 Critical Security Controls
► 19)
Secure Network Engineering aka
“Develop a Secure Infrastructure”
► 20) Penetration testing
#14
► Validate
audit log settings for each
hardware device and the software installed
on it, ensuring that logs include:





Date
Timestamp
source addresses
destination addresses
Any other useful information
#14
► Normalize
Logs
 Syslog
 Common Event Expression initiative
 Use normalization tools to convert logs
#14
► Reports
 Security personnel and/or system administrators
should run weekly reports that identify
anomalies in logs.
 They should then actively review the anomalies,
documenting their findings.
►A
log for the log events
#14
► Time
Synch
 Use at least two synchronized time sources
 All servers and network equipment should be in
synch.
►Test
►Validate
their synchness
Federal Security Standards
► NIST
Special Publication (SP) 800-37
 Categorize
►The
information system and the information
processed, stored, and transmitted by that system
based on an impact analysis
Federal Security Standards
► NIST
Special Publication (SP) 800-37
 Baseline
►Select
an initial set of baseline security controls for
the information system based on the security
categorization; tailoring and supplementing the
security control baseline as needed based on an
organizational assessment of risk and local
conditions.
Federal Security Standards
► NIST
Special Publication (SP) 800-37
 Implement
►The
security controls and describe how the controls
are employed within the information system and its
environment of operation.
Federal Security Standards
► NIST
Special Publication (SP) 800-37
 Assess
►Assess
the security controls using appropriate
assessment procedures to determine the extent to
which the controls are implemented correctly,
operating as intended, and producing the desired
outcome with respect to meeting the security
requirements for the system.
Federal Security Standards
► NIST
Special Publication (SP) 800-37
 Authorize
►Information
system operation based on a
determination of the risk to organizational operations
and assets, individuals, other organizations, and the
Nation resulting from the operation of the
information system and the decision that this risk is
acceptable.
Federal Security Standards
► NIST
Special Publication (SP) 800-37
 Monitor
►The
security controls in the information system on an
ongoing basis including assessing control
effectiveness, documenting changes to the system or
its environment of operation, conducting security
impact analyses of the associated changes, and
reporting the security state of the system to
designated organizational officials.
Federal Security Standards
► NIST
Special Publication (SP) 800-37
►The
final step in the cycle, Monitor, is of particular
importance because it evaluates the effectiveness of
a security control. But what if you only performed
this evaluation periodically—for example, to satisfy a
quarterly or annual audit for a regulation or other
compliance related demand? Unfortunately, it could
be months or even a year before you’d realize that
the security control was not functioning as intended.
Federal Security Standards
► NIST
Special Publication (SP) 800-37
 Assess
►Assess
the security controls using appropriate
assessment procedures to determine the extent to
which the controls are implemented correctly,
operating as intended, and producing the desired
outcome with respect to meeting the security
requirements for the system.
Some Things to Monitor
► Patch
management
► Network management tools
► Security tools such as:




Change management
Configuration management
Log monitoring
Vulnerability scanning solutions
10 Signs
you aren’t cut out for IT
► 10
signs that you aren't cut out for IT
 1: You lack patience
 2: You have no desire to continue your
education
Logs?
► Syslog
is the predominant standard for
computer system logging
► Microsoft, in its infinite wisdom chose their
own called “Windows Event Log”.
 There are several converters to convert the
Windows Event Log to the Syslog standard.
Log Log Log
► Many
incidents can be readily revealed with
a bit of logging and analysis those logs.
Logs
► Solutions
 Almost everything that has a log should have
the log turned on.
 Logs should include:
►Date/time
►Source
IP
►Destination IP
►Port
►Etc
Logs
► Solutions
 Use standard SYSLOG entries or use software
that converts logs to a common log format.
 Store logs for a while – space & DVDs are
cheap
 Create systems & procedures for analyzing logs.
►These
systems should have ‘normal’ items and
‘abnormal’ items
Logs
► Solutions
 All remote access logging:
►should
be in detail
►Should be rigorously analyzed.
 All security alerts should be logged.
►Workstation
►Servers
►Devices
Logs
► Solutions
 Use unified time
►This
allows logs to be matched up across many
devices and / or networks.
 Border devices
►Should
log verbosely
►Should log all traffic
 Blocked
 Allowed
Logs
► Solutions
 Logs should be secured
 Logs should be exported & saved on Write Once
devices.
or
 Logs should be written to dedicated logging
servers.
 The dedicated logging servers with separate
security credentials
Logs
► Solutions
 Test the logs and review after:
►Normal
/ acceptable traffic
►Push the system
►Attempt to penetrate the network.
 Inside
 Outside
►Compare
validity.
and correlate the data on all of the logs for
Logs
► Solutions
 Review
►Logs
everyday
►Use automated tools to analyze large amounts of
data.
 Test
►Attack
a system
►Test the response time.
 Discovery
 Action taken to attack
Log Review Tools
► Windows
-> Syslog conversion
 Snare agent
(intersectalliance.com/projects/index.html) and
ProjectLasso remote collector
(sourceforge.net/projects/lassolog) are used to
convert Windows Event Logs into syslog, a key
component of any log management
infrastructure today (at least until Visa/W7 log
aggregation tools become mainstream
10 Signs
you aren’t cut out for IT
► 10
signs that you aren't cut out for IT
 1: You lack patience
 2: You have no desire to continue your
education
 3: You refuse to work outside 9-to-5
Database Activity Management
► Database
Activity Monitoring
 (DAM) is a database security technology for
monitoring and analyzing database activity that
operates independently of the database
management system (DBMS)
 It does not rely on any form of native (DBMSresident) auditing or native logs such as trace
or transaction logs.
 DAM is typically performed continuously and in
real-time.
Database Activity Management
► Add
prevention and you get (DAMP)
 This extension to DAM goes beyond monitoring
and alerting to also block unauthorized
activities.
 DAM helps organizations address compliance:
►HIPAA
►PCIDSS
►Sarbanes-Oxley
►NIST
800-53
(SOX)
Database Activity Management
► Features




include:
Event aggregation
Correlation
Reporting
Auditing
► Does
not require access to native database
audit functions
Database Activity Management
► Privileged
User Monitoring:
 Monitoring privileged users:
►DBAs
►Sysadmins
►Developers
 who typically have unfettered access to corporate
databases
 Protects against external and internal threats
Database Activity Management
 Monitors all activities and transactions
 Identifies anomalous activities
►Viewing
sensitive data
►Creating new accounts
 (with superuser privileges?)
►Adding
or Deleting tables
Database Activity Management
► Most
organizations have perimeter
protection
► The next need is to monitor and protect
privileged user accounts
Database Activity Management
► There
is a high correlation between and
protection from the insider.
► Privileged users are capable of:
 Stored procedures
 Triggers
 Views
Database Activity Management
► Targeted
attacks frequently result in
attackers gaining privileged user
credentials:
 Monitoring of privileged activities is also an
effective way to identify compromised systems.
Database Activity Management
► Privileged
user monitoring helps ensure:
 Data Privacy
 Integrity
Implementation of Logs
► Procedures
and Tools to Implement
and Automate this Control
 Most Everything allows logging
 Evaluate what is and what is not being logged.
►compare
them with the asset inventory
Implemenatation of Logs
► Manual
inspection
 analyze logs on individual devices
 correlation (SIEM) tools can make audit logs far
more useful
Implementation of Logs
► SIEM
& Consolidation tools can be quite
helpful in identifying subtle attacks.
 These tools are not a replacement for skilled
information security personnel and system
administrators.
 Even with automated log analysis tools, human
expertise and intuition are often required to
identify and understand attacks.
Log Measurement - Manual
► Item:
Network time protocol (NTP)
 Measurement: Confirm that NTP is being
used to synchronize time for all devices
and that all clocks are in synch.
►Pass
or fail.
Log Measurement - Manual
► Item:
Vulnerability scanner
 Measurement: Run a non intrusive
vulnerability scanner against random
servers.
►Review
logs to determine whether the
information appeared in the logs.
Pass or fail.
Log Measurement - Manual
► Item:
Security Event Information
Management system (SEIM / SIEM /
etc)
 Measurement: Correlate logs to a central
source and determine that all servers are
properly logging.
►Compare
to inventory list
►Start @ 100% and back off 5% for each device
not logging.
SIEM
► security
information and event management
► security incident and event management
SIEM
► SIEM
 is a computerized tool used on data networks to
centralize the storage and interpretation of logs,
and events.
 The logs and events are generated by other
hardware and software products on the
network.
SIEM
► SIEM
should include:
 Gathering the logs
SIEM
► SIEM







should collect logs from:
Syslogs
Firewall logs
IDS logs
Windows server logs
Database logs
Web server logs
Application logs.
SIEM
► SIEM
may need to:
 Handle multiple data centers
 Collect data centrally
SIEM
► SIEM
Reports:
 Correlating
►Their
big value is in the correlation of data from
multiple sources in multiple formats.




Regulatory
Trouble-shooting
Investigating
Alerting
SIEM
► SIEM
can also detect:
 Distributed attacks
 Complicated attack paths
 Insider abuse
► As
well as:
 Normal network performance failures
►(Requires
a capable analyst)
Commercial Software Products
► CorreLog
Enterprise Server – software $5k
► ManageEngine EventLog Analyzer $350 / 10
sources
► NitroSecurity NitroView $29k
► Prism Microsystems EventTracker $30k
► Tripwire Log Center $19k
10 Signs
you aren’t cut out for IT
► 10
signs that you aren't cut out for IT
 1: You lack patience
 2: You have no desire to continue your
education
 3: You refuse to work outside 9-to-5
 4: You don’t like people
NMap
► Nmap
 A security scanner originally written by Gordon
Lyon (aka Fyodor Vaskovich)
►1)
used to discover hosts
►2) user to discover services on hosts
►3) can determine;






which ports are open and closed
the operating system
names and versions of the listening services
estimated uptime
type of device
presence of a firewall.
Nmap
NMap
► DEMO
10 Signs
you aren’t cut out for IT
► 10
signs that you aren't cut out for IT
 1: You lack patience
 2: You have no desire to continue your
education
 3: You refuse to work outside 9-to-5
 4: You don’t like people
 5: You give up quickly
Windows Log Analysis
► Free
Products
 LogZilla (code.google.com/p/php-syslog-ng)
 Analyzes Syslogs
 Is PHP-based visual front-end
►For
syslog servers
 Searches
 Reports
 etc
Windows Log Analysis
► Free
Products
 Splunk ( Free for first 500mb )
Windows Log Analysis
► Free
Products
► OSSEC (ossec.net):
 an open source tool
 Analyzes Real-Time:
►Unix
systems
►Windows servers
►Network devices
 Includes:
►default
alerting rules
Windows Log Analysis
► Free
Products
 Snare agent & ProjectLasso
(intersectalliance.com/projects/index.html) and
remote collector
(sourceforge.net/projects/lassolog)
►Open
sourc tool
►Analyzes real-time:
 Windows Event Logs
 Syslog
 Network devices
Windows Log Analysis
► Free
Products
 Log2timeline (log2timeline.net/)
 Analyzes Logs
►Used
as an investigation tool it can create a timeline
view out of raw log data
►Runs on Linux and Mac using Perl
Windows Log Analysis
► Free
Products
 syslog-ng (balabit.com/networksecurity/syslog-ng/)
 Open Source
 Analyzes
Windows Logs by installing an agent on the server
►Syslog
►
SpiceWorks
► Spiceworks
headquartered in Austin, Texas.
► It was formed in early 2006 to provide a
Facebook-like community integrated with a
free ad-supported IT:
 Systems Management
 Inventory
 Help desk software
SpiceWorks
► The
product is designed for network
administrators working in small- to mid-sized
businesses and managing up to a few thousand
network devices.
SpiceWorks
► SpiceWorks
discovers:
Windows
► Unix
► Linux
► Mac OS X
► Routers
► VOIP phones
► Printers
► etc.
►
SpiceWorks
► DEMO
10 Signs
you aren’t cut out for IT
► 10
signs that you aren't cut out for IT
 1: You lack patience
 2: You have no desire to continue your
education
 3: You refuse to work outside 9-to-5
 4: You don’t like people
 5: You give up quickly
 6: You’re easily frustrated
What exactly is WireShark?
► Wireshark
is a sniffer
 Sniffer, Packet Analyzer, also known as a network
analyzer, protocol analyzer.
 A software product and / or hardware product that
has the ability to intercept all network traffic and
allow for the analysis of the packets of data
contained in that traffic.
What exactly is WireShark?
► Originally
named Ethereal, Wireshark is a
free and open-source packet analyzer. It is
used for network troubleshooting, analysis,
software and communications protocol
development, and education
►It
runs on all popular computing
platforms, including Unix, Linux, and
Windows.
What is a sniffer?
► Protocol





Analyzer
Who is talking to who
What they are saying
Header and Overhead
Payload
Problems
Use of a sniffer for security:
► Basic
Information:
 7 layer OSI model
►Application
►Presentation
►Session
►Transport
►Network
►Datalink
►Physical
layers
Use of a sniffer for security:
►Many
believe that you look at everything from the
bottom up just like:
 Building a building
 Math
 Employment, etc
►But, that is not always true….
 With a house you start with the overall design….
►In the case of sniffing
 It is best to start somewhere in the middle, usually at TCP
or ICMP, then move down or up based upon what you
discover and what you are looking for.
Requirements of a Sniffer
► Hubs
► Switches
► Switches
 Port Mirroring
 Switched Port Analyzer (SPAN)
 Roving Analysis Port (RAP)
► Local
Connection must operate in
promiscuous mode
Example Sequence 1
► DHCP
request and response
► ARP for gateway
► DNS “A” request and response
► Web session setup
► Payload delivery
DHCP
DHCP: src & dst
DHCP: IP Component
DHCP: UDP Component
DHCP: BootStrap Parameters
DHCP: First Packet Results
DHCP: Second Packet Results
DHCP: Third Packet Results
DHCP: Fourth Packet Results
10 Signs
you aren’t cut out for IT
► 10
signs that you aren't cut out for IT
 1: You lack patience
 2: You have no desire to continue your
education
 3: You refuse to work outside 9-to-5
 4: You don’t like people
 5: You give up quickly
 6: You’re easily frustrated
 7: You can’t multitask
DNS
► DNS
is the internet’s system to translate the
human readable name such as
EvidenceSolutions.com to the actual IP
address of the desired site.
DNS: Request
DNS: Response
We get to the website: syn
We get to the website: syn & ack
Website final: ack
Website Sends Graphics
How to use the product….
► Basics
out of the way
 Start at the bottom & work your way up
►Works
great for troubleshooting
 Or start at the top an work your way down
►Not
so good for security- top down is best here
► Real
world virus example
 Multiple protocols
 Disguised traffic
 Peel the onion to analyze
Real World
STATISTICS- PROTOCOL
HIERARCHY
TCP CONVERSATIONS
Real World
► Compare
to your list of valid IP’s
► Compare to your list of valid protocols
► Compare with what you have seen in the
past
► Drill down into any anomalies or unusual
instances
FTP Traffic High…
FTP Traffic High… to .10
FTP Traffic High… between .10 & .25
.25 is the culprit but what is it doing
Conclusion for this scan:
► Normally,
a port scan shows up very loudly
and easily
► In this case, .2 was controlling .10 via a
remote trojan on port 25 (mail) . .10 was
passing the instructions on to .25 via port
21 (ftp)
Conclusion for this scan:
► All
of this trojan traffic passes firewall rules
and doen’t get a second glance.
► You would have busted .25 for the port
scan, but left .10, .25 infected and the
master of it all, .2, is still at large.
► Cool
sample traffic, including security
issues:
 http://wiki.wireshark.org/SampleCaptures#hea
d6c6fb4051dfbe9b992057ea1533eb8dc85c9a13a
► Filters
 http://wiki.wireshark.org/DisplayFilters
10 Signs
you aren’t cut out for IT
►
10 signs that you aren't cut out for IT










1: You lack patience
2: You have no desire to continue your education
3: You refuse to work outside 9-to-5
4: You don’t like people
5: You give up quickly
6: You’re easily frustrated
7: You can’t multitask
8: You have dreams of climbing the corporate ladder
9: You hate technology
10: You turn off your phone at night
►
By Jack Wallen; February 24, 2012
Evalution
►I
value your comments. Please fill in your
evaluation form found at the end of your
packet.
Scott Greene: Other topics available
Computer Forensics
Computer Forensics for Defense Attorneys
Personal Privacy in the Information Age
High Technology: Just where is technology going?
Bypassing Security: How They Steal Company Data
Fundamentals of Digital Forensics
Technology Forensics: Theory & Potential... is it Science or Art?
Technology Forensics: Case Examples
Technology Forensics: Intellectual property and identity theft
Technology Forensics: Hardware and Software tools / Show and Tell
Portable Devices Issues and Answers: A discussion about cell phones and the stories
they can tell.
► Anti-Digital Forensics. Or is it Digital Anti-Forensics?
► Data Security and Confidentiality Issues
► E-mail: The digital Smoking Gun
►
►
►
►
►
►
►
►
►
►
►
Contact Information
Scott Greene, SCFE
Evidence Solutions, Inc
866-795-7166
[email protected]