Transcript Document
Vulnerability Databases:
Everything is Vulnerable
Brian Martin Jake Kouns
© Open Security Foundation 2005
Vulnerability Databases:
Everything is Vulnerable
•Overview •Inherent Problems •Important Issues •Major Players •Research and Rankings •Future
© Open Security Foundation 2005
Vulnerability Databases:
Everything is Vulnerable
Overview
© Open Security Foundation 2005
Vulnerability Database Overview
• • • •
What is a Vulnerability Database (VDB)?
Database of information on security vulnerabilities. Simple!
What about “dictionaries” (CVE) or “searchable indexes” VDB!
Key is realizing VDBs will have their focus
– Comprehensive Vulnerability Database – Focused Vulnerability Database – Vulnerability Notification Services – Value Added Services © Open Security Foundation 2005
Brief History
• • • •
First VDBs were private, mostly maintained by hackers or budding security geeks (before security professionals were common) First public database?
– Unix Known Problem List – Internal Sun Microsystems Bug List – Early CERT database
VDBs abandoned (Fyodor), sold to corporations (BID), or home grown (X-Force) Additional VDBs continued to be launched to meet different demands (Secunia, OSVDB)
© Open Security Foundation 2005
Basics of a VDB
• • •
Vulnerability information gathered Identification number/name assigned Adherence to standard format
• • • •
Ability to search and display data Optional: Mail lists (private or public) Exports for integration Other services
© Open Security Foundation 2005
Purposes of a VDB
• • • •
Provide accurate information on security vulnerabilities Provide historic reference on software bugs Provide information on solutions Provide innovations to help organizations deal with vulnerabilities But are they?
© Open Security Foundation 2005
WIIFM – What’s in it for me?
• • •
Alerting/Notification
– Information provided in timely fashion
Detailed Content
– Concise description, additional analysis, references
Organized Information
– Vulnerability statistics – Trending – Historical context © Open Security Foundation 2005
Vulnerabilities Trends
CERT Vulnerability Counts ( 1995-2004) © Open Security Foundation 2005
Who uses a VDB?
• • • •
Administrators Auditors Security Testers
– Penetration Testing – Vulnerability Assessments – Risk Management
Criminals
– Hackers, Crackers, Blackhats, Greyhats, OH MY! © Open Security Foundation 2005
Legalities and Liability
• • • •
Issues with disclosure
– Bug finder and irresponsible disclosure – Do VDBs have a responsibility to be ethical for bug finders?
Liability for providing information
– Liability for including exploit code?
Copyrights on information
– Including unedited original source? – Re-branding or re-writing?
Confusing lawsuits
– Tegam vs. Guillaume Tena (France) – Sybase vs. NGSS? (US) – HP vs. NGSS? (US) © Open Security Foundation 2005
VDB Sociology
• • • • •
VDBs are taken for granted by users Users need them but do not appreciate Users rely on a VDB for 'thoroughness', when they usually are not Users quote VDB information as gospel, as if VDBs confirm and validate every entry Users typically have favorite VDB, and only use that one
© Open Security Foundation 2005
Vulnerability Databases:
Everything is Vulnerable
Inherent Problems
© Open Security Foundation 2005
Inherent Problems with VDBs
• • •
Dependency
– If no entry for Product X, assumption it is secure – Assume information is accurate, becomes gospel – Rely on VDB to alert you?
Lack of Updates
– Hard to update old entries (why don’t new players care about
old entries?)
– Solutions not there or not fully updated – Workarounds not accurate or helpful
Thoroughness
– “multiple” entries – No digging for details – Ignoring obscure products © Open Security Foundation 2005
Inherent Problems with VDBs
• •
Lack of standard
– Naming conventions – “multiples” vs. breaking out entries – What deserves an entry at all
Accuracy and Integrity
– Who updates? What motivation to be accurate? – Myth/Fake – Why is the information inaccurate? • Poorly written advisory, Lousy research • Poor vendor communication/verification – Why do VDBs trust anything and everything they read? • Number of database entries matter © Open Security Foundation 2005
Inherent Problems with VDBs
• •
Pros & Cons of adding entries
– Fast • No external references • Incomplete or inaccurate information – Slow • Not timely like many people want
Statistics & Metrics
– Lack of classification (leads to problems) – Lack of severity (debate unto itself) • Not only based on remote vs. local … • Availability of exploit • Impact of exploit • Installation base of software © Open Security Foundation 2005
Inherent Problems with VDBs
•
Relying on Bug finders
– Double edge sword, good bug hunters provide great
information, many do not
– Vulns being reported although previously disclosed – Not including versions or vendor site, and not easily
Google'd
– Vague information, untested – Advisories without dates (big vendors especially
guilty.. MS, IBM, Novell, Sun, HP)
– People try to use bug finding as a way to advertise
their security services
© Open Security Foundation 2005
Inherent Problems with VDBs
•
What else?
– Many don’t make database easily available in
full or not portable
– Don’t support third party utilities and use – VDB snobs, refuse to reference certain other
databases or sources
– Narrow focus on where to find vulnerability
information (life outside Bugtraq)
– Often don’t give credit where due – […] © Open Security Foundation 2005
Vulnerability Databases:
Everything is Vulnerable
Important Issues
© Open Security Foundation 2005
Important Issues for VDBs
• •
Most issues are easily overlooked 7 key issues for a VDB to address
• User Dependency • Content Updates • Content Depth • Standards • Accuracy and Integrity • Statistics and Metrics • Integration Ability © Open Security Foundation 2005
User Dependency • • • •
Can you rely on a VDB?
Do you verify the VDBs statements?
Do you read into the information and make assumptions?
Rely on VDB to alert you?
© Open Security Foundation 2005
Content Updates
• • • •
Turnaround on new entries Older entries need attention
– Updated external references – Updated solutions – Updated information on risk ratings
Do all VDBs care about older entries?
Corrections to entries
© Open Security Foundation 2005
Content Depth
• • • •
Number of entries
– Catalogue all vulnerabilities or just major issues
Vague information on vulnerabilities
– Often due to poor research or vendor not providing
details (thus, external references are important)
Effort to correlate or research
– Weeding out duplicate entries
Types of products cataloged
– Not just about Windows and Unix anymore © Open Security Foundation 2005
Standards
• • • • • •
Definition of a Vulnerability Naming Conventions Dates Write-ups Risk Ratings Solutions
© Open Security Foundation 2005
Accuracy and Integrity
• • • •
Who maintains the data How are updates justified Motivation for entries Motivation for accuracy
© Open Security Foundation 2005
Statistics and Metrics
• • • • • • •
How many entries exist?
How many entries are missing?
How do we know?
How many entries have solutions?
How many are critical?
How many vulns per month/year?
How many vulns per vendor/product?
© Open Security Foundation 2005
Integration Ability
• • • • •
Can users change or ask for updates Is the data easy to obtain Does the VDB support 3 rd parties Does the VDB reference all information Can users dynamically pull information
© Open Security Foundation 2005
Vulnerability Databases:
Everything is Vulnerable
Major Players
© Open Security Foundation 2005
Major Players
• • •
Comprehensive VDBs
– BID - http://www.securityfocus.com/bid – CVE - http://www.cve.mitre.org/ – ISS X-Force - http://xforce.iss.net/ – OSVDB – http://www.osvdb.org/ – Secunia - http://www.secunia.com/ – Security Tracker - http://www.securitytracker.com/
Vulnerability Notification Services
– CERT - http://www.cert.org/ – CIAC Advisory - http://www.ciac.org/ciac/index.html
Value Added Services
– ICAT - http://icat.nist.gov/icat.cfm © Open Security Foundation 2005
BID
• • •
Started in 1999, acquired by SecurityFocus on 07/17/2002 Full time dedicated resources Free, 72 hour delayed information (SF researched)
© Open Security Foundation 2005
BID – Pros/Cons
•
Pros
– Brand awareness – Very detailed and technical information provided – Quick posting of new vulnerabilities due to hosting of Bugtraq
mail list
•
Cons
– Practices changed once acquired by corporation – Little response to feedback provided – Slow to load, banners ads a pain, 39 images per entry – Product information based on erroneous assumptions © Open Security Foundation 2005
CVE/ICAT
• • • • • •
MITRE and NIST Full time dedicated resources, federal funding CVE started in 1999, ICAT ~2000 Both claim not to be a VDB ICAT adds vulnerability classification and statistics to a predominantly CVE based database Free
© Open Security Foundation 2005
CVE/ICAT – Pros/Cons
•
Pros
– Detailed statistics and classification scheme – Easy ability to download entire database – Widely adopted, heavily integrated into security products •
Cons
– Heavy use of CVE for vulnerability information – CVE “candidate” process slow and backlogged – Limited external references (ICAT) © Open Security Foundation 2005
ISS X-Force
• • • • • •
Run by Internet Security System (ISS) Full time resources dedicated Started around Aug, 1997 VDB is free and public Heavily used and referenced in ISS security products Fast and courteous reply to emails with questions or errors
© Open Security Foundation 2005
ISS X-Force – Pros/Cons
•
Pros
– Very detailed, very thorough, historical entries – Fairly standard naming conventions – Very thorough external references •
Cons
– Disclosure Issues – Many entries related to IDS events, not classic vulnerabilities – No easy export, can’t easily integrate © Open Security Foundation 2005
OSVDB
• • • • • •
Open Security Foundation, 501(c)3 non-profit organization 3 project leaders, over 200 volunteers since inception First started on 08/30/2002 Free security information Security community driven Vendor dictionary, ethical disclosure service, active integration
© Open Security Foundation 2005
OSVDB – Pros/Cons
•
Pros
– Vendor Neutral, Un-biased – Integration with open source products – Broad source for data importation (sources, dates) – Very thorough, attention to detail, historical entries •
Cons
– Slow updates on new vulnerabilities – Relies on community for resources – Currently no long term funding © Open Security Foundation 2005
Secunia
• • • • •
Corporation located in Denmark Full time staff Launched 03/26/2003 Focus on timely vulnerability alerts Free mailing list of new vulns mailed daily
© Open Security Foundation 2005
Secunia – Pros/Cons
•
Pros
– Free mailing list – Very strong on monitoring vendor advisories and updates – Attempt to work with open source community •
Cons
– Lack of standards/confusing standards • Issues lumped into “multiple” entries • Same vulnerability assigned a dozen entries, one per linux vendor – Only focuses on new vulnerabilities – Some solutions not practical or helpful © Open Security Foundation 2005
Security Tracker
• • • •
Corporation in MD, USA Full time resources dedicated Started in 2002 Free weekly summary of vulnerabilities, fee for instant alerts
© Open Security Foundation 2005
Security Tracker – Pros/Cons
•
Pros
– Maintain their own standards, uniform entries – Includes data source for vulnerability – Good data importation, monitor broad source of
information
•
Cons
– No statistics – Limited external references © Open Security Foundation 2005
CERT
• • • • • •
Carnegie Mellon, funded by US government Full time staff dedicated Started in 1988, after Morris worm Advisories for important issues Maintains CERT-VU/KB Database National Cyber Alert System
© Open Security Foundation 2005
CERT – Pros/Cons
•
Pros
– US Federally funded and supported – Providing reports to technical and non-technical – Statistics provided •
Cons
– Limited vulnerabilities tracked – Provide early information for exorbitant fee – Not always willing to coordinate with security community – Serious questions about statistics, efficiency of staff/funds – Overlap with CIAC and others © Open Security Foundation 2005
CIAC
• • • • •
US funded and supported, DOE Full time dedicated resources Started in 1989 Advisories for major issues Free service
© Open Security Foundation 2005
CIAC – Pros/Cons
•
Pros
– Stability, around since 1989 – Updated regularly •
Cons
– Limited vulnerabilities covered – Limited external references – Many advisories reprinted, no value added – Overlap with CERT © Open Security Foundation 2005
Additional Resources
• • •
Vulnerability Sources Not Included:
– COOP = https://cirdb.cerias.purdue.edu/coopvdb/public/ – Dragonsoft - http://vdb.dragonsoft.com/ – FrSIRT - http://www.frsirt.com/english/index.php – Securiteam - http://www.securiteam.com/ – Sec Watch- http://www.secwatch.org/
Focused Vulnerability Database
– Nikto, Nessus – Sun, HP, IBM, Oracle, Microsoft, etc
Vulnerability Sharing Clubs
– http://www.idefense.com/ – http://www.immunitysec.com © Open Security Foundation 2005
Government Funded
• • • • •
CERT
– The CERT/CC is funded primarily by the U.S. Department of Defense and the
Department of Homeland Security, along with a number of other federal civil agencies. Other funding comes from the private sector. As part of the Software Engineering Institute, we receive some funds from the primary sponsor of the SEI, the Office of the Under Secretary of Defense for Acquisition and Technology.
CIAC
– U.S. Department of Energy (DOE) funded
CVE
– CVE is sponsored by the National Cyber Security Division (NCSD) at the U.S.
Department of Homeland Security. US-CERT is the operational arm of the NCSD.
ICAT
– ICAT is maintained by the National Institute of Standards and Technology.
US-CERT
– US-CERT is part of the Department of Homeland Security •
Little overlap? Consolidation? Oversight and audit?
© Open Security Foundation 2005
Vulnerability Databases:
Everything is Vulnerable
Research and Rankings
© Open Security Foundation 2005
Data Harvesting
• •
Where is information usually gathered?
– Mail lists (Bugtraq, Full-disclosure, Vulnwatch, Ntbugtraq) – Vendors (advisories)
Where else should information be gathered?
– Mail lists (Freshmeat, Vuln-dev, Dailydave, Pen-test, other
specialty security focused lists)
– Vendors (Changelogs, Knowledge bases, Vendor forums) – Exploit archives © Open Security Foundation 2005
VDB Incest
•
Who references who? Who refuses?
– CVE:
ISS , BID , Secunia , SecurityTracker , OSVDB
– BID:
CVE , Bugtraq , ISS , Secunia , SecurityTracker , OSVDB
– ISS:
CVE , BID , Secunia , SecurityTracker , OSVDB
– Secunia:
CVE , OSVDB
– SecurityTracker:
CVE , OSVDB , Nessus
– Nessus:
CVE , BID, OSVDB
– OSVDB:
CVE , BID , Secunia , SecurityTracker , ISS , Nessus , Snort , more
•
Red denotes an apparent refusal to reference, even if the original point of disclosure or only available source.
© Open Security Foundation 2005
VDB Ratings
• • • • • •
Based on important issues identified Score of 1-10 provided for each of the 7 key performance areas 1 = lowest, 10 = highest Ratings given for each issue per VDB Provides baseline for expectations for each service Identifies areas of improvements
© Open Security Foundation 2005
VDB Individual Rankings
• • •
Ratings For Each Category Top 3 VDBs Top 3 Areas for VDB Improvement
•
See research posted at: http://www.opensecurityfoundation.org
© Open Security Foundation 2005
Vulnerability Databases:
Everything is Vulnerable
Future
© Open Security Foundation 2005
Future of VDBs
• • • • •
Long way to go Hope to improve existing resources
– Better search interfaces – Better upkeep of older entries
More services available to more people Further integration into products Better statistics and trending
© Open Security Foundation 2005
Standardization of Definitions
• • •
Risk ratings Vulnerability Classifications
– Local vs. Remote (Remote Local) – Impact assessment (CIA) – Exploit availability – Access required to exploit (Dependencies)
Vulnerability definitions and terminology
© Open Security Foundation 2005
VDBs Suck - Expect More
• • • • • • • • • • •
20 years since inception, Limited improvements Same mechanism for updating/verifying info Very few classify or assign risk Still no standardized classification for the few who do Still no standardized risk value for the few who do Still offer limited search ability overall Many don't follow their own standards consistently Most still very weak on external references Barely any new services or ways to use information Many don't seem to care about the vuln disclosure process (why did it take 20 years for a vendor dict to emerge?) Bottom line, VDBs need to drastically improve
© Open Security Foundation 2005
Vulnerability Databases:
Everything is Vulnerable
Brian Martin – [email protected]
Jake Kouns – [email protected]
Open Security Foundation
© Open Security Foundation 2005