Transcript CSSE Template
University of Southern California
Center for Systems and Software Engineering
Value Driven Security Threat Modeling for Off The Shelf Software Systems
Yue Chen USC Center for Systems and Software Engineering Emails: {yuec, boehm, lshep}@usc.edu
Feb 14, 2007 CSSE Annual Research Review Los Angeles, U.S.A.
©USC-CSSE 1
University of Southern California
Center for Systems and Software Engineering
Trends: Increasing Concerns on COTS Vulnerability
Increasing Trend of COTS Based Applications •
Increasing COTS usage
Data source:
[Boehm et al 2003][Standish]
•
Increasing number of COTS vulnerabilities published
Data source:
[CERT Statistics] ©USC-CSSE
University of Southern California
Center for Systems and Software Engineering
Prioritizing COTS Vulnerability is Important
• Significant Loss:
CSI/FBI 2006 Survey reported a total loss of $52,494,290 caused by security incidents from 313 organizations
• Limited resources for security:
47% spend equal or less than 2% in IT security
Data source: CSI/FBI Computer Crime and Security Survey 2006
• Challenge:
How to spend limited amount of security resources smartly?
©USC-CSSE
University of Southern California
Center for Systems and Software Engineering
Current Practices
• System context neutral, static vulnerability rankings
are recommended by authority organizations such as BugTraq/Symantec, CERT, ISS, NIST, Microsoft, SANS
• Coarse granularity of rankings:
three ~ four levels of rankings (e.g. Critical, Important, Moderate, Low) are used to differentiate 23,620 known COTS vulnerabilities
• An Real-life Story –
USC Ticketing Office System (Source: USC-ISD)
• Decisions made based on
[Butler, 2002]
– – –
Best knowledge Individual experience Ad hoc ©USC-CSSE
University of Southern California
Center for Systems and Software Engineering
Proposal: Threat Modeling framework based on Attack Path analysis (T-MAP)
A novel threat modeling framework for COTS based systems that is sensitive to system stakeholder value context, dynamic, and tool-automated
Current Approaches : COTS Vulnerability Value Neutral Assessment COTS Vulnerability Rankings T- MAP: COTS Vulnerability Value Neutral Assessment Scenario Evaluation COTS Vulnerability Rankings Evaluation criteria based on stakeholder value propositions ©USC-CSSE
University of Southern California
Center for Systems and Software Engineering Permitted Ports
Nature of The Problem
Attacking Paths Unblocked vulnerabilities Vulnerabilities impacting confidentiality, availability, integrity Blocked vulnerabilities Firewall Wrapper e.g. Windows Server 2003 e.g. SQL Server 2000 Software Applications, COTS e.g. Web Server e.g. CRM Server IT Infrastructure e.g. IIS 6.0
e.g. Regulatory Productivity Org. Values Reputation ©USC-CSSE T-MAP 6
University of Southern California
Center for Systems and Software Engineering
Castle Defense Analog
Measure the security of a castle by the value of treasures in the castle, the number of holes on the walls, as well as the size of the holes.
©USC-CSSE T-MAP 7
• • • • • • University of Southern California
Center for Systems and Software Engineering
T-MAP Framework
Step 1:
Identify key stakeholders and value propositions (the treasures in the castle);
Step 2:
Establish a set of security evaluation criteria based on stakeholder value propositions;
Step 3:
Use tool to enumerate and analyze attack paths based on a comprehensive COTS vulnerability database containing 23,620 vulnerability information (the holes);
Step 4:
Evaluate the severity of each scenario in terms of numeric ratings against the evaluation criteria established in Step 2 (the size of the holes);
Step 5:
The security threat of each vulnerability is quantified with the total severity ratings of all attack paths that are relevant to this vulnerability;
Step 6:
System total threat is quantified with the total severity ratings of all attack paths; [Note] Step 3 to 6 are tool automated by the
Tiramisu
©USC-CSSE
Tool
T-MAP 8
University of Southern California
Center for Systems and Software Engineering
Step 1-2: Evaluate Security against Criteria based on Stakeholder/Values
• • •
Evaluate the severity of security hazard scenarios against stakeholder/value impacts Involves both qualitative and quantitative criteria Technical approach:
Figure of Merits
and
Analytical Hierarchy Process (AHP)
–
A convenient tool to determine the priority of alternatives in terms of weight through pair-wise comparison
–
Invented by Dr. Saaty at Business School of Univ. of Pennsylvania in 1970s
[Saaty, 1980]
–
Recommended by Dr. Bodin, Dr. Gordon et al for security investment evaluation
[Bodin et al, 2005] ©USC-CSSE T-MAP 9
University of Southern California
Center for Systems and Software Engineering
Using AHP Determine the Weights – An Example
Example (from USC ISD Server X Case Study)
Weights derived through AHP pair-wise comparisons Possible Breach Scenarios Value Centric Criteria ©USC-CSSE T-MAP 10
University of Southern California
Center for Systems and Software Engineering
• •
Step 3-4: Enumerate Attack Scenarios
Enumerate the scenarios
how an attacker can compromise stakeholder values through COTS system vulnerabilities
Establish Structured Attack Graph
based on a comprehensive COTS vulnerability database involves 23,620 known vulnerabilities reside in 31,713 COTS software
©USC-CSSE T-MAP 11
University of Southern California
Center for Systems and Software Engineering
Step 5: Vulnerability Technical Severity Evaluation
•
Vulnerability Technical Severity Attributes
–
Impact on confidentiality, integrity and/or availability
– –
Remotely exploitable Require valid user account on victim host
–
Needs user activities
Based on an emerging national standard [CVSS] by NIST
©USC-CSSE T-MAP 12
University of Southern California
Center for Systems and Software Engineering
•
Step 6 T-MAP Severity Rating System
Severity Weight of Attack Path
P
:
•
Overall Security Threat Score of COTS System
G
:
•
ThreatKey of elements in Attack Graph:
•
Effectiveness of Security Practice: ©USC-CSSE T-MAP 13
University of Southern California
Center for Systems and Software Engineering
Security Investment Effectiveness Estimation
• How much security threats can be avoided by implementing Firewall, Software hardening (patching), user account control, or file system encryption?
• Results as well depends on the total value of the protected system * Case study results estimated by professional security manager at USC-ITS
©USC-CSSE T-MAP 14
University of Southern California
Center for Systems and Software Engineering
Tool Implementation – Project Code:
Tiramisu
How it works?
High-level software architecture CERT Symantec BugTraq FrSIRT NIST Microsoft SANS Vulnerability Information Scrawling Engine T-MAP Vulnerability DB
Tiramisu Tool
Tiramisu Front End
©USC-CSSE T-MAP 15
University of Southern California
Center for Systems and Software Engineering
Security Manager’s Ranking vs. Tiramisu Ranking
©USC-CSSE T-MAP 16
University of Southern California
Center for Systems and Software Engineering
Security Economics in Patching
• • •
Prioritize COTS Based System vulnerabilities under business context
–
“20% percent of vulnerabilities causes 80% of the security risks”, T-MAP tells what are the 20% Rational: Prioritize vulnerabilities with its
ThreatKey;
Example screenshot: ©USC-CSSE 17
• University of Southern California
Center for Systems and Software Engineering
COTS Security Economics – Finding Sweet Spots
•
Sweet spot to invest in Economic curve of security security patching
•
Also driven by the total (from USC Server X case study) value of system (from USC Server X case study)
Sweet spots to invest
©USC-CSSE 18
• • • • • University of Southern California
Center for Systems and Software Engineering
Limitations
Only sensitive to known COTS Vulnerabilities
–
Empirical study by Arora shows that the average attacks per host per day jumped from 0.31 to 5.45 after vulnerability get published Not sensitive to nuance in local system configurations
– –
Disabled services Services running on different privileges, etc.
Only cover “one-step-attacks” that exploiting COTS vulnerabilities Depends on comprehensive vulnerability database
–
Our database: 23,260 vulnerability published from 1999-2006 that resides in 31,313 COTS software Cannot effectively address attacks such as Phishing ©USC-CSSE T-MAP 19
University of Southern California
Center for Systems and Software Engineering
Hypothesis
Hypothesis #1 *:
For given COTS based system whose confidentiality, integrity and availability have different priorities to the stakeholder values, the accuracy of T-MAP results, measured by the Inaccuracy, the ratio of
the number of clashes between vulnerability priorities and stakeholder value priorities
and
the total number of comparisons
, will not make any difference comparing to the existing leading approaches (say, the average level of at least three leading approaches, if rating data available).
Clash
definition:
for two
technically identical
the stakeholder value impact severity of V prioritizing system assign V a
clash
A for this prioritizing system.
A vulnerabilities V A and V B , given is higher than V B , if a less or equal priorities as V B , it is counted as
* Hypothesis #1 is a null hypothesis.
studies and tool demos. We aim at disprove them through case Clash Counting Illustration Vuln.
V A
©USC-CSSE -
V B
X -
University of Southern California
Center for Systems and Software Engineering
Inaccuracy Comparison
•
Inaccuracy *
: the ratio of
the number of clashes between vulnerability priorities and stakeholder value priorities
and
the total number of comparisons * Inaccuracy
is a numerical value between 0~1, the larger value indicates the less accuracy For the USC ISD case study, the security manager prioritized 8 vulnerabilities, which translates to 28 comparisons, where 28 = 1+2+… + 7 T-MAP CVSS ISS # of Clashes 2 6 9
Inaccuracy 0.071
0.214
0.321
©USC-CSSE
University of Southern California
Center for Systems and Software Engineering
• •
Contributions
A framework to model CBS security
–
A COTS security evaluation framework that captures stakeholder value propositions
–
Distill the potential impacts of thousands of vulnerabilities into management friendly numbers at a high-level
–
Tool automated
Has the potential to pro-actively attack security in early life-cycle, instead of taking whatever as is reactively after system built
©USC-CSSE
University of Southern California
Center for Systems and Software Engineering
Discussions and Future Work
©USC-CSSE 23
University of Southern California
Center for Systems and Software Engineering
Thanks!
©USC-CSSE 24