Transcript Slide 1
A Mission-Centric Framework for Cyber Situational Awareness Metrics, Lifecycle of Situational Awareness, and Impact of Automated Tools on Analyst Performance S. Jajodia, M. Albanese George Mason University ARO-MURI on Cyber-Situation Awareness Review Meeting Santa Barbara, CA , November 18-19, 2014 Outline 2 Overview of Mason’s Role Year 5 Statistics Metrics Measuring Network Security Risk Diversity Lifecycle of Situational Awareness Impact of SA on Analyst Performance Conclusions ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 3 Overview of Mason’s Role ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Where We Stand in the Project 4 • • • Data Conditioning Association & Correlation •Software •Sensors, probes •Hyper Sentry •Cruiser Computer network Real World Information Aggregation & Fusion •Transaction Graph methods •Damage assessment Automated Reasoning Tools • R-CAST • Plan-based narratives • Graphical models • Uncertainty analysis • Enterprise Model • Activity Logs • IDS reports • Vulnerabilities Multi-Sensory Human Computer Interaction Cognitive Models & Decision Aids • Instance Based Learning Models • Simulation • Measures of SA & Shared SA System Analysts • • • Test-bed Computer network ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Our Vision 5 Scenario Analysis & Visualization Vulnerability Databases Zero-day Analysis NVD CVE Network Hardening Unexplained Behavior Analysis OSVD Cauldron Analyst Topological Vulnerability Analysis Cauldron Index & Data Structures Switchwall Stochastic Attack Models Graph Processing and Indexing Situation Knowledge Reference Model [Attack Scenario Graphs] Dependency Analysis Monitored Network NSDMiner Generalized Dependency Graphs Alerts/Sensory Data ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Overview of Contribution – Year 1 6 Technical accomplishments A topological approach to Vulnerability Analysis that overcomes the drawbacks of traditional point-wise vulnerability analysis Preliminary data structures and graph-based techniques and algorithms for processing alerts/sensory data A novel security metric, k-zero day safety, to assess how many zero-day vulnerabilities are required for compromising a network asset Major breakthroughs Capability of processing massive amounts of alerts in real-time Capability of forecasting possible futures of the current situation Capability of hardening a network against zero day vulnerabilities ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Overview of Contribution – Year 2 7 Technical accomplishments Generalized dependency graphs, which capture how network components depend on one other Probabilistic temporal attack graphs, which encode probabilistic and temporal knowledge of the attacker’s behavior Attack scenario graphs, which combine dependency and attack graphs Efficient algorithms for both detection and prediction A preliminary model to identify “unexplained” cyber activities, i.e., activities incompatible with any given known activity model Major breakthroughs Capability of generating and ranking future attack scenarios in real time ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Overview of Contribution – Year 3 8 Technical accomplishments An efficient and cost-effective algorithm to harden a network with respect to given security goals A probabilistic framework for localizing attackers in mobile networks A probabilistic framework for assessing the completeness and quality of available attack models (joint work with UMD and ARL) A suite of novel techniques to automatically discover dependencies between network services from passively collected network traffic Switchwall, an Ethernet-based network fingerprinting technique for detecting unauthorized changes to the L2/L3 network topology Major breakthroughs Capability of automatically and efficiently executing several important analysis tasks, namely hardening, dependency analysis, and attacker localization ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Overview of Contribution – Year 4 9 Technical accomplishments Effective and efficient methods for generating partial attack graphs on demand in order to enable efficient analysis of zero-day vulnerabilities A three-step process to assess the risk associated with zero-day vulnerabilities A prototype of the probabilistic framework for unexplained activity analysis Major breakthroughs Capability to reason about zero-day vulnerabilities and efficiently assess the risk associated with such vulnerabilities without generating the entire attack graph ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Overview of Contribution – Year 5 10 Technical accomplishments A suite of metrics for measuring network-wide cyber security risk based on attack graphs An approach to model network diversity as a security metric for evaluating the robustness of networks against zero-day attacks An analysis of how situational awareness forms and evolves during the several stages of the cyber defense process An analysis of how automated CSA tools can be used for improving analyst performance Major breakthroughs Capability of quantifying risk and resiliency using several metrics ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Quad Chart - Year 5 11 Objectives: Improve Cyber Situation Awareness via • Metrics for measuring network-wide cyber security risk • An better understanding of the impact of network diversity on the robustness of networks against zero-day attacks • A better understanding of how situational awareness forms and evolves • A better understanding of how automated CSA tools can improve analyst performance DoD Benefit: • Ability to quantitatively evaluate network-wide security risks • Ability to better design automated CSA tools that can effectively reduce the workload for the analysts and improve their performance Scientific/Technical Approach • Defining a hierarchy of attack graph based metrics, and developing metrics • Studying diversity as a network-wide metrics to asses resilience against zero-day attacks, and defining several diversity-based metrics: biodiversity inspired, least attacking effort, and average attacking effort • Studying situational awareness capabilities from a functional point of view, and identifying inputs, outputs, and lifecycle of the derived awareness • Examining the impact of automated tools on analyst performance ARO-MURI on Cyber-Situation Awareness Review Meeting Major Accomplishments • Defined a suite of metrics for measuring network-wide cyber security risk based on a model of multi-step attack vulnerability (attack graph) • Modeled network diversity as a security metric for evaluating the robustness of networks against zero-day attacks • Studied how situational awareness forms and evolves during the several stages of the cyber defense process, and how automated CSA tools can be used for improving analyst performance Challenges • Defining solid metrics that accurately capture risk and resilience November 18-19, 2014 12 Year 5 Statistics ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Year 5 Statistics (1/2) 13 Publications & presentations 3 papers published in peer-reviewed conference proceedings 1 paper published in a peer-reviewed journal 2 book chapters 1 book L. Wang, M. Albanese, and S. Jajodia, “Network Hardening: An Automated Approach to Improving Network Security,” ISBN 978-3319-04611-2, SpringerBriefs in Computer Science, 2014, 60 pages Supported personnel 2 faculty 1 doctoral student 1 undergraduate student ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Year 5 Statistics (2/2) 14 Patents Awarded during the reporting period Patents Disclosed during the reporting period Sushil Jajodia, Lingyu Wang, and Anoop Singhal, “Interactive Analysis of Attack Graphs Using Relational Queries”, United States Patent No. 8,566,269 B2, October 22, 2013. Steven Noel, Sushil Jajodia, and Eric Robertson, “Intrusion Event Correlation System”, United States Patent No. 8,719,943 B2, May 6, 2014. Massimiliano Albanese, Sushil Jajodia, and Steven Noel, “Methods and Systems for Determining Hardening Strategies”, United States Patent Application No. US 2014/0173740 A1, June 19, 2014. Honors & Awards Max Albanese received the 2014 Mason Emerging Researcher/Scholar/Creator Award ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 15 Metrics: Measuring Security Risk Steven Noel and Sushil Jajodia, “Metrics suite for network attack graph analytics,” Proceedings of the 9th Cyber and Information Security Research Conference (CISR 2014), Oak Ridge, TN, USA, April 8-10, 2014 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Overview 16 Attack (vulnerability dependency) graphs Combine information about topology, policy, and vulnerabilities Identify network vulnerability paths Provide qualitative rather than quantitative insights Attack graph metrics Capture Enable Look trends over time comparisons across organizations at complementary dimensions of security ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Cauldron Attack Graph 17 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Attack Graph Metrics 18 Network Topology Attack Graph Analysis XML CSV Graphical Metrics Engine Firewall Rules Cisco ASA Cisco IOS Juniper JUNOS Juniper ScreenOS … Nessus Retina nCircle nmap … Host Vulnerabilities ARO-MURI on Cyber-Situation Awareness Review Meeting Metrics Dashboard November 18-19, 2014 Attack Graph Metrics Families 19 Victimization: Individual vulnerabilities and exposed services each have elements of risk Size: The size of attack graphs is a prime indication of risk The larger the graph, the more ways to be compromised Containment: Networks are generally administered in pieces (subnets, domains, etc.) We score the entire network across individual vulnerability victimization dimensions Risk mitigation should aim to reduce attacks across such boundaries Topology: The connectivity, cycles, and depth of the attack graph indicate how graph relationships enable network penetration ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Metrics Hierarchy 20 Network Score Metrics Family Victimization Existence Individual Metrics Exploitability Overall Size Containment Topology Vectors Vectors Connectivity Machines Machines Cycles Vuln Types Depth Impact ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Victimization Metrics 21 Existence – relative number of ports that are vulnerable (on a 0 to 10 scale) Existence 10 sv sv s n Exploitability – average CVSS Exploitability Exploitability i 1 eui U U Impact – average CVSS Impact Impact i 1 mui U , U ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Size Family: Vectors Metric 22 Across domains: explicit vectors vi , j Within domain (implicit vectors) mi 1mj v j i Attack vectorsva i mi 1 j i v j i , j vi , j d m d Total possible attack vec tors v p m 1i si m ARO-MURI on Cyber-Situation Awareness Review Meeting va Vectors Size 10 v November 18-19, 2014 p Size Family: Machines Metric 23 Non-vulnerable machines m j 1 m j d Vulnerable machines r i 1 ri d r Machines Size 10 rm ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Containment Family: Vectors Metric 24 Across domains: explicit vectors vi , j Within domain (implicit vectors) mi 1mj v j i Attack vectorsva i mi 1 j i v j i , j vi , j d m d Attack vectors across domains vc i , j vi , j d ARO-MURI on Cyber-Situation Awareness Review Meeting vc November 18-19, 2014 va Vectors Containment 10 Containment Family: Machines Metric 25 Victims within domain only d mw i mi m, mi V Victims across domains ma i mi m, mi V d Machines Containment 10 ma ma mw ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Containment Family: Vulnerability Types 26 Vulnerability types within domain only tw i ti mi m, ti mi V d Vulnerability types across domains ta i ti mi m, ti mi V d ta Vuln Types Containment 10 ARO-MURI on Cyber-Situation Awareness ReviewtMeeting a tw November 18-19, 2014 Attack Graph Connectivity 27 Motivation: Better to have attack graph as disconnected parts versus connected whole One Component Two Components Less Secure ARO-MURI on Cyber-Situation Awareness Review Meeting Three Components More Secure November 18-19, 2014 Topology Family: Connectivity Metric 28 w 1 Metric 101 d 1 1 component 4 components 5 components 1 1 Metric 101 10 11 1 4 1 Metric 101 7 11 1 5 1 Metric 101 6 11 1 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Attack Graph Cycles 29 Motivation: For a connected attack graph, better to avoid cycles among subgraphs More Secure Less Secure ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Topology Family: Cycles Metric 30 s 1 Metric 101 d 1 4 components 5 components 10 components 4 1 Metric 101 7 11 1 5 1 Metric 101 6 11 1 10 1 Metric 101 1 11 1 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Attack Graph Depth 31 Motivation: Better to have attack graph deeper versus shallower One Step Deep 2 Steps Deep Less Secure ARO-MURI on Cyber-Situation Awareness Review Meeting 3 Steps Deep More Secure November 18-19, 2014 Topology Family: Depth Metric 32 10 Metric nd si c 1 i c 1 i 1 i n Shortest path 3/8 Shortest path 4/8 3 Metric 101 5.7 8 1 4 Metric 101 4.3 8 1 ARO-MURI on Cyber-Situation Awareness Review Meeting Shortest paths 2/3 and 1/5 Metric 10 2 1 3 1 5 1 2.3 2 8 3 1 5 1 November 18-19, 2014 Metrics Dashboard 33 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Trend Summary 34 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Example Network Topology 35 Partner Domains DMZ Internal Domains ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Attack Graph – Before Hardening 36 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Attack Graph – After Hardening 37 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 38 Metrics: Network Diversity L. Wang, M. Zhang, S. Jajodia, A. Singhal, and M. Albanese, “Modeling Network Diversity for Evaluating the Robustness of Networks against Zero-Day Attacks,” Proceedings of the 19th European Symposium on Research in Computer Security (ESORICS 2012), Wroclaw, Poland, September 7-11, 2014 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Overview 39 Zero-day attacks are a real threat to mission critical networks Governments and cybercriminals are stockpiling zero-day vulnerabilities1 The NSA spent more than $25 million a year to acquire software vulnerabilities Example. Stuxnet exploits 4 different/complementary zero day vulnerabilities to infiltrate a SCADA network But what can we do about unknown attacks? 1 http://krebsonsecurity.com/2013/12/how-many-zero-days-hit-you-today/ ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 How Could Diversity Help? 40 Stuxnet’s attack strategy party (e.g., contractor) organization’s network machine with Siemens Step 7 PLC 3rd The degree of software diversity along potential attack paths can be considered a good metric for the network’s capability of resisting Stuxnet ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Existing Work on Diversity 41 Software diversity has long been regarded as a security mechanism for improving robustness The degree of diversity along potential attack paths is an indicator of the network’s capability of resisting attacks Tolerating attacks as Byzantine faults by comparing outputs or behaviors of diverse variants Limitations: At a higher abstraction level, as a global property of an entire network, network diversity and its impact on security has not been formally modeled ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Our Contribution 42 We take the first step towards formally modeling network diversity as a security metric We propose a network diversity function based on well known mathematical models of biodiversity in ecology We design a network diversity metric based on the least attacking effort We design a probabilistic network diversity metric to reflect the average attacking effort We evaluate the metrics and algorithms through simulation The modeling effort helps understand diversity and enables quantitative hardening approaches ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Bio-Diversity and Richness of Species 43 Literature on biodiversity confirms a positive relationship between biodiversity and the ecosystem’s resistance to invasion and diseases Richness of species Effective number or resources The number of different species in an ecosystem Limitation: ignores the relative abundance of each species Measures the equivalent number of equally-common species, even if in reality all species are not equally common Limitation: assumes all resources are equally different Similarity-Sensitive Effective Richness We can use a resource similarity function to account for differences between resources ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Resource Graph 44 Syntactically equivalent to an attack graph Models causal relationships between network resources (rather than vulnerabilities) Vertices: zero-day exploits, their pre- and post-conditions Edges: AND between pre-conditions, OR between exploits On which path should we compute the diversity metrics? ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Selecting the Least Diverse Path(s) 45 Intuitively, it should be the “shortest” path 1 or 2 have the minimum number of steps, but 4 may take less effort than 1! 2 or 4 have the minimum number of resources? But they both have 2 resources, so which one is better? 4 minimizes #resources/#steps? But what if there is a path with 9 steps and 3 resources? 1/3<2/4, but it clearly does not represent the least attack effort! ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Network Diversity in Least Attack Effort 46 We define network diversity as: 𝑚𝑖𝑛𝑖𝑚𝑢𝑚 # 𝑜𝑓 𝑟𝑒𝑠𝑜𝑢𝑟𝑐𝑒𝑠 𝑜𝑛 𝑎𝑛𝑦 𝑝𝑎𝑡ℎ 𝑚𝑖𝑛𝑖𝑚𝑢𝑚 # 𝑜𝑓 𝑠𝑡𝑒𝑝𝑠 𝑜𝑛 𝑎𝑛𝑦 𝑝𝑎𝑡ℎ Note: These may or may not be the same path! In this case: 2 (path 2, 4) / 3 (path 1, 2) Determining the network diversity is NP-hard Our heuristic algorithm only keeps a limited number of local optima at each step ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Network Diversity in Average Effort The least attacking effort-based metric only provides a partial picture of the threat We now define a probabilistic network diversity metric based on the average attacking effort Defined as 𝑝1 , 𝑝2 where 𝑝1 is the probability an attacker can compromise a given asset now, and 𝑝2 is the probability he/she can still compromise it if all the resources were to be made different (i.e., every resource type would appear at most once) ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 47 Simulation Results Accuracy and Performance 48 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 49 Lifecycle of Situational Awareness M. Albanese and S. Jajodia, “Formation of Awareness,” to appear in Cyber Defense and Situational Awareness, A. Kott, R. Erbacher, C. Wang, eds., Springer Advances in Information Security, 2014. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Cyber Defense Process at a Glance 50 The overall process of cyber defense relies on the combined knowledge of actual attacks and effective defenses It ideally involves every part of the ecosystem It also entails the participation of individuals in every role within the organization The enterprise, its employees and customers, and other stakeholders Threat responders, security analysts, technologists, tool developers, users, policymakers, auditors, etc. Defensive actions are not limited to preventing the initial compromise They also address detection of already-compromised machines and prevention or disruption of attackers’ subsequent actions The defenses identified deal with reducing the initial attack surface Hardening device configurations, addressing long-term threats (such as APTs), disrupting attackers’ command-and-control of implanted malicious code, and establishing an adaptive defense and response capability ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Cyber Defense Critical Functions 51 Learning from attacks Prioritization Establishing common metrics to provide a shared language for all parties involved to measure the effectiveness of security controls Continuous diagnostics and mitigation Prioritizing controls that will provide the greatest risk reduction and protection against current and future threats Metrics Using knowledge of actual attacks that have compromised a system to provide the foundation to learn from these events and build effective, practical defenses Carrying out continuous measurement to test and validate the effectiveness of current security controls, and to help drive the prioritization of the next steps Automation Automating defenses so that organizations can achieve reliable, scalable, and continuous monitoring of security relevant events and variables ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Cyber Defense Roles 52 Security Analyst Security Engineer Responsible for designing a security system or its major components Security Administrator Responsible for performing security monitoring, detecting security incidents, and initiating incident response Security Architect Responsible for analyzing and assessing existing vulnerabilities in the IT infrastructure, and investigating available tools and countermeasures Responsible for managing organization-wide security systems Security consultant/specialist Responsible for different task related to protecting computers, networks, software, data, and/or information systems against cyber threats ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Questions 53 Current situation. Is there any ongoing attack? If yes, where is the attacker? Impact. How is the attack impacting the enterprise or mission? Can we asses the damage? Evolution. How is the situation evolving? Can we track all the steps of an attack? Web Server (A) Catalog Server (E) DB Server (G) Behavior. How are the attackers expected to behave? What are their strategies? Forensics. How did the attacker create the current situation? What was he trying to achieve? Prediction. Can we predict plausible futures of the current situation? Local DB Server (B) Internet Mobile App Server (C) Order Processing Server (F) Local DB Server (D) ARO-MURI on Cyber-Situation Awareness Review Meeting Information. What information sources can we rely upon? Can we assess their quality? Scalability. How can we ensure that solutions scale well for large networks? November 18-19, 2014 1 – Current Situation 54 Is there any ongoing attack? If yes, what is the stage of the intrusion and where is the attacker? Capability Input IDS logs, firewall logs, and data from other security monitoring tools Output Effectively detecting ongoing intrusions, and identifying the assets that might have been compromised already A detailed mapping of current intrusive activities Lifecycle This type of SA may quickly become obsolete – if not updated frequently – as the intruder progresses within the system ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 2 – Impact 55 How is the attack impacting the organization or mission? Can we assess the damage? Capability Input Knowledge of the organization’s assets along with some measure of each asset’s value Output Accurately assessing the impact (so far) of ongoing attacks An estimate of the damage caused so far by the intrusive activity Lifecycle This type of SA must be frequently updated to remain useful, as damage will increase as the attack progresses ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 3 – Evolution 56 How is the situation evolving? Can we track all the steps of an attack? Capability Input Situational awareness generated in response to the questions 1 &2 Output Monitoring ongoing attacks, once such attacks have been detected A detailed understanding of how the attack is progressing Lifecycle This capability can help address the limitations on the useful life of the situational awareness generated in response to questions 1 & 2 ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 4 – Behavior 57 How are the attackers expected to behave? What are their strategies? Capability Input Past observations and knowledge of organization’s assets Output Modeling the attacker’s behavior in order to understand its goals and strategies A set of formal models (e.g., game theoretic, stochastic) of the attacker’s behavior Lifecycle The attacker’s behavior may change over time, therefore models need to adapt to a changing adversarial landscape ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 5 – Forensics 58 How did the attacker create the current situation? What was he trying to achieve? Capability Input Situational awareness gained is response to question 4 Output Analyzing the logs after the fact and correlating observations in order to understand how an attack originated and evolved A detailed understanding of the weaknesses and vulnerabilities that made the attack possible Lifecycle This information can help security engineers and administrators harden system configurations to prevent similar incidents from happening again ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 6 – Prediction 59 Can we predict plausible futures of the current situation? Capability Predicting possible moves an attacker may take in the future Input Situational awareness gained in response to questions 1, 3, and 4 Output A set of possible alternative scenarios that may realize in the future Lifecycle This type of SA may quickly become obsolete ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 7 – Quality of Information 60 What information sources can we rely upon? Can we assess their quality? Capability Input Information sources Output Assessing the quality of the information sources all other tasks depend upon A detailed understanding of how to weight different sources when processing information in response to other questions Lifecycle Needs to be updated when the information sources change ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 61 Impact of SA on Analyst Performance M. Albanese, H. Cam, and S. Jajodia, “Automated Cyber Situation Awareness Tools for Improving Analyst Performance,” Cybersecurity Systems for Human Cognition Augmentation, Springer 2014. ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Overview 62 Automated Cyber Situation Awareness tools and models can enhance performance, cognition and understanding for cyber professionals monitoring complex cyber systems In most current solutions, human analysts are heavily involved in every phase of the monitoring and response process Ideally, we should move from a human-in-the loop scenario to a human-on-the loop scenario Human analysts should have the responsibility to oversee the automated processes and validate the results of automated analysis of monitoring data To this aim, it is highly desirable to have temporal models such as Petri nets to model and integrate the concurrent operations of cyber-physical systems with the cognitive processing of analyst ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Petri Net Models for SA 63 P1: Firewall receives packets P2: Sensor’s measurements are collected P3: Vulnerability scanner scans P4: Recovery tools run P5: Reject firewall rule-matched packets P6: Pass rule-nonmatched packets P7: Attackability conditions of system P8: Vulnerabilities exist P9: Active malicious codes P10: Assets compromised P11: Impact of assets damages P12: Assets recovered partially P13: Available assets P14: Analyst observes events P15: Analyst considers potential actions P16: Analyst determines impact of actions Integrating Cybersecurity Operations with Cognitive Analytical Reasoning of Analysts P1 T1 deny P5 P2 P3 T2 T3 pass P8 P7 P6 P4 T4 T5 P9 T6 P10 T8 T10 T1: Apply firewall ruleset against packets T2: Alarm probability exceeds threshold T3: Find new vulnerabilities T4: Activated malicious packets 14 T5: Intrusion attempts T6: Propagate impact of damages T7: Patch vulnerabilities, and recover damages T8: Evict compromised non-recoverable assets 12 T9: Recover assets fully T10: Analyst creates a hypothesis T11: Analyst takes an action to verify his/her hypothesis T12: Analyst determines the difference (error) between actual impact and his/her intended impact of action P P15 P11 T7 P12 T9 P13 T11 T ARO-MURI on Cyber-Situation Awareness Review Meeting P16 November 18-19, 2014 64 Conclusions ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 Conclusions 65 The focus in Year 5 was on integration of previous contributions refinement of the CSA framework definition of metrics attack graph based diversity based better understanding the overall process lifecycle of CSA role of the analyst Some of these capabilities will be further refined in a side project ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014 66 Questions? ARO-MURI on Cyber-Situation Awareness Review Meeting November 18-19, 2014