Transcript No Slide Title

ART-SCENE: Modelling Complex
Design Trade-off Spaces with i*
Neil Maiden
Head of Centre
Centre for HCI Design
Centre for HCI Design
Overview
Our Centre and our research
– Who and what are we
ART-SCENE
– A research prototype for scenario-based requirements
engineering and trade-off decision-making
A pattern language for submarine manoeuvring
– Research questions and approach
– A pattern language modelled using i* formalism
– Research questions revisited
Future research to evaluate ART-SCENE
– Pattern language is baseline for scalable systems
engineering research
Centre for HCI Design
The Centre and its Environment
University Research Centre
– Independent department (www-hcid.soi.city.ac.uk) in
City’s School of Informatics (www.soi.city.ac.uk)
Objectives
– Undertake world-class basic and applied research into
the design of complex systems of which people are a
significant component
Staff/students
– Six academic, 7 research staff, 10 PhD students, 1
administrator and 1 visiting fellow
Research income
– £1.6m income from 11 new research projects started
since January 2000
Centre for HCI Design
SIMP Basics and Partners
Systems Integration for Major Projects
– Part of EPSRC’s Systems Integration Initiative
– August 2000-August 2003, £800,000, 15 person-years
Academic partners
– City: Neil Maiden and 2 research staff
– UMIST: Alistair Sutcliffe and 2 research staff
– QMW: Norman Fenton and 1 research staff
Industrial partners
– BAE SYSTEMS: Very complex systems of systems
– Kennedy-Carter: Allan Kennedy
– Intellectual Capital Services: Philip M’Pherson
Produce general outcomes for systems engineering
Centre for HCI Design
Our Research Objective
Deliver new decision-making capabilities
– Make requirement-architecture trade-offs through
scenario analyses
• Analyse platform behaviour with different architectures through
scenario analyses to determine requirement compliance
– Improve on ad-hoc modelling and decision-making
capabilities when engineering large systems
Key research question
– Is it possible to assess conformance of different
platform/equipment configurations to whole-system
requirements in the context of different operational
scenarios using a pattern-driven approach
– Informed by innovative theories of knowledge reuse
from analogical reasoning research?
Centre for HCI Design
Start by Defining Patterns
SIMP defines a pattern as
– A reusable architecture (the solution) to a collection of
interconnected requirements (the problem) in welldefined scenarios (the context)
– The architecture rationale in terms of decisions made to
trade-off requirements to select the best-fit architecture
solution
– The effectiveness of the solution architecture in the
defined scenarios, expressed as measurable levels of
requirements compliance
Therefore what we need is
– Semantics for expressing the elements of the problem,
solution and context of a pattern, interconnections
between these elements, and formal constraints on
constituent knowledge of a pattern
Centre for HCI Design
Defining Pattern Languages
SIMP defines a pattern language as
– A collection of patterns that, together, constitute an
organisation’s systems engineering knowledge for a
defined domain
– The space of possible architecture solutions that have
been considered to meet interconnected requirements,
and trade-offs in that space
– Associations between patterns in the pattern language,
expressed as associations between different elements in
2 or more patterns
– Facts and rules about requirements, architectures and
scenarios in the defined domain
– A glossary of terms about the elements of requirements,
architectures and scenarios in the defined domain
Centre for HCI Design
ART-SCENE
Research software prototype
– Analysing Requirements Trade-offs with Scenario-driven
Evaluations
– Developed to investigate our key research question
ART-SCENE is a partial prototype
– Implement using Microsoft VISIO2000, Visual Basic,
Access, Visual InterDev, and Excel
– Integrate with software tools for requirements
management and cost-effectiveness calculation
Develop and evaluate 2 versions during the project
– First version due in March 2002
– Incremental evaluation and implementation
Centre for HCI Design
What ART-SCENE Seeks…..
Integrated modelling and computational environment
Scenarios with
CREWS-SAVRE
Y
Y
Y
M
M
M
Scenario Analyser
scenarios
i* modelling
with REDEPEND
requirements
Y
MM
requirements
architecture
Requirement
compliance
Computed
outcomes
Y
Y
scenarios
M
Scenario
outcome
Patterns and
other analyses
Computed
outcomes
architectures
Architectures
with iUML
A large research challenge that might fail...
Centre for HCI Design
The ART-SCENE Architecture
Copernicus
Goals
Scenario
authoring
ReDepend
Architecture
Goals, tasks
& trade-offs
Scenario
elaboration
Generated
scenarios
New system
requirements
Scenario
walkthrough
Generated
scenarios
Requirements structures
Customer
scenarios
Scenario
Scenario
Scenario
Scenario
analysis
analysis
analysis
analysis
OCD
iUML
Patterns
Computed
scenario outcomes
Computation
Engine
Measures of
system
effectiveness
Centre for HCI Design
REDEPEND Software Prototype
VISIO2000-based software prototype
i* Modelling
palette
Common i*
goal structure
Soft goal
contribution
links
Centre for HCI Design
Using Patterns in Trade-Off Analysis
Key research question
– Analyse platform behaviour with different architectures
through scenario analyses to determine requirement
compliance
requirements
Scenario to
acquire
Y
Fit
criteria
Scenario
outcome
How architecture
Performance data
Y
Y
M
MM
Test
outcome
performs
in terms
of
requirements
against fit criteria
scenarios
Scenario to
analyse
architecture
Centre for HCI Design
Modelling Patterns Using i* Semantics
Link requirements and architectures
– Individual agent dependencies
• Link system, sub-system and adjacent system agents in terms
of goal and soft goal dependencies
– Individual means-end links
• Connect architecture components (tasks at the moment) to
system goals/soft goals
– Patterns of agent dependencies and means-end links
• Set of agent dependencies and means-end links that
encapsulate the association of an architecture to a requirement
Define the architecture trade-off space
– Negative contribution links between soft goals
• Definition of the design trade-off space in terms of soft goals
Centre for HCI Design
BAE SYSTEMS Pattern Language
Develop pattern language to model
– Knowledge of design trade-offs made about
manoeuvring systems of in-service submarines
• Stable domain that affords extensive reuse
– Elicit from BAE SYSTEMS expert systems engineers
Investigate 3 research questions
1. Can experts agree a set of categories of design tradeoff decisions that are reusable in systems engineering?
2. Can experts articulate the essence of the decision
categories that distinguish them from other decisions?
3. Can we model this essence using the I* formalism so
that the experts agree that it is a accurate
representation?
Centre for HCI Design
Method for Pattern Modelling
Worked with pairs of BAE SYSTEMS engineers
– Developed list of key trade-off decisions
Incremental modelling of patterns
– Open-ended 3-hour elicitation sessions with engineers
– First-cut modelling of each pattern with i*
– Focused 2-hour presentations to change and improve i*
pattern model with engineers
– Agree and sign-off i* pattern model
Further i* modelling of manoeuvring domain
– Standard SD model of manoeuvring agents and their
dependencies
– Standard SR model of manoeuvring soft goals and their
contribution links
Centre for HCI Design
Answering Our Research Questions
We posited 3 research questions
1. Can experts agree a set of categories of design tradeoff decisions that are reusable in systems engineering?
• YES: Results produced a list of >10 basic decision categories
2. Can experts articulate the essence of the decision
categories that distinguish them from other decisions?
• YES: Experts were able to verbalise and agree discriminating
characteristics of each decision category
3. Can we model this essence using the i* formalism so
that the experts agree that it is accurate representation?
• YES: Four comprehensive i* models of 4 patterns - Experts
stated that i* captured most essential characteristics
• Apart from elements of the solution architecture
– This implies the need for extensions to the semantics….
Centre for HCI Design
How To Improve the Patterns
Better representation of solution elements
– Architectures of agents, components and connections
– Extend with or link to the iUML approach
Continuous as well as discrete solution spaces
– Complex systems have continuous solution spaces
• Impossible to enumerate all solutions using a qualitative
modelling approach
– A means of linking characteristics of this solution space
with solution architecture and component elements
Attaching performance data to patterns
– We can define requirement-architecture trade-offs
– Performance data is complex in submarine design
• From complex, domain-specific simulation models… But….
Centre for HCI Design
ART-SCENE: Putting It All Together
Manoeuvring pattern language
– Provides the basis for assessing effectiveness of
patterns in trade-off decision-making
– Integrate the ART-SCENE prototype and evaluate it
using BAE SYSTEMS submarine manoeuvring and
Eurocontrol’s air traffic management domain
• REDEPEND i* modelling software tool
• CREWS-SAVRE scenario elaboration and generation
• Architecture modelling using formal iUML approach
Basis for exploring research question
– Is it possible to assess conformance of different
platform/equipment configurations to whole-system
requirements in the context of different operational
scenarios