Transcript Software Architecture Metrics - New Jersey Institute of Technology

(Some) Software Engineering
Research at NJIT
Sergio Bogazzi, Yao Fei
Chen, GuanJie Jiang,
BoYu, Ali Mili
Projects
Software Architecture Analysis
Verification and Validation of Online
Adaptive Systems
Software Engineering Trends
Analyzing Redundancy
Professor Ali Mili
2
Software Architecture Metrics
A Study of Change and
Error Propagations
Project Objective
Defining and Investigating Quality
Metrics for Software Architectures.
Applying these Metrics to Large Scale
Software Architectures.
Correlate the Metrics with Independent
Observations of Quality.
Professor Ali Mili
4
Research Plan
Define and validate computable metrics for
software architectures based on information
theoretic measures
Automate the process of computing these
metrics.
Apply the process and metrics to a NASA case
study.
Validate the correctness and usefulness of
these metrics.
Professor Ali Mili
5
Selected Quantitative Factors
Error Propagation. Significance for
reliability.
Change Propagation. Significance for
Maintainability.
Requirements Propagation. Significance for Evolvability.
Professor Ali Mili
6
What are these Metrics useful for?
By looking at the Error Propagation Metrics,
the architect would be able to identify
components that have high potential to
propagate errors through the system,
The error propagation matrices, are also very
useful in direction of test plans, critical
components that show high tendency to
propagate error to other system components
should be more thoroughly tested.
Professor Ali Mili
7
What are these Metrics useful for? (cont’d)
By looking at the Change Propagation Metrics
(CPM), the architect would be able to identify
components that have high potential to
propagate the changes inside them to other
components.
From CPM, the architect would also be able to
identify components that have high potential
to be affected by the changes in other
components
Professor Ali Mili
8
Project Overview
Define Information theoretic metrics
Coupling, cohesion, redundancy
Define quantitative factors that are relevant
to qualitative attributes of the architecture
Error Propagation
Change Propagation
Establishing analytical and/or empirical
relationship between the metrics and the
quantitative factors
Automate computation of the metrics
Professor Ali Mili
9
Premises of Our Approach
Architectural Level precludes a logic,
semantic-based approach; hence we use a
stochastic approach.
Information Theory has a wide range of
functions that quantify random variables.
We use analytical and empirical means to
correlate metrics to qualitative properties.
Professor Ali Mili
10
Recent Accomplishments
Further Analysis of the Correlation
between Error Propagation and
Computable Metrics.
Initiating the Analysis of Change
Propagation in the Context of State
Based Components.
Automation of the derivation of metrics
from architectural descriptions (in UML,
by Nicholay) and from source code (by
Sergio).
Professor Ali Mili
11
Information Coupling and
Cohesion Metrics
We treat coupling in the true
architectural sense, as a property of the
connector between components.
Coupling in our approach becomes a
function of an ordered pair of
components characterizing their
informational interdependence.
Cohesion of a component measures its
internal information flow.
Professor Ali Mili
12
Definition of Quantitative Factors:
Error Propagation
S
A
B
X
x`
OR
x
x
x`
Error
 The error propagation (EP) is a measure of the
likelihood that an error in the message sent by A will
propagate into B.
Professor Ali Mili
13
Static Error Propagation Matrix SEPM
(Flattened architecture of a NASA case study)
B
A
C1
C1
C2
C3
C4
C5
C6
C7
C8
C9
C10
1
0.18
0.11
0.17
0
0
0.23
0
0
0.08
C2
0.11
1
0.26
0.15
0.57
0.18
0
0
0
0
C3
0.38
0
1
0
0
0
0
0
0
0
C4
0.44
0
0
1
0
0
0
0
0
0
C5
0.58
0
0
0
1
0
0
0
0
0
C6
0.59
0
0
0
0
1
0
0
0
0
C7
C8
0
0
0
0
0
0
1
0
0
0
C9
0
0.5
0
0
0
0
0
1
0
0
C10
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
1
Error Propagation Coefficient (static) = 0.0503
Professor Ali Mili
14
Dynamic Error Propagation Matrix DEPM
(Flattened architecture of a NASA case study)
B
A
C1
C1
C2
C3
C4
C5
C6
C7
C8
C9
C10
1
0.37
0.03
0.03
0
0
0.32
0
0
0
C2
0.11
1
0.32
0.19
0.67
0.28
0
0
0
0
C3
0.15
0
1
0
0
0
0
0
0
0
C4
0.3
0
0
1
0
0
0
0
0
0
C5
0.76
0
0
0
1
0
0
0
0
0
C6
0.73
0
0
0
0
1
0
0
0
0
C7
C8
0
0
0
0
0
0
1
0
0
0
0
0.52
0
0
0
0
0
1
0
0
C9
C10
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
1
Error Propagation Coefficient (dynamic) = 0.0534
Professor Ali Mili
15
Future Work
Empirical Validation of Error Propagation
Analysis
UML Model
Extract
Model Info.
Components,
Connectors
Design
Experiments
Connector,
Message
Corrupt
Model
Simulate
and Log
Fault
Model
Summit
BASIC
Error Logs
Analyze
Logs
Empirical Error Propagation
MATLAB
Normal Log
Professor Ali Mili
16
Change Propagation
We define Change Propagation from
component A to component B as the
probability that a change in A due to
corrective/ defective maintenance requires a
change in B to maintain the overall function
of the system.
We are conducting empirical experiment and
trying to propose an analytical formula that
approximate change propagation
Professor Ali Mili
17
Analyzing Change Propagation: Simple Model
Model architectural components as input-output transformers
C : IC  OC, from the set of inputs (IC ) of the component C
to the set of its outputs (OC ).
Consider elementary two-component (pipeline?) architecture,
where the first component A “feeds” its output to the second
component B as its input. The functionality of the
architecture can be presented in the form of the following
diagram:
Professor Ali Mili
18
Empirical work
In our experiment, we randomly select a
large amount of changes within each
component, then go into the source code
level to see, if the change will propagate to
other components.
By computing for each pair of components (A,
B), the number for which change in A cause a
change in B, we are able to derive the change
propagation matrix.
Professor Ali Mili
19
Change Propagation Matrix
C1
C1
C2
C3
C4
1
1/11
2/11
1/11
C2
1
C3
1/5
C5
C6
C7
C8
1/11
1
C4
1
C5
1
C6
1/3
C9
C10
2/11
1/11
1/11
3/11
11
2/8
1/8
1
4/8
1
1/3
2/3
3
3/8
1/8
8
1/3
3
1/8
2/8
8
1
6/8
8
1/3
1
C12
1/3
3
1
C13
2/2
0
1
C14
2
1
C15
1/1
2/5
C17
C18
C18 Changes
0/2
1
1
C16
C17
2
1/3
1/3
C16
2/2
1/3
C11
C15
5
C8
1/8
C14
4/5
1
1/8
C13
3
4/8
C9
C12
2/3
C7
C10
C11
4/5
1
2/5
2/5
1/1
1/3
0
1/1
1
1
1/5
5
1
1/3
1/3
1/3
1
1
3
Sum
67
The experiment is still undergoing
Professor Ali Mili
20
Automated Tool
Simplify application of the theory and
methods developed to industrial-strength
systems
Give project manager an ability to get a
glimpse into the quality of software
architecture  Project Manager View
Configure the tool for handling specific
ADLs and CASE tools  Analyst View
Professor Ali Mili
21
Benefits
The developments of techniques for
measuring error/change propagation help
analysts identify trouble spots in the
architecture (advances the state of the art)
The development of automated tools help the
analyst apply these techniques on large
architectures (advances the state of the
practice)
Professor Ali Mili
22
Future Work
Use a tool such as SIAT TM, or Understand TM
to measure static information theoretic
metrics
Compare the measures obtained to those
obtained at the design level
Compare information coupling and cohesion
measures with traditional measures based on
McCabe.
Perfect the tools
Professor Ali Mili
23
Verification and Validation of
Online Adaptive Systems
Jian GuanJie, Ali Mili, NJIT
Bojan Cukic, Yan Liu, WVU
Professor Ali Mili
24
The Intelligent Flight Control System (IFCS)
IFCS Project: NASA DFRC; NASA ARC;
Boeing; ISR; WVU.
Professor Ali Mili
25
Traditional V&V Techniques
Fault Avoidance
Fault Removal
Fault Tolerance
All are inapplicable to adaptive
learning systems
Professor Ali Mili
26
Refinement-based V&V of
adaptive systems
Functional Envelope
Monotonic Learning
Safe Learning
Professor Ali Mili
27
Functional Envelope
H
F
R
X
X
Fh
Rh 
Rh
Y
Y

F
 h
Professor Ali Mili
28
Initial Weights
Input
Iteration Times with Output
10
20
50
100
500
2000
0=1.0
Converge
7285
(1,1)
0.95718
0.88763
0.64715
0.57213
0.50373
0.12861
0.04999388
(1,0)
0.88929
0.75727
0.49946
0.48526
0.51578
0.88881
0.95669980
(0,1)
0.88985
0.76131
0.50934
0.48949
0.51579
0.88868
0.95675480
(0,0)
0.74329
0.58170
0.41602
0.45580
0.50102
0.09960
0.03909299
0=0.5
7560
(1,1)
0.70029
0.58756
0.53496
0.52377
0.51087
0.14863
0.04999296
(1,0)
0.60074
0.51103
0.48290
0.48596
0.49385
0.87160
0.95670090
(0,1)
0.60987
0.52137
0.48944
0.48869
0.49424
0.87141
0.95675580
(0,0)
0.55051
0.49504
0.48686
0.49964
0.51760
0.11487
0.03909090
0=0.0
8926
(1,1)
0.50565
0.50740
0.50880
0.50976
0.51116
0.50880
0.04999402
(1,0)
0.48353
0.48525
0.48714
0.48836
0.48878
0.49985
0.95669870
(0,1)
0.49364
0.49367
0.49203
0.49037
0.48888
0.50006
0.95675415
(0,0)
0.51421
0.51434
0.51342
0.51227
0.51134
0.51613
0.03909125
0 [-0.3, 0.3]
8942
(1,1)
0.51080
0.51098
0.51101
0.51096
0.51118
0.50909
0.04999226
(1,0)
0.48304
0.48416
0.48627
0.48794
0.48876
0.49888
0.95670134
(0,1)
0.49480
0.49397
0.49197
0.49027
0.48885
0.49911
0.95675653
(0,0)
0.51010
0.51027
0.51051
0.51073
0.51137
0.51662
0.03908928
Professor Ali Mili
29
Monotonic Learning----MLP
Professor Ali Mili
30
Monotonic Learning----DCS
Professor Ali Mili
31
Safe Learning
h, Fh
( Rh
Professor Ali Mili
S)
32
Novelty Detection
Professor Ali Mili
33
Experimental Results of Novelty Detection Using Association Rule
Learning
Professor Ali Mili
34
Programming Language Trends:
An Empirical Study
Yao Fei Chen, Ali Mili
New Jersey Institute of Technology
Introduction
As part of the project of monitoring software
engineering technical trends, we are trying to
research the evolution of high level
programming language.
What are the possible factors which can affect
programming language trends?
How can we quantify these factors?
How can we watch/predict/adapt to/affect the
trends based the factors?
Professor Ali Mili
36
Possible Factors Which Affect Trends
Intrinsic factors are the factors which can be
used to describe the general design criteria of
programming language.
Generality
Orthogonality
Reliability
Maintainability
Efficiency
Simplicity
Implementability
Machine Independence
Extensibility
Expressiveness
Influence/Impact
Professor Ali Mili
37
Extrinsic factors are the factors which are not
directly related to the general attributes of a
programming language, but still can affect the
trend of programming language.
Institutional Support
Industrial Support
Governmental Support
Organizational Support
Grassroots Support
Technology Support
Professor Ali Mili
38
Quantifying Factors
Quantifying intrinsic factors:
All intrinsic factors should be considered when one designs a
programming language. So, we will try to go over all features of a
programming language to check if it matches these factors. Then, we
will assign a score to each factor for each programming language.
Quantifying extrinsic factors:
We will do survey for every extrinsic factor and assign scores for them.
Professor Ali Mili
39
Models Construction&Validation

Use statistics method to construct&validate
Models
Factors that
might
affect the
trend
I1
I2
I3
O1
O2
F (I1, I2, …, In)
Om
In
Factors that
can be used
to evaluate
the trend
Function used to
describe the
relationship among
factors.
Professor Ali Mili
40
Evaluating Programming Languages

By analyzing a set of programming language, we expect that we
can find out statistics models and use them to predict the future
trend of a programming language.
















COBOL
FORTRAN
LISP
ALGOL
PASCAL
C
C++
JAVA
ADA
ML
APL
MODULA
EIFFEL
PROLOG
SMALLTALK
SCHEME
Professor Ali Mili
41
Analyzing Redundancy
Ali Mili, NJIT
Bojan Cukic, WVU
Jules Desharnais, Laval University
Outgrowth of Dryden Project
What is Redundancy?
Does Redundancy characterize a state or its
representation?
Is all redundancy amenable to state redundancy?
Can we talk about redundancy without Fault
Tolerance?
Instant Redundancy vs Temporal Redundancy
Professor Ali Mili
43
Three Views of Redundancy
Non-Injectivity of Representation Function
Duplication of State information
Scale of Fault Tolerant Capabilities.
Professor Ali Mili
44
Non Surjectivity
Representation function from States to
Representations
Total (representability): integers.
Injective (precision): reals.
Surjective (Non-redundancy)
No representation functions satisfy all. Most
functions satisfy none.
Professor Ali Mili
45
Duplication of State Information
Axiomatization of functions quantifying
duplication
Zero for redundancy free state
One for simple duplication for redundancy
free state.
N-1 for N-modular redundancy
Continuous real values in-between.
Professor Ali Mili
46
Scale of Fault Tolerant
Capabilities
Levels of Correctness:
Correctness, Maskability, Recoverability,
Non-recoverability.
Recovery is necessary and sufficient:
Recoverability.
Sufficient conditions of recoverability.
Professor Ali Mili
47