Transcript NF SEER-SEM - Center for Software Engineering

A Neuro-Fuzzy Model with
SEER-SEM for Software Effort
Estimation
Wei Lin Du, Danny Ho*, Luiz F. Capretz
Software Engineering, University of Western Ontario, London, Ontario, Canada
* NFA Estimation Inc., Richmond Hill, Ontario, Canada
November 2010
Agenda





Purpose
SEER-SEM
NF SEER-SEM
Evaluation
Conclusion
Purpose


Integrate neuro-fuzzy (NF) technique
with SEER-SEM
Evaluate estimation performance of NF
SEER-SEM versus SEER-SEM
Agenda





Purpose
SEER-SEM
NF SEER-SEM
Evaluation
Conclusion
SEER-SEM


SEER-SEM was trademarked by Galorath
Associates, Inc. (GAI) in 1990
Effort estimation is one of the SEERSEM algorithmic models
Size
Effort
Personnel
Cost
Environment
Complexity
Constraints
SEER-SEM
Estimation
Processing
Schedule
Risk
Maintenance
SEER-SEM Effort Estimation

Software Size


Lines, function points, objects, use cases
Technology and Environment Parameters






Personal capabilities and experience (7)
Development support environment (9)
Product development requirements (5)
Product reusability requirements (2)
Development environment complexity (4)
Target environment (7)
SEER-SEM Equations
E
C
te


0 . 393489
C
where:

K
K,
tb
ParmAdjust ment
C
tb

D
0 .4
(
Se
1 .2
)
,
C te

 ctbx  
  3 . 70945  ln 

 4 . 11  
 2000  exp 


5  TURN




E Development effort
K Total lifecycle effort including development and maintenance
Se Effective size
D Staffing complexity
Cte Effective technology
Ctb Basic technology
Agenda





Purpose
SEER-SEM
NF SEER-SEM
Evaluation
Conclusion
NFA
USA Patent No. US-7328202-B2
V1 , V 2 ,  , V M
RF1
Preprocessing
ARF1
NFB1
FM1
Neuro-Fuzzy
RF2
Inference
System
…
ARF2
NFB2
Algorithmic Model
(PNFIS)
…
RFN
FM2
ARFN
FMN
NFBN
where N is the number of contributing factors,
M is the number of other variables in the Algorithmic Model,
RF is Factor Rating,
ARF is Adjusted Factor Rating,
NFB is the Neuro-Fuzzy Bank,
FM is Numerical Factor/Multiplier for input to the Algorithmic Model,
V is input to the Algorithmic Model,
and Mo is Output Metric.
Output Metric
Mo
NFB
Layer1
Layer3
Layer2
w1

Ai1
Layer4
w1
Layer5
FMPi1
N
w1 FMP i1
ARFi
Ai2

N
FMi

…
…
AiN

…FMPi2
w N FMP iN
N
wN
wN
FMPiN
where
ARFi is Adjusted Factor Rating for contributing factor i,
Aik is fuzzy set for the k-th rating level of contributing factor i,
w k is firing strength of fuzzy rule k,
w k is normalized firing strength of fuzzy rule k,
FMP ik is parameter value for the k-th rating level of contributing factor i,
and FM i is numerical value for contributing factor i.
NF SEER-SEM
Size, SIBR
ACAP
AEXP
NF1
NF2
…
Complexity
(Staffing)
NFm
P1
SEER-SEM
Software
Estimation
Effort
Estimation
Algorithmic
Model
P2
P34
Effort
Estimation
E

0 . 393489

K, K

D
0 .4
(
Se
C te
1 .2
)
Agenda





Purpose
SEER-SEM
NF SEER-SEM
Evaluation
Conclusion
Performance Metrics
Relative Error (RE)
= (Est. Effort – Act. Effort) / Act. Effort
 Magnitude of Relative Error (MRE)
= |Est. Effort – Act. Effort | / Act. Effort
 Mean Magnitude of Relative Error (MMRE)
= (∑MRE) / n
 Prediction Level (PRED)
PRED(L) = k / n

Design of Evaluation
Case ID
Description
C1
No outliers
C2
Including all outliers
C3
Excluding part of outliers
C4-1
75% for Learning, 25% for testing
C4-2
50% for Learning, 50% for testing
MMRE Results
Case ID
MMRE (%)
SEER-SEM
Validation
Change
C1
84.39
61.05
-23.35
C2
84.39
59.11
-25.28
C3
84.39
59.07
-25.32
C4-1
50.49
39.51
-10.98
C4-2
42.05
29.01
-13.04
Negative value of MMRE change means improvement
MMRE Results
Summary of MMRE Validation
110.00%
MMRE and Change
90.00%
70.00%
SEER-SEM
50.00%
Validation
30.00%
Change
10.00%
-10.00%
-10.98%
-23.35%
-30.00%
C1
C2
-25.28%
C3
-25.32%
C4-1
-13.04%
C4-2
-19.59%
Average
PRED Results
Average of
SEER-SEM
Validation
Change
PRED(20%)
39.76%
27.48%
-12.28%
PRED(30%)
49.27%
36.46%
-12.81%
PRED(50%)
62.02%
55.35%
-6.67%
PRED(100%)
85.55%
97.69%
12.14%
Positive value of PRED change means improvement
Summary of Evaluation Results



MMRE is improved in all cases, with the
greatest improvement over 25%
Average PRED(100%) is increased by
12%
NF SEER-SEM improves MMRE by
reducing large MREs
Agenda





Purpose
SEER-SEM
NF SEER-SEM
Evaluation
Conclusion
Conclusion


NF with SEER-SEM improves
estimation accuracy
General soft computing framework
works with various effort estimation
algorithmic models
Future Directions


Evaluate with original SEER-SEM
dataset
Evaluate general soft computing
framework with:


more complex algorithmic models
other domains of estimation
THANKS !
Any Questions?