Transcript No Slide Title

Metoder och tekniker för granskning av OOmodeller - del som berör litteraturstudier
och inhämtning av information inom
Ericsson
Sesam Seminarium 2000-10-19
Claes Norelöv
[email protected]
Ericsson Saab Avionics AB
Disposition
• Frågeställningar
• Klassificering av granskningstekniker
• Experiment 1: Perspective Based Reading vs
Checklist Based Reading (Laitenberger et al)
• ”Reading Techniques” för granskning av OOmodeller (Travassos et al)
• Experiment 2: Vertical & Horizontal Reading
(Travassos et al)
• ”Guided Inspection” (Melissa L. Major)
Ericsson Saab Avionics AB
Frågeställningar
• Vilka problem/brister finns idag genom att man
granskar dokument? Dessa är ju enbart en vy av
en OO-orienterad modell.
• Hur skall man rent arbetsmässigt gå tillväga när
man granskar modeller? Skiljer sig förfarandet
vid granskning under olika faser av utvecklingen,
vilka aspekter av modellen granskas?
• Vilka krav ställs på konfigurationsstyrning? Vad
utgör Configuration Items i modellen?
Ericsson Saab Avionics AB
Frågeställningar, forts
• Givet att man har tekniker för granskning av OOmodeller och arbetar direkt på modellunderlag i
stället för dokument, hur hanterar man då
kvalitetsaspekterna som man förmodligen åtagit sig
enligt någon standard
(ISO 9000-3, MIL-STD 498, ISO 12207, RTCA-DO
178B etc).
Ericsson Saab Avionics AB
Frågeställningar, forts
• Man har alltså att ta ställning till frågeställningar
kopplade till följande processteg i en en
granskningsprocess:
• Planering av granskning
• Genomförande av granskning (Lämpliga
tekniker att granska modeller)
• Återkoppling av granskningsresultat
Ericsson Saab Avionics AB
Informationsinhämtning #1
• Frågeställningarna har varit vidarebefordrade i två
nätverk
• Ericsson Review and Inspection Academy
• TTG Programvara inom Saab
Ericsson Saab Avionics AB
Informationsinhämtning #2
• Litteratursökning har gjorts på webben. och ett
antal träffar angående “reading techniques” har
studerats:
• University of Maryland Depatment of Computer
Science
(http://www.cs.umd.edu/projects/SoftEng/ESEG/)
• Fraunhofer Center for Experimental Software
Engineering, Maryland (http://fc-md.umd.edu/),
Kaiserslautern (http://www.iese.fhg.de)
Ericsson Saab Avionics AB
Classification of Inspection Techniques
Ericsson Saab Avionics AB
Experiment #1: How to conduct an Inspection?
Software
Product
1
organizer
Reading
Techniques
Planning
Form
Planning
2
inspector
Detection
Defect
Report
Form
3
moderator
inspectors
author
Collection
Defect
Collection
Form
Roles
4
Activities
Products
Software
Inspection
author
Correction
Defect
Correction
Form
Corrected
Software
Product
Ericsson Saab Avionics AB
Classification of Inspection Techniques #1
• Ad-hoc
• Inspectors intuition & experience
• Checklist-based reading (CBR)
• Shortcomings
• Based on past defect information (new defect types?)
• Too many questions & rare to find concrete instructions
(HOW?)
• Does not require the inspector to document his or hers
analysis
• Inspector often get swamped with unnecessary details
Ericsson Saab Avionics AB
Classification of Inspection Techniques #2
• Scenario-based reading
extend the work of Parnas and Weiss on Active
Design Reviews and allocate specific
responsibilities to inspectors and provide guidance
in the form of so-called scenarios (what to check &
how to perform the required check)
Ericsson Saab Avionics AB
Classification of Inspection Techniques #3
• Scenario-based reading
• Defect-based Reading
• Reading scenarios based on a defect taxonomy for
inspection of functional requirements documents
<Porter et al.>
• FP-based Reading
• A FP scenario consist of questions about a specific FPelement (input, output, file, inquiries)
Ericsson Saab Avionics AB
Classification of Inspection Techniques #4
• Scenario-based reading (Continued)
• Perspective-based Reading (PBR)
• Checking from a particular stakeholders perspective
(customer, requirements, design, test, maintenance)
• A PBR-scenario consists of activities an inspector is to
perform to extract information or build abstractions from
the inspected document/artifact and questions to
analyze the extracted information.
• Generalizing Perspective-based Inspection to handle
Object Oriented Development Artifacts <Laitenberger,
Atkinson>
Ericsson Saab Avionics AB
Classification of Inspection Techniques #5
• Scenario-based reading (Continued)
• Traceability-based Reading <Travassos et al>
• The scenarios describe how to perform correctness and
consistency checks among various UML-models.
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Experiment #1
(Laitenberger et al)
An Experimental Comparison of
Reading Techniques for Detection in
UML Design Documents
Ericsson Saab Avionics AB
Experiment#1: Software Inspection Process
• Planning
• Defect Detection
• Empirical findings reveal that the synergy effect of inspection
meetings is rather low in terms of impact of defects detected.
Therefore it can be considered as an individual rather than a
group activity.
• Defect Collection
• Often performed in a team meeting led by an inspection
moderator
• Defect Correction
• Performed by the author
Ericsson Saab Avionics AB
Experiment#1: Rationale for a new
Inspection Technique
• Increased Effectiveness
The proportion of defects found during an inspection
• Decreased Cost
Cost is defined as the effort involved in finding a
single defect
• In the experiment the cost has been separated for the two
phases
• individual defect detection
• defect collection in a meeting
Ericsson Saab Avionics AB
Experiment #1: Compare Reading Techniques
Planing
Overview
Follow-up
Detection
Collection
Correction
Process
• Organizer
• Moderator
• Inspector
• Author
• (Recorder)
• (Presenter)
Roles
Inspection
Products
• to be inspected
• to conduct an
inspection
Reading Techniques
 Ad-hoc
 Checklists
 RSA
 Defect-based Reading
 Perspective-based Reading
Ericsson Saab Avionics AB
Experiment#1: Experimental Hyphoteses
• The Effectiveness of PBR is larger than the
Effectiveness of CBR for Teams
• Per-defect Detection Cost for Teams is
Lower with PBR than with CBR
• Meeting Cost for Teams is Lower with PBR
than with CBR
• Overall Inspection Cost is Lower with PBR
than with CBR
Ericsson Saab Avionics AB
Experiment#1: Subjects
• Experiment performed in the context of a
course in object-oriented development
• 18 practitioners with various backgrounds
and experience
•
•
•
•
Already written UML Documents (median 3 on a 5p scale)
Developing Design Documents (some experience)
Programming (1-16 years, median 2 years)
Inspections (none had participated in an inspection)
Ericsson Saab Avionics AB
Experiment#1: Materials
• Checklist (CBR)
• Scenarios (PBR)
• Designer Scenario
• Tester Scenario
• Implementor Scenario
Ericsson Saab Avionics AB
Results of the Experiment
• Defect Detection
PBR exhibits a 41% effectiveness improvement over
CBR
• Cost per Defect (Detection Phase + Meeting Phase)
The average relative improvement of all inspection
teams shows that PBR exhibits 58% cost per defect
improvement over CBR.
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Reading Technique - Definition
• Reading Technique can be defined as a series of
steps for the individual analysis of a textual
software product to achieve the understanding
needed for a particular task.
Ericsson Saab Avionics AB
Reading Technique 1
• First, the series of steps gives the reader guidance
on how to achieve the goal for the technique. By
defining a concrete set of steps, we give all
readers a common process to work from which
we can later improve based on experience.
Ericsson Saab Avionics AB
Reading Technique 2
• Secondly, a Reading Technique is for individual
analysis, meaning that the aim of the technique is
to support the understanding process within an
individual reader.
Ericsson Saab Avionics AB
Reading Technique 3
• Finally, the techniques strive to give the reader
the understanding that they need for a particular
task, meaning that the reading techniques have a
particular goal and they try to produce a certain
level of understanding related to that goal.
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Object Oriented Reading Techniques
• Two main questions drive all of the Object
Oriented reading techniques.
• Do all of the design artifacts describe the same
system, i.e. are the individual artifacts consistent with
each other?
Horizontal Reading
• Do the design artifacts, as a whole, accurately describe
the same system that the requirements describe, i.e.
are the requirements and the design consistent?
Vertical Reading
Ericsson Saab Avionics AB
• One reading
technique
(Scenario) for each
pair of or group of
diagrams that
could usefully be
compared to each
other.
Ericsson Saab Avionics AB
Horisontal Reading Scenario Version #1
Ericsson Saab Avionics AB
Experiment #2
(Travassos et al)
Techniques for OO Inspection
Ericsson Saab Avionics AB
Experiment #2 - Goal
• To evaluate feasibility of applying reading
techniques
• To receive feedback to create a second
version of the techniques
Ericsson Saab Avionics AB
Experiment #2 - Subjects
• 44 undergraduates from a SW Engineering
Course with a mix of SW experience
• 32% had some previous industry experience in
SW design from requirements and use cases
• 9% had no experience in SW design
• 59% had classroom experience
• All students were trained in OO
development, UML and OO SW design
activities as part of the course
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Experiment #2 - Procedure and Result
• Subjects randomly assigned to 15 team
• One subject performed vertical reading
• the other two divided the horizontal techniques
between them
• Teams detected 11 defects on average
• 11,7 for horizontal (defects of ambiguities and
inconsistency between design artifacts)
• 10,4 for vertical (omitted and incorrect
functionality)
Ericsson Saab Avionics AB
Experiment #2 - Lessons Learned
• The first lesson learned was:
OO reading techniques should concentrate on semantic,
not syntactic, issues.
Ericsson Saab Avionics AB
Experiment #2 - Lessons Learned
• Second lesson learned was:
Reading techniques need to include not only instructions
for the reader, but some motivation as to why those
instructions are necessary. The information in the
qualitative data suggested that subjects were interested in
why the steps of the technique were important and useful,
as opposed to just being told what to do.
Ericsson Saab Avionics AB
Experiment #2 - Lessons Learned
• The third lesson that was learned was:
The level of granularity of the instructions needs to be precisely
described. Discussing functionality is a difficult but necessary part of
the reading techniques. The difficulty comes from the many different
levels of granularity at which system behavior can be described. To
solve this problem we decided to define 3 distinct ways of discussing
functionality, ranging from very specific system messages (the
communications between objects that work together to implement
system behavior) to system services (the steps by which some larger
goal or functionality is achieved) and finally, at the top level, system
functionalities (the behavior of the system, from the user’s point of
view).
Ericsson Saab Avionics AB
Horisontal Reading Scenario Version #2
Ericsson Saab Avionics AB
Experiment #2 - Observational Technique
• Employing techniques (observational and
inquisitive) to observe subjects while working
with the reading techniques may be a way to
capture more complete and accurate
information about what is really going on
(e.g. the subject’s thought process)
Ericsson Saab Avionics AB
Experiment #2 - Observational Technique
• Observations
• Having applied the techniques once, the time
spent to apply the techniques diminished
• The complexity of the artifact being inspected is
an ifmportant actor (noticed especially when reading
classes belonging to inheritance hierarchies)
• Some domain knowledge seemed to be necessary
to support horizontal reading (Readers suggested that
the entire set of requirements was not needed but some
description of the system, its main concepts and objectives)
Ericsson Saab Avionics AB
Experiment #2 - Observational Technique
• Observations
• Necessary level of object-oriented development
expertise.
When different levels of abstraction were used (e.g.,
references to objects versus classes), readers without a
more detailed knowledge of the OO paradigm encountered
difficulty. This does not imply that readers need experience
in OO development, just some knowledge of the OO
concepts such as class, object, attribute and behavior
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Guided by
Testcases
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB
Ericsson Saab Avionics AB