Transcript Automatically Extracting Configuration Constraints

Automatically Extracting
Configuration Constraints
Sarah Nadi *, Thorsten Berger *, Christian Kästner +,
and Krzysztof Czarnecki *
Product Line Engineering Workshop, Univ. of Waterloo
Dec. 9th 2013
*
+
Variability in real life
Command navigation package
• Integrated Garage Door Opener
• Electronic Compass
Premium Package
• Black Dacota Leather
• Automatic Trunk
Selection will result in:
Comfort Seats, Front
+ Addition of premium package
Black Leatherette
- Removal of Black Leathertte
Sarah Nadi
2
Handling variability
Build Independently
Clone & Own
Share Assets
Software Product Lines
(SPL)
Product configuration
Variability modeling
Components
DSLs
Generators
Preprocessors
Design patterns
…
[Dubinski et al., CSMR ‚13]
Sarah Nadi
Slide credits: T. Berger
3
But.. How can we build an SPL?
Build from scratch
P1
X
P2
P3
…
Migrate
Expensive and not always possible
Sarah Nadi
4
What is involved in migrating?
Identify Configurable Features
Detect Variation Points
Identify Feature Dependencies
Refactor Code
Create Variability Model
Refactor Architecture
[She et al., ICSE‘11]
Sarah Nadi
5
Variability model constraints
Hierarchy Constraint:
MP3 => Media
Cross-tree Constraint:
Camera => High Res.
[Benavides et al., 2010]
Sarah Nadi
6
Can we automatically
extract constraints?
To what extent?
Sarah Nadi
7
Scope
• C based systems using conditional compilation
• Focus on build-time variability
• Identify two sources of constraints
Sarah Nadi
8
1. Conditional build-time errors
Specification 1: Every valid
configuration of the system must
not contain build-time errors.
Sarah Nadi
9
Pre-processor Error if
ASH && NOMMU
Invalid Configuration
Constraint:
ASH => !NOMMU
Parser Error if
ASH && EDITING && !MAX_LEN
Type Error if
ASH && EDITING_VI && MAX_LEN &&
!EDITING
Sarah Nadi
10
2. Feature effect
• Avoid meaningless configurations which do not
add/remove parts of the code
• If we add/remove a feature, we want to get different
functionality
• Determine under which configurations, a feature has an
effect on the code
Specification 2: Every valid
configuration should yield a
lexically different program
Sarah Nadi
11
Feature effect
MAX_LEN && EDITING && ASH
MAX_LEN && EDITING_VI && ASH
MAX_LEN =>
ASH && (EDITING || EDITING_VI)
Sarah Nadi
12
Extract constraints by brute force
P1
P2
P3
…
Not scalable--- 2n combinations
Pn
Build
Individually
If every configuration with feature X
compiles except when Y is also selected
X => !Y
If every configuration with feature Z does
not change the selected code except if W
is also selected
Z => W
Sarah Nadi
13
Extracting constraints in a single pass
• Make use of variability-aware parsing & type checking to
mimic build-time behaviour.
Sarah Nadi
[Kästner et al., OOPSLA ‘11]
14
Infrastructure
• Developed tool
• FarCE – Feature Constraints Extraction
https://bitbucket.org/tberger/farce
• Rely on previous work:
• TypeChef – Type-Checking Ifdef Variability
https://github.com/ckaestne/TypeChef
• KBuildMiner
http://code.google.com/p/variability/wiki/PresenceConditionsExtraction
• To analyze variability models:
• LVAT https://code.google.com/p/linux-variability-analysis-tools
• CDLTools https://bitbucket.org/tberger/cdltools
Sarah Nadi
15
Empirical study
• Objectives:
• O1: Evaluate accuracy and scalability of extraction
• O2: Quantitatively and qualitatively study kinds of (extractable)
constraints in real-world systems
• Used four systems with existing variability models
uClibc
1,628 C files
367 features
BusyBox
535 C files
844 Features
eCos
579 C files
1,254 features
Linux Kernel
7,691 C files
6,559 Features
• Compare extracted constraints to existing hierarchy &
crosstree edges in the model
Sarah Nadi
16
Is the extraction accurate?
Specification 1 is 95% accurate
Specification 2 is 76% accurate
Sarah Nadi
17
Which constraints are recovered?
Hierarchy
edges
reflected in
code
nesting
Crosstree
edges
prevent buildtime errors
Can automatically recover 23% of variability model constraints!
5% by Specification 1
17% by Specification 2
Sarah Nadi
18
What about constraints not found?
• Qualitative analysis of 144 unrecovered constraints
Manual analysis of constraints is hard!
29% unknown
21% additional analyses 19% limitation in comparison
Some constraints are non-technical & need
expert knowledge
Qualitative analysis is subjective
19% configurator-related
9% domain knowledge
3% limitation in extraction
Sarah Nadi
19
Challenges
• Presence conditions and constraints explode
• Limit complexity of constraints used
• Use non-SAT based constraint combination techniques
• Different ways to compare constraints
• Our comparison is limited to binary constraints
• Other techniques which may be used?
• Understand the intent of different constraints without
interviewing developers
Sarah Nadi
20
Extracting configuration constraints from code
Automatically Done
Conditional Build-time errors & Feature Effect
Accurate
(95% & 76% respectively)
Extracts substantial parts of VM
An expert may still be needed
(Avg. 23% & up to 65%)
Questions?
Sarah Nadi
(finishing soon … interested in a post doc)
[email protected]
Sarah Nadi
http://swag.uwaterloo.ca/~snadi
21
Pre-processor Error if
ASH && NOMMU
Invalid Configuration
Constraint:
ASH => !NOMMU
Linker Error if
ASH && EDITING && !INIT
Parser Error if
ASH && EDITING && !MAX_LEN
Type Error if
ASH && EDITING_VI && MAX_LEN &&
!EDITING
Sarah Nadi
22
Constraint formulas
Preprocessor, parser, and type-checking
constraints
Linker constraints
conditional symbol table
Feature effect
Sarah Nadi
23
How are constraints used?
• Hierarchy edges are mainly reflected in how features are
used/nested in the code (Spec 2: feature effect analysis)
• Cross tree edges are often used to prevent build-time
errors (Spec 1: conditional build-time errors)
Sarah Nadi
24
Partial pre-processor (lexer)
slide credits: C. Kästner
Sarah Nadi
25
parser
slide credits: C. Kästner
Sarah Nadi
26
Is the analysis scalable?
Can analyze Linux files in 12hr with parallelization
Sarah Nadi
27