Transcript Slide 1

Functional Dependence and
Equivalence Class Factors in
Combinatorial Test Designs
George B. Sherwood
Testcover.com
1
Copyright © 2014 Testcover.com, LLC. All rights reserved.
This talk is about:
• Evaluation of test design coverage before test
case generation
–
–
–
–
Partition includes test case generation combinations
Equivalence class includes expected result combinations
Typically a partition spans multiple equivalence classes
Formulas for equivalence class coverage
• Design choices to improve coverage
– Use increased generation strength t
– Align partitions with equivalence classes as needed
Testcover.com
2
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Body mass index report requirements
R1. Input data for patient database table:
Age in years
Weight in pounds
Height in inches
Sex (female, male)
Intake in kilocalories per day
R2. Compute & store body mass index:
BMI = 703 x Weight / Height2
R3. Age ≥ 65: Generate Medicare report
Testcover.com
3
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Body mass index report requirements
continued
R4. Age < 20 Generate Child report:
Girl, percentile from female BMI-age table
Boy, percentile from male BMI-age table
R5. Age ≥ 20 Generate Adult report:
Underweight, BMI < 18.5
Normal, 18.5 ≤ BMI < 25.0
Overweight, 25.0 ≤ BMI < 30.0
Obese, 30.0 ≤ BMI
Testcover.com
4
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Functional dependence
Mapping from a determinant set to one element in a dependent set
Examples:
y = sin(x)
US postal address index → ZIP code
l-tuple → m-tuple
BMI = 703 x Weight / Height2
determinant factor values → functionally dependent factor values
Testcover.com
5
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Functionally dependent test factors
• A test factor is functionally dependent when
its value is identified by l determinant factors
Example:
BMI is determined by l = 2 factors, Weight & Height
Weight, Height → BMI
• Independent factors which are not part of this
relation are nondeterminant
Example:
n = 3 nondeterminant factors, Age, Sex & Intake
Testcover.com
6
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Equivalence classes
• Equivalence classes group test factor
combinations by similar expected results
Example:
Age ≤ 0 leads to invalid, error handling class
• Classes help insure test design coverage
Example:
The Medicare, Child and Adult reports each have
multiple, valid equivalence classes
Report
Valid equivalence classes
Medicare
no
yes
Child
no
girl
boy
Adult
no
underweight
normal
Testcover.com
overweight
obese
7
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Equivalence classes are functionally dependent
Input, configuration values → result
→ equivalence class
Equivalence class factor examples:
l n
Age → Medicare classes 1 4
Age, Sex → Child classes
Age, Weight, Height → Adult classes
2 3
3 2
Equivalence class factor values
Age
19
67
Factors for test case generation
Independent input factors
Weight Height
Sex
Intake
2000
female
131
64
3000
male
131
64
Testcover.com
Factors determined from test cases
Functionally dependent factors
Medicare
Child
Adult
no
girl
no
normal
yes
no
8
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Determinant coverage
Combinations to reach equivalence classes – Medicare
Input factors
Determinant
Age
42
67
Equivalence class factors
Nondeterminant
Weight
Height
Sex
Intake
Functionally
dependent
Medicare
no
yes
Child
Adult
Combinations to reach equivalence classes – Child
Input factors
Determinant
Age
19
19
42 or 67
Nondeterminant
Weight
Equivalence class factors
Determinant
Height
Sex
female
male
Nondeterminant
Intake
Medicare
Functionally
dependent
Child
girl
boy
no
Adult
Combinations to reach equivalence classes – Adult
Input factors
Determinant
Age
19
42 or 67
42 or 67
42 or 67
42 or 67
Testcover.com
Equivalence class factors
Nondeterminant
Weight
Height
131
131
180
180
64
71
64
71
Sex
Intake
Medicare
Child
Functionally
dependent
Adult
no
normal
underweight
obese
overweight
9
Copyright © 2014 Testcover.com, LLC. All rights reserved.
180 lb
131 lb
Adult report classes
71 in
64 in
Testcover.com
10
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Nondeterminant coverage
• Nondeterminant factors provide coverage unrelated to the
functional dependence
• A nondeterminant subarray J consists of a functionally
dependent factor and its n nondeterminant factors.
Example:
Nondeterminant
subarray
J –j Child
Functionally
dependent
factor
– Child
Input factors
Determinant
Age
Nondeterminant
Weight
Height
Equivalence class factors
Determinant
Sex
Nondeterminant
Intake
Medicare
Functionally
dependent
Child
Adult
• J has nondeterminant strength s when every subarray of s
factors including the functionally dependent factor has
every s-tuple in at least one test case.
Example: When s = 3 each pair of nondeterminant factor
values is associated with each Child equivalence class.
Testcover.com
11
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Nondeterminant strength formula
s=t-l+1
Test cases from one strength-4 partition for multiple equivalence classes – Child
Input factors
Determinant
Test Case
1
2
3
4
5
6
7
8
9
…
Age
19
19
19
19
19
19
19
19
42
Nondeterminant
Weight
131
131
131
131
180
180
180
180
131
Height
64
64
71
71
64
64
71
71
64
Determinant
Sex
female
male
female
male
female
male
female
male
female
Nondeterminant
Intake
2000
3000
3000
2000
3000
2000
2000
3000
3000
Equivalence class factors
An expected result
Functionally
dependent
Child
girl
boy
girl
boy
girl
boy
girl
boy
no
BMI
22.5
22.5
18.3
18.3
30.9
30.9
25.1
25.1
22.5
Medicare
no
no
no
no
no
no
no
no
no
Adult
no
no
no
no
no
no
no
no
normal
Age, Sex → Child classes
s = t - l + 1 = 4 - 2 + 1 = 3.
All pairs of nondeterminant factor values are associated with each Child class.
s ≤ n + 1 because there are n + 1 columns in the subarray.
Testcover.com
12
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Coverage formulas
Partition contains
Strength to cover
equivalence class(es)
Nondeterminant strength
multiple values for an
equivalence class factor
t≥l
s = t – l + 1, t ≤ l + n
s = n + 1,
t≥l+n
one value for an equivalence
class factor
any
s = t + 1,
s = n + 1,
t≤n
t≥n
Multiple equivalence classes – Coverage depends on l
Example: t ≥ 3 to reach all classes; choose t = 3
Medicare classes: l = 1; s = 3
Child classes:
l = 2; s = 2
Adult classes:
l = 3; s = 1 (coverage problem)
One equivalence class – Coverage is independent of l
Example: any strength to reach one class; choose t = 2
s = 3 for each class in its own partition
Testcover.com
13
Copyright © 2014 Testcover.com, LLC. All rights reserved.
One strength-3 partition for multiple equivalence classes – Medicare
Input factors
Determinant
Test Case
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Age
19
19
19
19
19
19
42
42
42
42
42
42
67
67
67
67
67
67
Equivalence class factors
Nondeterminant
Weight
131
131
131
180
180
180
131
131
131
180
180
180
131
131
131
180
180
180
Height
64
64
71
64
71
71
64
64
71
64
71
71
64
64
71
64
71
71
Sex
female
male
male
female
female
male
female
male
female
male
female
male
female
male
female
male
female
male
Intake
2000
3000
2000
3000
2000
3000
2000
3000
3000
2000
2000
3000
2000
3000
3000
2000
2000
3000
Functionally
dependent
Medicare
no
no
no
no
no
no
no
no
no
no
no
no
yes
yes
yes
yes
yes
yes
Child
girl
boy
boy
girl
girl
boy
no
no
no
no
no
no
no
no
no
no
no
no
Adult
no
no
no
no
no
no
normal
normal
underweight
obese
overweight
overweight
normal
normal
underweight
obese
overweight
overweight
An expected result
BMI
22.5
22.5
18.3
30.9
25.1
25.1
22.5
22.5
18.3
30.9
25.1
25.1
22.5
22.5
18.3
30.9
25.1
25.1
Age → Medicare classes
s = t - l + 1 = 3 - 1 + 1 = 3.
All pairs of nondeterminant factor values are associated with each Medicare class.
Testcover.com
14
Copyright © 2014 Testcover.com, LLC. All rights reserved.
One strength-3 partition for multiple equivalence classes – Child
Input factors
Determinant
Test Case
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Age
19
19
19
19
19
19
42
42
42
42
42
42
67
67
67
67
67
67
Nondeterminant
Weight
131
131
131
180
180
180
131
131
131
180
180
180
131
131
131
180
180
180
Height
64
64
71
64
71
71
64
64
71
64
71
71
64
64
71
64
71
71
Determinant
Sex
female
male
male
female
female
male
female
male
female
male
female
male
female
male
female
male
female
male
Nondeterminant
Intake
2000
3000
2000
3000
2000
3000
2000
3000
3000
2000
2000
3000
2000
3000
3000
2000
2000
3000
Equivalence class factors
An expected result
Functionally
dependent
Child
girl
boy
boy
girl
girl
boy
no
no
no
no
no
no
no
no
no
no
no
no
BMI
22.5
22.5
18.3
30.9
25.1
25.1
22.5
22.5
18.3
30.9
25.1
25.1
22.5
22.5
18.3
30.9
25.1
25.1
Medicare
no
no
no
no
no
no
no
no
no
no
no
no
yes
yes
yes
yes
yes
yes
Adult
no
no
no
no
no
no
normal
normal
underweight
obese
overweight
overweight
normal
normal
underweight
obese
overweight
overweight
Age, Sex → Child classes
s = t - l + 1 = 3 - 2 + 1 = 2.
All nondeterminant factor values are associated with each Child class.
Testcover.com
15
Copyright © 2014 Testcover.com, LLC. All rights reserved.
One strength-3 partition for multiple equivalence classes – Adult
Input factors
Determinant
Test Case
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Age
19
19
19
19
19
19
42
42
42
42
42
42
67
67
67
67
67
67
Weight
131
131
131
180
180
180
131
131
131
180
180
180
131
131
131
180
180
180
Equivalence class factors
Nondeterminant
Height
64
64
71
64
71
71
64
64
71
64
71
71
64
64
71
64
71
71
Sex
female
male
male
female
female
male
female
male
female
male
female
male
female
male
female
male
female
male
Intake
2000
3000
2000
3000
2000
3000
2000
3000
3000
2000
2000
3000
2000
3000
3000
2000
2000
3000
Medicare
no
no
no
no
no
no
no
no
no
no
no
no
yes
yes
yes
yes
yes
yes
Child
girl
boy
boy
girl
girl
boy
no
no
no
no
no
no
no
no
no
no
no
no
Functionally
dependent
Adult
no
no
no
no
no
no
normal
normal
underweight
obese
overweight
overweight
normal
normal
underweight
obese
overweight
overweight
An expected result
BMI
22.5
22.5
18.3
30.9
25.1
25.1
22.5
22.5
18.3
30.9
25.1
25.1
22.5
22.5
18.3
30.9
25.1
25.1
Age, Weight, Height → Adult classes
s = t - l + 1 = 3 - 3 + 1 = 1.
Some nondeterminant factor values might not be associated with each Adult class.
Neither male nor 2000 is associated with the underweight class.
Testcover.com
16
Copyright © 2014 Testcover.com, LLC. All rights reserved.
One strength-4 partition for multiple equivalence classes – Adult
Input factors
Equivalence class factors
Test Case
Age
Weight
Height
Sex
Intake
Medicare
Child
Functionally
dependent
Adult
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
19
19
19
19
19
19
19
19
42
42
42
42
42
42
42
42
67
67
67
67
67
67
67
67
131
131
131
131
180
180
180
180
131
131
131
131
180
180
180
180
131
131
131
131
180
180
180
180
64
64
71
71
64
64
71
71
64
64
71
71
64
64
71
71
64
64
71
71
64
64
71
71
female
male
female
male
female
male
female
male
female
male
female
male
female
male
female
male
female
male
female
male
female
male
female
male
2000
3000
3000
2000
3000
2000
2000
3000
3000
2000
2000
3000
2000
3000
3000
2000
2000
3000
3000
2000
3000
2000
2000
3000
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
yes
yes
yes
yes
yes
yes
yes
yes
girl
boy
girl
boy
girl
boy
girl
boy
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
no
normal
normal
underweight
underweight
obese
obese
overweight
overweight
normal
normal
underweight
underweight
obese
obese
overweight
overweight
Determinant
Nondeterminant
An expected result
BMI
22.5
22.5
18.3
18.3
30.9
30.9
25.1
25.1
22.5
22.5
18.3
18.3
30.9
30.9
25.1
25.1
22.5
22.5
18.3
18.3
30.9
30.9
25.1
25.1
Age, Weight, Height → Adult classes
s = t - l + 1 = 4 - 3 + 1 = 2.
All nondeterminant factor values are associated with each Adult class.
Testcover.com
17
Copyright © 2014 Testcover.com, LLC. All rights reserved.
One
equivalence
class –classes
Adult underweight
Onestrength-2
strength-2partition
partitionfor
forone
multiple
equivalence
– Medicare
Input factors
Determinant
Determinant
Test Case
uw1
uw2
uw3
uw4
uw5
uw6
Age
65
65
64
20
64
20
Weight
118
128
128
118
118
128
Equivalence class factors
Nondeterminant
Nondeterminant
Height
72
70
72
70
70
72
Sex
male
female
male
male
female
female
Intake
3000
2000
2000
2000
3000
3000
Functionally
dependent
Medicare
yes
yes
no
no
no
no
Child
no
no
no
no
no
no
Functionally
dependent
Adult
underweight
underweight
underweight
underweight
underweight
underweight
An expected result
BMI
16.0
18.4
17.4
16.9
16.9
17.4
One partition for each Adult underweight class:
Age, Weight, Height → Adult classes
s = t + 1 = 2 + 1 = 3.
All pairs of nondeterminant factor values are associated with each Adult class.
One partition for both Medicare classes:
Age → Medicare classes
s = t - l + 1 = 2 - 1 + 1 = 2.
All nondeterminant factor values are associated with each Medicare class.
Boundary cases:
Medicare: lower Age
Adult: lower Age
Adult, underweight: upper BMI
Testcover.com
18
Copyright © 2014 Testcover.com, LLC. All rights reserved.
154 lb
174 lb
64 in
175 lb
155 lb
129 lb
66 in
Testcover.com
211 lb
70 in
118 lb
72 in
128 lb
Adult report classes and boundaries
61 in
19
Copyright © 2014 Testcover.com, LLC. All rights reserved.
Testers have choices
Equivalence class factor values in a partition: multiple
1
Number of partitions:
Test target:
Dependence on oracle:
Freedom to test equivalence class boundaries:
1
large
some
limited
multiple
smaller
more
yes
Strength to reach equivalence class:
l
t–l+1
any
Nondeterminant strength (s ≤ n + 1):
Testcover.com
t+1
20
Copyright © 2014 Testcover.com, LLC. All rights reserved.