Transcript Test Driven Development

FIT for Embedded System Test Development
PRESENTER : Jingwen Chen
1
Main topics





Agile methodology from the desktop world
XP Inspired embedded system life-cycle
Adapting FIT into embedded system
development
– What is FIT/FitNesse
– EmbeddedFitNesse testing framework
– MatLab/C++ FitNesse testing framework
Issues with FIT in embedded design
Conclusion
2
What we want to avoid ?

System defects
– Software errors cost $59 billion / year
– Baxter Healthcare has to recall all its 256,000
Colleague Volumetric Infusion Pumps because
the hardware / software problems that already
cause 3 deaths and 6 serious injuries.

We want to find an appropriate and efficient
defect free development process for
embedded devices
3
Agile methodology from the
desktop business world


Minimize risk by developing software in
short iterations, ech iteration is like a
miniature software project of its own
Extreme Programming (XP)
http://www.extremeprogramming.org/map/
project.html
4
5
Agile methodology from the
desktop business world

Explore design space
– Test Driven Development (TDD)


Improved defect reduction
Enhanced communication between the
customers and developers
– Extremes of knowledge within a multi-disciplinary
team
– Customer: product end-users and researchers
– Developer: Engineers
6
XP inspired (XPI) product
lifecycle for biomedical
embedded system
Stage 1 : XPI Product envisagement
– Customers together with developers produce
“executable system specification”
– System acceptance tests
Stage 2 : XPI Proof of concept
– Algorithms developed in MATLAB and validated by
the acceptance tests written in first stage
– Additional unit test in MATLAB using MUnit
7
XP inspired (XPI) product
lifecycle for biomedical
embedded system
Stage 3 : XPI Prototyping of initial product
– Forming the initial product using host machine
simulation / embedded system
– System moved into the C/C++ production
language and validated via tests written in stage
1 and stage 2
– Additional embedded unit tests added for new
environment.
Stage 4 : XPI Full production of system
– The production of the full system on real
embedded system target board
– Satisfy the acceptance tests and unit tests
developed
8
Build an appropriate Agile
development environment for
embedded system


A mechanism to allow customers
without embedded system
knowledge to form a part of the
development team
FitNesse/FIT is one possible solution
9
What is FIT/FitNesse ?



A software testing tool designed for
customers with limited IT knowledge
A wiki-based system for creating, organizing,
and running FIT test tables
Fit test tables are easy to be read and
written by customer
10
An example of FIT test table
TestFixtures. CheckLifeSign()
items
inputValue
testResults?
bloodPressure
95
normal
heartRate
80
normal
temperature
38
high
expected
normal actual
11
Starting from a simple
temperature measurement
embedded system

Test software function:
CalculateTemperature ( ) running on the target
board convert pulse width to actual temperature
Voltage pulse:
Time High
Time Low
Input device:
TMP03 thermal sensor
Application Layer CalculateTemperature( )
System Software Layer
Hardware Layer
VDK real-time
operating system
Analog Device
ADSP-BF533 processor
Target Platform
12
Stage 1.
XPI Product envisagement

Customers together with the developers write
the Fit test table
DSPFixture.VDSPColumnFixtureTest
VoltHigh
VoltLow
outputTemperature(F)
10
100
195
20
100
155
30
100
115
13
Stage 2.
XP Inspired Proof of concepts


MatLabFitNesse testing framework
Developing the function in MatLab validated by
the fit test in stage 1
DSPFixture.VDSPColumnFixtureTest
VoltHigh
VoltLow
outputTemperature(F)
10
100
195
20
100
155
30
100
115 expected
FitNesse
Web server
Wiki editor
and storage
Runner
Fit
Server
Fixture for
Test table
MATLAB
API
135 actual
MATLAB
ENGINE
14
Stage 3.
XPI initial production system

Migrate the MatLab code into embedded system
language validated via tests for stage 1 and stage 2
using host machine simulation
HOST MACHINE
DSPFixture.VDSPColumnFixtureTest
VoltHigh
VoltLow
outputTemperature(F)
10
100
195
20
100
155
30
100
115 expected
135 actual
EmbeddedFitNesse
FitNesse
Web server
Wiki editor
and storage
C++
Simulation
Environment
Runner
Fit
Server
Fixture for
Test table
MATLAB
API
MATLAB
ENGINE
15
Stage 4.
XPI full production system

Running the function on real embedded target
platform and satisfying the original test in stage 1
HOST MACHINE
DSPFixture.VDSPColumnFixtureTest
VoltHigh
VoltLow
outputTemperature(F)
10
100
195
20
100
155
30
100
115 expected
135 actual
EmbeddedFitNesse
FitNesse
Web server
Wiki editor
and storage
C++
Simulation
Environment
Runner
Fit
Server
Fixture for
Test table
DSP Interface class
MATLAB
API
VDSP API
VDSP Environment
MATLAB
ENGINE
S.M.I.L.E
TARGET PLATFORM
16
Some tests we have done for
embedded hardware:
Lab2TestFixtures.SetCoreTimerAssembly
setCOUNT
setPERIOD
setSCALE
readTCOUNT()
readTPERIOD()
readTSCALE()
2000
2000
1
2000
2000
1
2000
3000
3
2000 expected
3000
3
1000
2
3000 actual
1000
1000


2
1000
An error on the ADSP-BF533 processor was found
by undergraduate students taking our embedded
systems course.
They were using TDD to develop the register
setting function SetCoreTimer ( ) running on the
embedded target board.
17
Issues with the full
EmbeddedFitNesse testing
framework
Long communication path !
HOST MACHINE
DSPFixture.VDSPColumnFixtureTest
VoltHigh
VoltLow
outputTemperature(F)
10
100
195
20
100
155
30
100
115 expected
135 actual
EmbeddedFitNesse
FitNesse
Web server
Wiki editor
and storage
C++
Simulation
Environment
Runner
Fit
Server
Fixture for
Test table
DSP Interface class
MATLAB
API
VDSP API
VDSP Environment
MATLAB
ENGINE
S.M.I.L.E
TARGET PLATFORM
18
FitNesse in embedded design
Is FitNesse a good solution for embedded
system Agile development ?

How to use FIT tables to specify an embedded
system properly ?
– Embedded system acceptance tests includes:




system operations
external and internal stimulus responses
interrupt latency
system reliability
– Extend the fit test table to handle diverse test cases.
 timing diagrams
 signal comparison figures

Efficiency and reliability concerns
– Make the EmbeddedFitNesse testing frame work
19
more efficiently
Conclusion




XPI Lifecycle embedded system
development
EmbeddedFitNesse testing framework
MatLab/C++ FitNesse testing framework
Issues with FIT in embedded design
20
Thank you !
21