Transcript Research Software Doesn't Have to be Buggy: A Model-Driven, Test-Driven and Agile

Research Software Doesn't Have to
be Buggy:
A Model-Driven, Test-Driven and Agile
Process Applied in a University
Research Team
November 2013
Timothy C. Lethbridge
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 1
Motivations for This Talk
I was explaining to researchers at a conference how I
have been able to maintain Umple
– They were interested in the process I followed
It has been a very positive experience to experience
‘real’ software development
– And not get bogged down it
– While maintaining quality
– And getting good feedback from students
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 2
Research Software: Often Short Life or
‘Not Production Quality’
Software developed by computer science researchers
often only lasts as long as one student’s PhD
–Perhaps a little longer
It usually gets messy
–The next generation of students often wants to rewrite it
The principal investigator wants to move on to other
things
–The software becomes a drag
–Lack of resources for maintenance
–Grants are for research, not development
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 3
Why would we want research software
to last for the long haul?
To really ‘get it right’
To act as infrastructure for countless experiments and
idea exploration
To allow the professor and students to get really good
at real software engineering
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 4
My Vision
Prodution quality software that persists for 20 or more
years
Continual improvement of the process
Top-notch training, education, and research platform
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 5
My Approach
Adopt software engineering best practices in the
software used in the research
Blend and mode-switch research approaches
– Design based research
– Action research
– Grounded theory
– Empirical experimentation
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 6
The Platform: Umple
Model-oriented compiler
– Adds UML and patterns to Java, C++ etc.
– Textual UML
– High quality code generator for class and state diagrams
– Web-based + command line + Eclipse
Main site: http://www.umple.org
Web app: http://try.umple.org
Manual: http://manual.umple.org
Open source:
– Google: http://code.umple.org
– Mirrors on Github and SourceForge
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 7
Robustness maintained because of /
despite contributions by >30 HQP
2 completed PhDs (+ 7 underway)
4 completed Masters (+ 1 underway)
22 undergraduate capstone projects (+ 6 underway)
3 postdocs
UCOSP: Students from UBC, Simon Fraser, UAlberta,
USask, URegina, Laurentian, Waterloo, Wilfred Laurier,
Guelph, Bishops, Sherbrooke, UNB, Dalhousie
International contributions from Brazil.
Countless students taught in classrooms
–Umple helps students learn UML (CSEET 2011)
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 8
Essential elements of the process
Red ones are key
Open source
Multi-level test driven development
Model driven development
Carefully categorized issues list
Continuous integration
Lean live documentation for users and developers
Low management overhead
Marketing and publication
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 9
Open Source
Open to the world as early as possible
If there are spinoffs, let them be based on service, not
software
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 10
Multi-level test driven development
Use test cases as specifications
Multiple levels in the layered architecture
– For Umple: Parsing, metamodel instance, generation,
execution
In Umple, the compiler itself is a big test
Note: We have not applied this yet to the user interface
http://qa.umple.org
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 11
Model driven development
Always start with the model and generate code from it
– It is possible
– Requires indoctrination, and students sometimes have
to be asked to start again!
Simplifies code, reduces volume, enforces abstraction
That is what Umple is about, but we have also
developed other systems in Umple
– Key Umple benefits compared to other MDD tools
• Model is code; traditional code embedded in mode
• No need to edit generated code
• Bidirectional traceability
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 12
Carefully managed issues list
Very-easy, easy, moderate, hard, very-hard
Defect, enhancement, undergraduate project, graduate
project
Priority: Critical, Vhigh, High, Med, Low, Vlow
Ownership of issue, but not code
http://bugs.umple.org
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 13
Continuous integration
Single master trunk for all work
Server builds then posts results
– http://cc.umple.org
Test result browser
– http://qa.umple.org
– 100% test pass rate maintained 95% of the time
– 100% test pass rate within an hour of a failure 99.9%
of the time
– Only two ‘backouts’ in the last year (failed tests that
could not be quickly fixed)
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 14
Lean Live Documentation for
Developers and End Users
Code commenting is key:
– But comments transfer into documentation
– Auto-generated diagrams are a result of MDD
– http://metamodel.umple.org
– http://grammar.umple.org
User manual is generated: http://manual.umple.org
If someone doesn’t understand something they update
the wiki
– http://wiki.umple.org
– Exploration of requirements done in issues and wikis
– Developers log experiences and progress for all to see
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 15
Low Management Overhead
As a professor, no time for much project management
– About 2 hours a week direct interaction with students
actively working on the software
– I manage issues, set up policies, refine documents
– Periodically I work on an issue
Code sprint (20 hours) once a semester
Code review of patches and commits
– Focus on checking the tests
– Delegation of inspection to others
– Some refactoring
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 16
Marketing and Publication
Not just journals and conferences
We need users and feedback from users of our tools
Live mirrors: Google Code, Github,
Facebook, Google+, Twitter, Email lists, blogs
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 17
Use of our software in teaching
Several professors have taught software engineering
concepts using our work
– Introduction to software engineering
– Advanced software design
– Software engineering graduate course
– Score project at ICSE
– Others
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 18
Introducing students to Umple
1. Train in the use of the software
2. Development environment setup
3. Coaching on process
4. Code sprint
5. Have them create several patches for easy to
moderate problems in several architecture areas
6. Make them a committer (happens for most students)
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 19
Conclusions
Make your academic software high quality by using
–TDD
–MDD
–Continual intregration
–Careful agile process
It works!
Benefits
–You and your students get better at software
engineering
–You can do better research because your infrastructure
is better
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 20
Questions?
This talk is available at:
http://bit.ly/1fMkhzZ
Or
http://www.eecs.uottawa.ca/~tcl/presentations/CSERNov2013-Talk-RshSWNotBuggy.ppt
Quality Process for Research Software - Timothy C. Lethbridge
Nov 2013- 21