Transcript TestLink Research Presentation

Jeff Tonkovich
Today’s Topic
 A (brief) comparative analysis of Excel and TestLink as
Test Management Tools across five categories:
 Setup
 Usability
 Reporting
 Efficiency
 Unique Features
Research Methodology
 30 minute interviews with 5 engineers with hands-on
experience with both Excel and TestLink
 Used standard questionnaire with each engineer plus
an open-comments section
 Obtained access to over 100 presentation slides for
previous effort to transition test from Excel to a formal
Test Management System (TMS)
 TestLink was the eventual winner
Setup
 Excel
 Deployment time varies but is generally very low

May be longer if deploying via SharePoint w/ user accounts
 Manual population of test cases / requirements
 TestLink
 Web-based w/ central DB
 Requires a server with Apache, MySql, PHP & Testlink
 Deployment time varies and increases with project size


Import test cases, setup user accounts, import requirements
Approx. 2 hours for medium size project
Usability
 Excel
 Used extensively in school, minimal learning-curve, extensive
help and documentation available
 Not ideal for multiple-user scenarios

Merging worksheets, check-in/out via SharePoint
 TestLink
 Perfect for collaboration between multiple testers
 User Management: able to assign roles users

E.G. Test Lead, Tester, Program Manager, etc
 Poor documentation, particularly for new features
 Rule of thumb:
 30+ requirements: TestLink, otherwise Excel
Reporting
 Excel
 Very simple to generate test reports with graphs

Pass/fail statistics, weekly reports, etc
 TestLink
 No major gain over Excel for simple tasks
 Built-in report generators


Auto-generated report showing requirements missing TCs
Customizable via PHP
 RVTM auto-generated with click of a button!
Efficiency
 TestLink
 Rapid Test Report generation

“days” -> “hours”
 Huge time savings in requirements maintenance

4x time savings vs Excel
 Rapidly generate Test Plans (TP)

TPs for different platforms, functionality (e.g. UI), etc
 Test reporting less prone to human error

Bugs in Excel Macros, inconsistent reporting, forgot who
tested what, etc
Unique Features
 Excel
 Simple to use but amazingly powerful
 Customizable via macros, VB script and add-ons
 TestLink
 Email notifications

Notify test engineers which test cases / suites are assigned to
them
 Version control
 Test Cases and requirements
 Metadata tagging
 Bugzilla integration
Summary
 All interviewees unanimously agreed that TestLink is
superior to Excel for Test Management
 Exceptions for very small projects and projects using
Excel that are almost over
 Use in CS577
 Projects are small and overhead may not be worth it

Mitigate by setting up / configuring prior to semester start
 Auto-generate RVTM / Traceability Matrix (portions)
 Fully-functional Demo available on TestLink website