A Comparative Analysis of TestLink and Excel as Test Management

ToolsJeff Tonkovich

Today’s TopicA (brief) comparative analysis of Excel and

TestLink as Test Management Tools across five categories:SetupUsabilityReportingEfficiencyUnique Features

Research Methodology30 minute interviews with 5 engineers with

hands-on experience with both Excel and TestLink

Used standard questionnaire with each engineer plus an open-comments section

Obtained access to over 100 presentation slides for previous effort to transition test from Excel to a formal Test Management System (TMS)TestLink was the eventual winner

SetupExcel

Deployment time varies but is generally very low May be longer if deploying via SharePoint w/ user accounts

Manual population of test cases / requirementsTestLink

Web-based w/ central DBRequires a server with Apache, MySql, PHP & TestlinkDeployment time varies and increases with project size

Import test cases, setup user accounts, import requirements Approx. 2 hours for medium size project

UsabilityExcel

Used extensively in school, minimal learning-curve, extensive help and documentation available

Not ideal for multiple-user scenarios Merging worksheets, check-in/out via SharePoint

TestLinkPerfect for collaboration between multiple testersUser Management: able to assign roles users

E.G. Test Lead, Tester, Program Manager, etcPoor documentation, particularly for new features

Rule of thumb:30+ requirements: TestLink, otherwise Excel

ReportingExcel

Very simple to generate test reports with graphs Pass/fail statistics, weekly reports, etc

TestLinkNo major gain over Excel for simple tasksBuilt-in report generators

Auto-generated report showing requirements missing TCs

Customizable via PHPRVTM auto-generated with click of a button!

EfficiencyTestLink

Rapid Test Report generation “days” -> “hours”

Huge time savings in requirements maintenance 4x time savings vs Excel

Rapidly generate Test Plans (TP) TPs for different platforms, functionality (e.g. UI), etc

Test reporting less prone to human error Bugs in Excel Macros, inconsistent reporting, forgot

who tested what, etc

Unique FeaturesExcel

Simple to use but amazingly powerfulCustomizable via macros, VB script and add-ons

TestLinkEmail notifications

Notify test engineers which test cases / suites are assigned to them

Version control Test Cases and requirements

Metadata taggingBugzilla integration

SummaryAll interviewees unanimously agreed that TestLink

is superior to Excel for Test ManagementExceptions for very small projects and projects using

Excel that are almost overUse in CS577

Projects are small and overhead may not be worth it Mitigate by setting up / configuring prior to semester start

Auto-generate RVTM / Traceability Matrix (portions)Fully-functional Demo available on TestLink website

Thank You!