Improving product usability with task complexity metricsBen ColbornTechnical Publications, Nutanix, Inc.

2

What Technical Writers Do

Bureau of Labor Statistics, U.S. Department of Labor, Occupational Outlook Handbook, 2014-15 Edition, Technical Writers, on the Internet at http://www.bls.gov/ooh/media-and-communication/technical-writers.htm (visited October 13, 2014).

3

Request from engineering

It would be nice for the doc team to have a metric for complexity of procedures. … That will bring objectivity to the whole topic and we can track progress as well.

— Binny Gill, Director of Engineering, April 2012

4

Outline

1.Foundations2.Mapping3.Outcome

Foundations

6

What makes a task difficult to complete?

• Materials• Time to complete• Context switching• Choices or paths through the task• Multipurpose tools or many single-purpose tools

7

Views of content

Content in a database Content as a database

“An object that can be retrieved from its indexed location, like locating a dining room chair in an Ikea warehouse.”

“A record that can be examined and presented from different angles based on different properties, which can be selected based on any of these properties, and which can be related to other records based on common properties.”

Mark Baker, “The difference between content in a database and content as a database”, spfe.info, Feb. 5, 2012

8

Good documentation practices

Minimalism Task orientationUsefulnessUsability

Structured authoring Semantic taggingModularity

Information architecture ConsistencyVersion control Release history

Topic ID management

9

Instructions for Computer vs. Person

A list of instructions for a computer to do

Program ProcedureA list of instructions for a person to do

Mapping

11

Software Metrics

Lines of code Statements in a program

Complexity Paths through the program

Function points Functionality provided to the user

Coupling How independent modules are

Cohesion Internal relationships in a module

12

Software to Documentation Metric Mapping

Software metric Documentation metric

Lines of code Steps/substeps

Complexity Choice points

Function points User-supplied parametersTyped text

Coupling BranchesGUI screens/menus

Cohesion Notes/Cautions/Warnings/Dangers

Human factors Interface switchesCommand-line interface commands

13

Metric to XPath Mapping (1)

Documentation metric XPath expression

Steps step[not(substeps)]substep

Choice points choiceschoicetablestep[contains(text(), "If")]

Branches step//xref

nCLI commands step//codeblock[starts-with(text(), "ncli")]

root commands step//codeblock[starts-with(text(), "#")]step//codeblock[contains(text(), "sudo")]

Non-root commands step//codeblock[starts-with(text(), "$")]

14

Metric to XPath Mapping (2)

Documentation metric XPath expression

GUI screens/menus uicontrolwintitle

Interface switches *[contains(text(), "vSphere client")]*[contains(text(), "ssh")]*[contains(text(), "web console")]

Notes/Cautions/Warnings/Dangers

note[not(@type)]note[@type="caution"]note[@type="warning"]note[@type="important"]note[@type="danger"]

User-supplied parameters varname

Typed text userinput

15

Implementation

Script (PowerShell, Python)

1. Declare Xpath expressions2. Open each file in the repository3. Count occurences of each expression and sum4. Write to log file

Open Toolkit Plugin See ditanauts QA plugin

16

Storing for comparison

Outcome

18

Complexity measures over time (1)

2 2.1 2.5.3 2.6.2 2.6.4 3 3.1 3.5 40

20

40

60

80

100

120

Configuring the Controller VM Cluster Software

Configuring the ESXi Host

Upgrading vSphere on a Host (NX-1000/3050/6000 with 16 GB DOM)

Configuring HA and DRS in vCenter

Configuring Host Networking (ESXi)

Adding a Nutanix Node to vCenter

Configuring SNMP

Installing the Controller VM (ESXi)

Migrating a VM to Another Cluster

Version

Com

plex

ity

19

Complexity measures over time (2)

2.5.3 2.6.2 2.6.4 3.0 (NX3000) 3.1 (NX1000_NX3050_

NX6000)

3.5 (NX1000_NX3050_NX6000_NX7000)

4.0 (NX1000_NX3050_

NX6000)

4.0 (NX1000_NX3050_NX6000_NX7000)

4.0.2 (NX1000_NX3050_NX6000_NX7000)

0

20

40

60

80

100

120

140

Drive Replacement (Complexity)

20

Quantifying improvement & prioritizing

While the engineers are focused on their individual sub-areas, the complexity of the overall workflows is not always clear to them. Using these metrics has converted a subjective and often contentious topic into an objective topic which now encourages more constructive discussions in my team. It also helps me demonstrate to upper management and stakeholders the improvements being made in … system design.

— Binny Gill, Director of Engineering, August 2012

21

Other applications

• Documentation complexity• Content inventory

22

Q & A

ditanauts.org