UXPA Boston Card Sorting

IA Issues? Online Card Sorting is Not Enough.

Andrew Schall, Principal ResearcherKey Lime Interactive

Principal Researcher & Sr. Director, Key Lime Interactive

Past President, UXPA Washington DC

Co-author, Eye Tracking in User Experience Design

Andrew Schall

Founder & President, UXPA South Florida

What’s covered in the next 45 minutes Why card sorting? Online card sorting: The good, bad, & ugly In person card sorting: Going old school Hybrid card sorting method: The best of both worlds Q & A

A super quick review of card sorting

Understanding how your users think about the organization of content

If your users can’t find the information that they are seeking, it might as well not be there at all.

The card sorting research methodCard sorting is a technique that can help gain insights into how your users think about the organization of your content. This user research method can be performed using an online tool or in person using physical cards.

Need to learn more about card sorting? Read this book:

What we learn from card sorting Which items participants think should be grouped together What to call each collection of grouped items How many categories participants think the items should be

grouped into

Online Card Sorting

Moving from physical to digital

Easy methods to recruit participants

Convenient method for capturing data

Automated data aggregation and visualization

The benefits of online card sorting

Online card sorting has become a very convenient and common way to collect this information from users.

Recruit 24/7 – participants can complete the activity on their schedule from anywhere.

Fast and easy data collection – card sort software is relatively easy to use and straightforward for most participants.

Large sample size – it is easy to quickly obtain a large sample size for statistical analysis.

Who you think your participant is

Who your participant really is

Online card sorting tells you what, but not whyCollecting data on how your users sort items into categories tells you what they did, but not why they did it.

Quantitative results can be inconclusive

Online survey to the rescue!

Quotes from actual study participants

“It’s good” “Fine” “Clear” ”No questions” “Confusing” “Didn’t understand some of the cards” “I didn’t know what was meant by _______” “I wasn’t sure where to put the _____ card” “Activity took WAY too long!” “I’m not getting paid enough to do this.”

Less than helpful qualitative responses

In Person Card Sorting

Old school card sorting

In person card sorting method

Typically scheduled for an hour and are conducted one-on-one with a facilitator.

Sessions are typically recorded with an overhead camera capturing the table with the cards.

Participants are provided with physical materials that include: Cards labeled with the name of each item to be sorted Pens or markers for making notes Sometimes color-coded stickers for further annotations such as

priority/importance

Physical cards are intuitive

More flexibility

Higher engagement

Benefits of in person card sorting

Understand participant’s thought process – Think aloud protocol can be used to have the participant explain why they are sorting cards in a certain way.

Observe nonverbal responses – Do they appear deep in thought, confused or frustrated? Do they visibly hesitate to place a card or create a label for a category?

Provide motivation – Card sorting is a laborious and mentally demanding task. Participants often lose motivation to complete the activity. The facilitator can provide encouragement to the participant to keep going.

Scheduling logistics

Lots of manual data entry

Hybrid Card Sorting

A hybrid approach

Start with an online open card sort study to see general trends in how users sort items and for a higher sample size

Follow up with an in-person study with a smaller sample size to understand participants’ thought process including insights into how they would use the content.

Don’t forget to iterate! Card sorting (or tree testing) should be done to validate any changes that you make to your IA.

Good research means asking the right questions

What does this mean to you?

Asking about the meaning of cards and categories (if closed) Suggestions for improving labels

How is this relevant to you?

Asking about the relative importance of content items Context of use – In what situations would they need to access this

information, and how often?

Why would you group these together?

Inquire about thought process of how items are related to each other Sensing level of uncertainty, confusion or frustration

In summary

Don’t exclusively rely on online card sorting for your IA research needs! Use the power of think aloud to learn from your card sort participants. Consider a hybrid approach to gain the benefits of both

online/unmoderated and in person/moderated card sorting. Don’t forget to iterate. You should validate the changes that you make

to an IA based on card sorting results.

Andrew Schallandrew@keylimeinteractive.comwww.keylimeinteractive.com