IA Issues? Online Card Sorting is Not Enough.Andrew Schall, Principal ResearcherKey Lime Interactive
Principal Researcher & Sr. Director, Key Lime InteractivePast President, UXPA Washington DCCo-author, Eye Tracking in User Experience Design
Founder & President, UXPA South Florida
Whats covered in the next 45 minutesWhy card sorting?Online card sorting: The good, bad, & uglyIn person card sorting: Going old schoolHybrid card sorting method: The best of both worldsQ & A
A super quick review of card sorting
Understanding how your users think about the organization of contentIf your users cant find the information that they are seeking, it might as well not be there at all.
The card sorting research methodCard sorting is a technique that can help gain insights into how your users think about the organization of your content. This user research method can be performed using an online tool or in person using physical cards.Need to learn more about card sorting? Read this book:
What we learn from card sortingWhich items participants think should be grouped togetherWhat to call each collection of grouped itemsHow many categories participants think the items should be grouped into
Online Card Sorting
Moving from physical to digital
Easy methods to recruit participants
Convenient method for capturing data
Automated data aggregation and visualization
The benefits of online card sortingOnline card sorting has become a very convenient and common way to collect this information from users.
Recruit 24/7 participants can complete the activity on their schedule from anywhere.Fast and easy data collection card sort software is relatively easy to use and straightforward for most participants.Large sample size it is easy to quickly obtain a large sample size for statistical analysis.
Who you think your participant is
Who your participant really is
Online card sorting tells you what, but not whyCollecting data on how your users sort items into categories tells you what they did, but not why they did it.
Quantitative results can be inconclusive
Online survey to the rescue!
Quotes from actual study participants
Its goodFineClearNo questionsConfusingDidnt understand some of the cardsI didnt know what was meant by _______I wasnt sure where to put the _____ cardActivity took WAY too long!Im not getting paid enough to do this.Less than helpful qualitative responses
In Person Card Sorting
Old school card sorting
In person card sorting methodTypically scheduled for an hour and are conducted one-on-one with a facilitator.Sessions are typically recorded with an overhead camera capturing the table with the cards.Participants are provided with physical materials that include:Cards labeled with the name of each item to be sortedPens or markers for making notesSometimes color-coded stickers for further annotations such as priority/importance
Physical cards are intuitive
Benefits of in person card sortingUnderstand participants thought process Think aloud protocol can be used to have the participant explain why they are sorting cards in a certain way.Observe nonverbal responses Do they appear deep in thought, confused or frustrated? Do they visibly hesitate to place a card or create a label for a category? Provide motivation Card sorting is a laborious and mentally demanding task. Participants often lose motivation to complete the activity. The facilitator can provide encouragement to the participant to keep going.
Lots of manual data entry
Hybrid Card Sorting
A hybrid approachStart with an online open card sort study to see general trends in how users sort items and for a higher sample sizeFollow up with an in-person study with a smaller sample size to understand participants thought process including insights into how they would use the content.Dont forget to iterate! Card sorting (or tree testing) should be done to validate any changes that you make to your IA.
Good research means asking the right questions
What does this mean to you?Asking about the meaning of cards and categories (if closed)Suggestions for improving labels
How is this relevant to you?Asking about the relative importance of content itemsContext of use In what situations would they need to access this information, and how often?
Why would you group these together?Inquire about thought process of how items are related to each otherSensing level of uncertainty, confusion or frustration
In summaryDont exclusively rely on online card sorting for your IA research needs!Use the power of think aloud to learn from your card sort participants.Consider a hybrid approach to gain the benefits of both online/unmoderated and in person/moderated card sorting.Dont forget to iterate. You should validate the changes that you make to an IA based on card sorting results.