Using multiple methods to assess the usability of credit card disclosures

Angela Colteracolter@gmail.com

Using Multiple Methods | Angela Colter

What are disclosures?

card member agreements

solicitation letters

Why was the U.S. Congress interested?

1. Assess the need to revise disclosures so they better inform consumers of now-common penalty rates and fees

2. Find out how effectively pricing practices are disclosed to cardholders

3. Determine whether penalty charges contribute to cardholder bankruptcies

Using Multiple Methods | Angela Colter

Three evaluation methods

1. Readability analysis using formulas

2. Expert review using plain language guidelines

3. Usability test

Using Multiple Methods | Angela Colter

Why those methods?

Using Multiple Methods | Angela Colter

Strengths Weaknesses

Readability formulas

Quantitative data

Quick & cheap

Only counts what can be counted

Doesn’t tell you what to do about it

Plain language review

Predicts problems and gives guidelines for improvement

Also relatively cheap

Doesn’t mean users will have problems

Usability test Shows how people actually use an interface

Provides quantitative and qualitative data

Not quick and not cheap

Small sample size can be a hard sell

Readability analysis

• Used FOG, SMOG, and Flesch Grade Level

• Took multiple samples

• Carefully prepped the text (removed URLs, trailing periods and hard returns)

• Used Readability Analyzer software

• Explained strengths and weaknesses of readability formulas in the final report

Using Multiple Methods | Angela Colter

Readability analysis findings

Using Multiple Methods | Angela Colter

Flesch Grade Level

SMOG

FOG

Average grade level of cardmember agreements

12

Readability analysis findings

Using Multiple Methods | Angela Colter

Flesch Grade Level

SMOG

FOG

Average grade level of cardmember agreements

12

Average grade level of annual percentage rate info

17

Readability analysis findings

Using Multiple Methods | Angela Colter

Flesch Grade Level

SMOG

FOG

Average grade level of cardmember agreements

12

Median reading level of U.S. adults8

Average grade level of annual percentage rate info

17

Expert review

Used a plain language guidelines document developed by the U.S. Securities and Exchange Commission in 1998

Deals with organization, content and design

Using Multiple Methods | Angela Colter

http://www.sec.gov/news/extra/handbook.htm

Expert review findings

Using Multiple Methods | Angela Colter

The disclosures violated many of the guidelines. The report listed each guideline and gave examples where disclosures followed or violated them.

Expressing a complicated, multistep

process in paragraph form makes it

difficult to understand the relationships

between steps.

By using bullet points, it’s much easier

to see what the steps are and when they

are applied.

Guideline: Use bullet points

Expert review findings

Using Multiple Methods | Angela Colter

At times, gave suggestions for improvements

Original (101 words)

Rewrite (48 words)

If at any time during any rolling consecutive twelve billing cycle period you fail to make two Minimum Payments on a timely basis or exceed your Credit Limit twice we may elect to increase your Purchase, Cash Advance and/or Balance Transfer APRs to the Penalty APRs. All Penalty APRs will remain in effect until, in a subsequent rolling consecutive six billing cycle period, you do not exceed your Credit Limit at any time and you make all of your required Minimum Payments on a timely basis when, in your next billing cycle, all Penalty APRs will no longer apply.

If you pay late or go over your credit limit twice in a year, the interest rate you pay on most things goes up to the default rate. It will go back down when you pay on time and don’t go over your credit limit for six months.

Guidelines: Eliminate jargon

Use short words

Use short sentences

Usability test

Tested the material with 12 users

Gave them two sets of tasks using two different documents: one solicitation letter, one card member agreement

Topics covered:• Annual percentage rates

• Grace period

• Minimum payment

• Late fee

• Over limit fee

Using Multiple Methods | Angela Colter

• Cash advance fee/APR

• Balance transfer fee/APR

• Finance charges/balance computation method

• Changes to terms

• Payment allocation

Usability test plan

Can they use disclosures to locate information?

Do they interpret the info correctly once found?

Using Multiple Methods | Angela Colter

Question Correct answer

Usability test findings

Gave an overview citing task success rate

Followed with a full explanation of outcome, observations and participant quotes

Using Multiple Methods | Angela Colter

Task 8: Finance Charges / Balance Computation Method

Solicitation Letters: Only 1 of 12 located the information and seemed to know the significance of the term average daily balance. No one knew what two-cycle ADB meant.

Cardmember Agreements: All were able to locate the information, but only one was able to correctly interpret the information she read.

This was, by far, the task with the lowest success rate.

Only one out of the 12 participants located the name of the balance computation method used by the credit card in the solicitation letter. Two participants said things like “I know it’s based on the average daily balance, but that's not what it’s called.” Part of the problem may have been the label used on the Schumer box: Balance Calculation Method for Purchases didn’t seem to convey “this is how we calculate the interest you owe.”

Only one person gave a plausible explanation for the definition of what Average Daily Balance Computation Method means. No one who was asked what Two-Cycle Average Daily Balance Computation Method means even tried to guess.

Using Multiple Methods | Angela Colter

Usability test findings

Question P# Doc Time Correct? P’s Answer Analysis

This card uses the “two-cycle average daily balance” method to figure out how much interest you owe. What does it mean?

P6 SL #1 0:15 N -- You pay interest on this month’s and last month’s balance.

It means if you made a partial payment last month, you are paying interest on principle you have already paid off.

No one got it right.

P8 SL #1 0:50 N “I can’t understand it”

P7 SL #2 1:35 N “I really don’t know”

P9 SL #2 -- N “You got me”

P1 SL #3 1:10 N Described averaging

P3 SL #4 0:16 N “No clue”

Detailed information and performance metrics were included in the report’s appendix.

Complimentary approach

Using Multiple Methods | Angela Colter

Findings Weakness

Readability formulas

12th grade reading level Doesn’t tell you what to do (but P.L. guidelines do)

Plain language review

Docs don’t provide hierarchy, group related information, use markers (TOC, meaningful headings), or use simple words and sentences

Doesn’t mean users will have problems(but usability testing shows where they do)

Usability test Users had trouble locating info and often didn’t understand it once they had located it.

Disincentives for reading terms.

Small sample size can be a hard sell(but video puts a face on problems; findings from formulas and P.L. review found some of the same issues identified in testing)

Outcomes

Using Multiple Methods | Angela Colter

Communicating concepts wasn’t the main problem, unfair practices were.

The U.S. Congress and Federal Reserve passed laws and regulations that:

• ban universal default and “two-cycle” billing

• require high interest balances be paid first

• require opt-in to over limit fees

• disclose consequences of making only the minimum payment on monthly bills, not just in disclosures.

Further reading

Using Multiple Methods | Angela Colter

GAO Report: CREDIT CARDS Increased Complexity in Rates and Fees Heightens Need for More Effective Disclosures to Consumers

http://www.gao.gov/new.items/d06929.pdf

A Plain English Handbook: How to create clear SEC disclosure documents

http://www.sec.gov/news/extra/handbook.htm

Questions?

Angela Colter

Email: acolter@gmail.comTwitter: @angelacolter