1. Webinar: Mobile Banking Benchmark Ann Rochanayon, Sr. Director CX/ UX Research at UserZoom Sneha Kanneganti, UX Researcher at UserZoom Alfonso de la Nuez, Co-Founder and Co-CEO at UserZoom #uzwebinar

2. Ann Rochanayon Sr. Director CX/UX Research UserZoom Alfonso de la Nuez Co-Founder and Co-CEO UserZoom Speakers: Sneha Kanneganti UX Researcher UserZoom 3. www.userzoom.com Chat box is available if you have any questions There will be time for Q&A at the end We will be recording the webinar for future viewing All attendees will receive a copy of the slides/recording Twitter hashtag: #uzwebinar Quick Housekeeping 4. www.userzoom.com About UserZoom / Mobile Research About UX Benchmarking Mobile Banking Competitive Benchmark Study Methodology Overall Findings Detailed Task Findings Tips on Mobile Research Q&A Agenda 5. www.userzoom.com About UserZoom UX Consultants since 01, SaaS since 09 In US, UK, Germany and Spain UserZoom offers an all-in-one, multi channel and agile approach to managing digital Customer Experiences by enabling brands to cost- effectively plan, research, design, and measure CX and UX on a unified software platform. 6. Platform Overview www.userzoom.com Mobile Video Recording Remote Usability Testing VOC Solutions UX & Usability Solutions Results Select Product Features Intercept Surveys Feedback Tabs Mobile App VOC Remote Usability Testing Mobile Usability Testing Card Sorting Tree TestingScreenshot Click Testing Screenshot Timeout Testing Online Surveys Usability Data Survey Responses IA Data Behavioral Data Video Recording 7. www.userzoom.com Highlighted Customers 8. www.userzoom.com What is UX Benchmarking The process of comparing a websites (or any digital product) performance, specifically in relation to UX metrics, including effectiveness, efficiency, overall satisfaction and quality perception, to another that is widely considered to be an industry standard benchmark or best practice UX Benchmarking provides a snapshot of the performance of your product and helps you understand where you are in relation to a particular standard. 9. www.userzoom.com Why is UX Benchmarking important Measuring the quality of UX is more important than ever Consumers have gained a lot of power in recent years More competition, more maturity in the online marketplace You cant really manage what you cant measure. UX Pros must deliver more measured value. Mgmt Teams LOVE metrics UX Benchmarking allows you to MEASURE performance in two ways: 1. Baseline studies: Measure UX over time, compare evolution of your own product and/or service 2. Against competitors within your industry 10. www.userzoom.com 4 Must-dos with UX Benchmarking 1. Plan ahead: Define measurable & comparable UX-related KPIs (will show them in the case study ahead) Select competitors to benchmark against Define target audience Identify common site activities/tasks (recommended between-subjects design) 2. Conduct research using task-based testing 3. Gather both quantitative & qualitative data Use the quantitative data to test for statistically significant differences between means or proportions Use qualitative data to support your findings and understand the why 4. Research at least twice a year (or even quarterly) to measure design changes and trends in expectations 11. www.userzoom.com Agenda Mobile Banking Competitive Benchmark 12. www.userzoom.com Methodology 13. www.userzoom.com Goals Determine which bank offers the best mobile experience for prospective customers Assess the impact of mobile banking experience on brand perception, Net Promoter Score, and conversion Sites evaluated Participants Tested from February 24 March 5, 2014 Between-subjects design N=200, 50 participants per site Ages 26+ Not a current customer of the site evaluated Have prior experience with mobile banking Participated via. SmartPhone Methodology: Study Background 14. www.userzoom.com Methodology: Recruitment SSI is the premier global provider of sampling, data collection and data analytic solutions for survey research, reaching respondents in 86 countries via Internet, telephone, mobile/wireless and mixed-access offerings. SSI staff operates from 25 offices in 18 countries, offering sample across every mode, online and CATI data collection, questionnaire design consultation, programming and hosting, online custom reporting and data processing. SSIs 3,300 employees serve more than 3,000 clients worldwide. For additional information, please visit www.surveysampling.com. 15. www.userzoom.com Methodology: Study Flow Tasks Research Account: Research the savings accounts available at [Bank] in the [ZIP / POSTAL CODE]. Find the minimum balance for the savings account that will earn you the most interest. Customer Service: Find a phone number you can call to ask questions about banking with [BANK]. What are the last four digits of the number? Find a Branch: Find a [BANK] branch in [CITY, ZIP / POSTAL CODE] to open a savings account in person. Determine what information the site provides about the branch (e.g., services available, phone number, address) Introduction Screener Wrap-up 16. www.userzoom.com Methodology: Participant Experience 17. www.userzoom.com Methodology: Participant Background 90% 87% 76% 74% 72% 71% 60% 43% Check account balance View transactions Find an ATM / branch View / pay bills View account statement Transfer Money Research banking services (e.g., accounts, credit cards, loans) Deposit money Banking Activities Conducted via Smartphone N=198 18. www.userzoom.com Summary 19. www.userzoom.com Summary (1of 2) CIBC had the highest overall ratings. Participants found it easiest to research accounts on CIBC compared to the other sites and encountered the least number of issues when performing the three tasks. KeyBank tied with CIBC for the highest overall satisfaction ratings and the highest Net Promoter Score. Although task success was very low for researching an account, ease of use ratings and frustration ratings were not significantly different from the top site (CIBC). Participants found KeyBank easiest to use when finding customer service information and also rated KeyBank highest when finding a branch USAA performed third-best in the study. Participants found the site harder to use when researching accounts and finding customer service information and generally found that too many clicks were required to find information. 20. www.userzoom.com Summary (2 of 2) Wells Fargo had the lowest overall ratings for ease of use, organization and layout and satisfaction primarily due difficulties in navigation and participants having to use the full site to perform the tasks Although Wells Fargo has a mobile optimized web site, participants were forced to navigate to the full site to research an account For task 2, locating a customer service phone number 22% reported frustration in using the full site, even though this information was available on the mobile site Clicking on the incorrect link was among the top problems Participants brand perception improved significantly from pre to post mobile site evaluation However, Wells Fargo had the smallest improvement in brand perception, while the brand perception improvements in the other site were comparable Net promoter scores for Wells Fargo was lowest overall (- 44%), while KeyBank had the highest Net Promoter Score (8%) Bank Delta Wells Fargo + 13% KeyBank + 31% USAA + 32% CIBC + 27% 21. www.userzoom.com Overall Findings 22. www.userzoom.com Overall Findings Key Performance Metrics KeyBank USAA CIBC Ease of use 30%* 56% 46% 58% Organization and Layout 36% 46% 40% 50% Satisfaction 34% 50% 42% 50% Likelihood to open an account 16% 16% 14% 22% Site with the highest rating Site with the lowest rating*Significantly lower than site with highest rating at 90% CI Ease of use, Organization and Layout, and Satisfaction: Responded 6-7 on a 7-point scale when asked, Please rate your experience with the following aspects of the mobile site. where 1 = Negative and 7 = Positive Likelihood to open an account: Responded 6-7 on a 7-point scale when asked, Based only on your experience today, how likely would you be to open a bank account with [BANK]? where 1 = Not at all likely and 7=Extremely likely 23. www.userzoom.com Overall Findings CIBC Video Question Compared to the other sites, CIBC rated highest across all key performance measures: Ease of use Organization and layout Overall satisfaction Likelihood to open an account 24. www.userzoom.com Overall Findings Privacy and Trust The bank seems to handle privacy and security well 32% 32% 28% 36% The mobile online bank seems trustworthy 34% 44% 32% 44% I would trust this bank with my personal information 32% 38% 34% 42% Site with the highest rating Site with the lowest rating*Significantly lower than site with highest rating at 90% CI Based on your experience with the [BANK] site on your mobile device, please rate your level of agreement with the following statements. Responses of 6-7 shown from a 7-point scale where 1 = Strongly disagree and 7=Strongly agree 25. www.userzoom.com Overall Findings Brand Perception 13% 11% 12% 17% 26% 42% 44% 44% Wells Fargo KeyBank USAA CIBC Pre Brand Perception Post Brand Perception N=48 N=50 N=18 N=50 N=25 N=50 N=46 N=50 Bank Delta Wells Fargo + 13% KeyBank + 31% USAA + 32% CIBC + 27% Pre vs. Post Brand Perception Based on what you know / your experience today, what is your perception of [BANK]? Responses of 6-7 shown from a 7-point scale where 1 = Negative and 7=Positive 26. www.userzoom.com Overall Findings Brand Attributes 11% -9% 10% -17% -5% 1% 5% -8% -13% -5% -3% 16% 25% 15% 14% 4% -10% -12% 43% -16% -5% 2% 0% -4% 20% 4% 8% 10% 4% -6% 42% 10% 2% 0% -2% 4% 11% 2% -3% 6% 2% 3% 11% -4% -5% 1% -1% -1% Wells Fargo KeyBank USAA CIBC Reliable Convenient Competitive Easy access Comprehensive Trustworthy Secure Exclusive Expensive Untrustworthy Unreliable Difficult Change in Associated Brand Attributes Pre vs. Post Mobile Site Evaluation Positive Negative Which of the following words or phrases would you use to describe [BANK]? 27. www.userzoom.com Overall Findings Wells Fargo Video Question Although fewer participants associated Wells Fargo with the Expensive after evaluating the mobile site, 10% still considered Wells Fargo to be Expensive. 28. www.userzoom.com Overall Findings Net Promoter Score 60% 26% 52% 38% 24% 40% 30% 46% 16% 34% 18% 16% Wells Fargo KeyBank USAA CIBC Detractor (0-6) Passives (7-8) Promoters (9-10) - 44% 8 % - 34% - 22% NPS How likely is it that you would recommend this website to a friend or a colleague? 29. www.userzoom.com Task 1 Research Account Description: Research the savings accounts available at [Bank] in the [ZIP / POSTAL CODE]. Find the minimum balance for the savings account that will earn you the most interest. 30. www.userzoom.com Task 1- Research Account Task success 34% 4%* 20% 28% Ease of use 24% 22% 20%* 36% Did not encounter problems / frustrations 8%* 22% 28% 32% Time on task (min) 3.07 2.21 1.35 1.52 Site with the highest rating / fastest time on task Site with the lowest rating / slowest time on task *Significantly lower than site with highest rating at 90% CI Task Success: Selected the correct response when asked, What is the minimum balance for a savings account that will earn you the most interest? Ease of use: Responded 6-7 on a 7-point scale when asked, How easy or difficult was it to find the minimum balance for the savings account that will earn you the most interest? where 1 = Difficult and 7=Easy. Did not encounter problems / frustrations: Selected I did not encounter problems and frustrations when asked, Which of the following problems or frustrations, if any, did you encounter while researching savings accounts at [BANK] from your mobile device? 31. www.userzoom.com Task 1 Amount of Information 42% 54% 44% 26% 40% 38% 50% 70% 18% 8% 6% 4% Wells Fargo Key Bank USAA CIBC Far Too Much Just Right Too Little Information comparing each account with the other I would have liked to see more information on how long before you accumulate interest Ability to search that would then connect to full site Clearer info about options on mobile site Information about their savings account. There should be a little more information on every title/name of the accounts. A comparison chart for accounts Key details like minimum amounts and interest Details like fees for if your balance drops below the minimum How would you rate the amount of information available on the site? What information was lacking? 32. www.userzoom.com Task 1 Problems and Frustrations 46% 18% 22% 14% 28% 20% 16% 28% 18% 10% 24% 2% 2% 8% 12% 6% 12% 4% 8% 12% 12% 40% 32% 8% 24% 12% 8% 22% 12% 6% 4% 0% 8% 4% 6% 20% 16% 14% 6% 4% 4% 28% 6% 12% 14% 2% 8% 10% 4% 18% 38% 16% 8% 2% 4% 32% Had to use the full site Full site did not fit onto my mobile screen Had to readjust screen frequently Links were too close together Text was too small / difficult to read Clicked on the wrong link Account information was scattered Too many clicks required to find information Difficult to compare accounts Not enough account information available Information was confusing / unclear Site was slow Encountered errors / technical issues I did not encounter any problems or Wells Fargo Key Bank USAA CIBC Content Navigation Site Performance Site Interaction Which of the following problems or frustrations, if any, did you encounter while researching savings accounts at [BANK] from your mobile device? [Check all that apply] 33. www.userzoom.com Task 1 Wells Fargo First clicks Sign On 17% ATMs/Location s 15% CEO Mobile 4% Quick Guide 45% Full Site 19% Most efficient path 34. www.userzoom.com Task 1 Wells Fargo Video Session Replay 35. www.userzoom.com Task 2 Customer Service Description: Find a phone number you can call to ask questions about banking with [BANK]. What are the last four digits of the number? 36. www.userzoom.com Task 2 Customer Service Task success 66%* 68%* 94% 72%* Ease of use 74% 84% 72% 78% Did not encounter problems / frustrations 58%* 82% 76% 86% Time on task (min) 0.43 0.28 0.39 0.30 Task Success: Selected the correct response when asked, What are the last four digits of the phone number you would call for questions about banking? Ease of use: Responded 6-7 on a 7-point scale when asked, How easy or difficult was it to find a phone number? where 1 = Difficult and 7=Easy. Did not encounter problems / frustrations: Selected I did not encounter problems and frustrations when asked, Which of the following problems or frustrations, if any, did you encounter while finding a phone number? Site with the highest rating / fastest time on task Site with the lowest rating / slowest time on task *Significantly lower than site with highest rating at 90% CI 37. www.userzoom.com Task 2 USAA Video Session Replay 38. www.userzoom.com Task 2 Customer Service 22% 2% 2% 2% 2% 8% 6% 2% 4% 4% 58% 0% 2% 0% 0% 4% 6% 2% 2% 2% 4% 82% 2% 0% 4% 0% 0% 2% 14% 4% 0% 0% 76% 0% 4% 4% 4% 2% 4% 6% 2% 2% 0% 86% Had to use the full site Site did not fit onto my mobile screen Had to readjust screen frequently Links were too close together Text was too small / difficult to read Clicked on the wrong link Too many clicks required to find information Not enough information provided Site was slow Encountered errors / technical issues I did not encounter any problems or frustrations Wells Fargo Key Bank USAA CIBC Site Interaction Navigation Content Technical Issues Which of the following problems or frustrations, if any, did you encounter while finding a phone number? [Check all that apply] 39. www.userzoom.com Task 3 Find a Branch Description: Find a [BANK] branch in [CITY, ZIP / POSTAL CODE] to open a savings account in person. Determine what information the site provides about the branch (e.g., services available, phone number, address) 40. www.userzoom.com Task 3 - Find a Branch Task success 72%* 86% 84% 76% Ease of use 58%* 78% 62% 64% Did not encounter problems / frustrations 46%* 74% 56% 62% Time on task (min) 1.07 1.03 1.18 1.14 Task Success: Selected Yes when asked, Were you able to find a bank branch in the 90015 zip code? Ease of use: Responded 6-7 on a 7-point scale when asked, How easy or difficult was it to find a bank branch on your mobile device? where 1 = Difficult and 7=Easy. Did not encounter problems / frustrations: Selected I did not encounter problems and frustrations when asked, Which of the following problems or frustrations, if any, did you encounter while finding a bank branch on your mobile device? Site with the highest rating / fastest time on task Site with the lowest rating / slowest time on task *Significantly lower than site with highest rating at 90% CI 41. www.userzoom.com Task 3 CIBC Video Session Replay 42. www.userzoom.com Task 3 Problems and Frustrations 12% 2% 2% 0% 6% 12% 10% 10% 10% 4% 8% 46% 0% 4% 0% 2% 4% 2% 10% 2% 2% 0% 4% 74% 8% 4% 0% 6% 0% 10% 8% 22% 6% 4% 0% 56% 0% 6% 2% 4% 4% 2% 6% 12% 4% 4% 0% 62% Had to use the full site Site did not fit onto my mobile screen Links were too close together Had to readjust screen frequently Text was too small / difficult to read Information was confusing / unclear Not enough information provided Too many clicks required to find information Clicked on the wrong link Site was slow Encountered errors / technical issues I did not encounter any problems or frustrations Wells Fargo Key Bank USAA CIBC Site Interaction Content Navigation Site Performance Which of the following problems or frustrations, if any, did you encounter while finding a bank branch on your mobile device? [Check all that apply] 43. www.userzoom.com Mobile Best Practices 44. www.userzoom.com Best Practices Mobile Research (1of 5) 1. Duration Restrict mobile study length to 15 minutes or 2-3 tasks and 15 questions. After this threshold, the drop-off rate doubles. As the duration increases, incentives must increase commensurately. 2. Expectations Provide clear descriptions about the task. E.g. How many tasks? Websites? Do users have to visit a location? E.g. A store, at home, outside, etc. Goal: Improve completion rates & improve quality of response 45. www.userzoom.com Best Practices Mobile Research (2 of 5) 3. Study Requirements State the mobile software requirements. E.g., iOS 4+ or Android 4+ Notify participants if there is a minimum battery charge. E.g. 30% min battery to record video session replay Inform participants if Wifi should be enabled 46. www.userzoom.com Best Practices Mobile Research (3 of 5) 4. Question types Use rating scales, radio or multiple choice responses and drag and drop features. Restrict open ended question type to the minimum. Use video question type to minimize using tiny keyboard. Minimize matrix questions which can be difficult for users to respond on small screens. Keep list choices to a minimum. More than 10 will require scrolling 47. www.userzoom.com Best Practices Mobile Research (4 of 5) 5. Instructions Provide concise and easy-to-read task descriptions Highlight key parts through the use of bolding, a different color, underlining, or italics to facilitate scanning. Update participants of their progress through the study (E.g., Task 2 of 3) 48. www.userzoom.com Best Practices Mobile Research (5 of 5) 6. Recruitment Use panel vendor that specializes in mobile recruitment Explore methods for recruitment, email invitation, text message or QR code 7. Incentives Higher incentives may be required for mobile studies ~ $ 12 25 depending on the complexity and duration of the study 49. www.userzoom.com Q&A Follow us: Twitter: @userzoom Linkedin: https://www.linkedin.com/company/userzoom