Upload
fqcijk
View
1.641
Download
0
Tags:
Embed Size (px)
DESCRIPTION
! 然后香港六合彩牵着我的手一蹦一跳地跟着我走了出来,让我感觉自己象是牵了只哈巴狗——香港六合彩那大龄女青年的模样呢?怎么不装了? 一路上香港六合彩根本就没认为我是真带香港六合彩去见赵玉,不断地要求我带香港六合彩在校园里参观参观,当是欧洲半日游.于是我说瞧,那是澡堂,那是茅坑,这都是香港六合彩学校的重要场所行了行了,别闹了,今天有正事儿哪. 直到走上外语楼那座小山时,香港六合彩才终于慢慢明白了赵玉的客观存在性和酷哥唐飞的唯物辩证性,香港六合彩的笑容一点一点地凝结在脸上,也不再叽叽歪歪个不停了. 当香港六合彩来到外语楼前准备往里走时,香港六合彩停了下来,以一种不敢相信的目光望着我,呆呆地问:是真的?! 我点头. 香港六合彩的眼泪立刻卟卟地滚落了下来——当然,这一招香港六合彩两年前就已经臻峰肇极了,向来是说来就来——香港六合彩狠狠地甩开我的手,喊道:唐飞!你说过要等我一万年的!你说你要等我的!你骗我!你骗我! 我说雨儿 香港六合彩大叫我不听我不
Citation preview
Measuring ROI on ACSI Surveys
Tim EvansTim EvansSocial Security AdministrationSocial Security AdministrationOffice of Electronic ServicesOffice of Electronic Services
[email protected]@ssa.gov
IntroductionIntroduction
ROI on your surveys not directly attributable to ROI on your surveys not directly attributable to your survey expenditures—you may be your survey expenditures—you may be validating prior workvalidating prior work
Ways of using ACSI data for ROI purposesWays of using ACSI data for ROI purposes– Identify user behavior that explicitly shows your site’s Identify user behavior that explicitly shows your site’s
valuevalue– Identify data that shows how you can improve your Identify data that shows how you can improve your
site to increase its valuesite to increase its value– Identify and focus your (limited) resource prioritiesIdentify and focus your (limited) resource priorities
SSA and ACSISSA and ACSI
Began Q1/04Began Q1/04Four current surveysFour current surveys– Main siteMain site– FAQ’s siteFAQ’s site– Retirement Planner siteRetirement Planner site– Internet Social Security Benefit Application--ISBA (not Internet Social Security Benefit Application--ISBA (not
published)published)
Two more surveys plannedTwo more surveys planned– Business Services On-line (employer wage reporting)Business Services On-line (employer wage reporting)– Disability (largest single group of surveyed visitors Disability (largest single group of surveyed visitors
say they came for disability info)say they came for disability info)
SSA ROI for ACSI Surveys--SSA ROI for ACSI Surveys--SummarySummary
Main survey shows high value + return on 2003 Main survey shows high value + return on 2003 site redesignsite redesign
FAQ survey shows clear reduction in use of FAQ survey shows clear reduction in use of high-cost support channels as direct result of high-cost support channels as direct result of FAQ’s availabilityFAQ’s availability
Retirement Planner survey shows high value of Retirement Planner survey shows high value of Planner + the opportunity to focus our resourcesPlanner + the opportunity to focus our resources
ISBA Survey helps clarify resource priorities in ISBA Survey helps clarify resource priorities in limited-budget environmentlimited-budget environment
Main Site Survey, Q3/04Main Site Survey, Q3/04http://www.socialsecurity.govhttp://www.socialsecurity.gov
OverallOverall 7575
ContentContent 8080
FunctionalityFunctionality 7676
Look & FeelLook & Feel 7979
NavigationNavigation 7373
SearchSearch 7474
Site PerformanceSite Performance 8383
Compare to ForeSee Results Compare to ForeSee Results BenchmarksBenchmarks
Overall Satisfaction: +5 Overall Satisfaction: +5 vis a visvis a vis Q2/04 Q2/04 Gov’t benchmark; +4 Gov’t benchmark; +4 vis a vis vis a vis overalloverall
Look & Feel: +4/+3Look & Feel: +4/+3
Content: +2/+2Content: +2/+2
Navigation: +5/+3Navigation: +5/+3
2003 Site Redesign2003 Site Redesign
Site-wide adoption of CSS Style Sheets Site-wide adoption of CSS Style Sheets for common look & feel (colors, fonts, etc.)for common look & feel (colors, fonts, etc.)
Added common header with top nav-linksAdded common header with top nav-links
Primarily implemented via common Primarily implemented via common DreamWeaverDreamWeaver template for easy template for easy propagation site widepropagation site wide
Context: ~80 SSA webmasters with Context: ~80 SSA webmasters with central, but informal governancecentral, but informal governance
SSA FAQ’s Survey, Q3/04SSA FAQ’s Survey, Q3/04http://ssa-custhelp.ssa.gov/cgi-bin/ssa.cfg/php/endhttp://ssa-custhelp.ssa.gov/cgi-bin/ssa.cfg/php/end
user/entry.phpuser/entry.php
Overall ScoreOverall Score 7474
ContentContent 8181
FunctionalityFunctionality 7777
Look & FeelLook & Feel 7979
NavigationNavigation 7575
SearchSearch 7575
Site PerformanceSite Performance 8484
Custom Questions Results Reveal Custom Questions Results Reveal ROI--#1ROI--#1
If there were no FAQ’s, how would you have If there were no FAQ’s, how would you have contacted Social Security to ask your question? contacted Social Security to ask your question? (N=1273)(N=1273)– Send e-mailSend e-mail 16%16%– Browse web siteBrowse web site 8 8– Call 800 numberCall 800 number 3939– Call local SSA officeCall local SSA office 2222– Visit local SSA officeVisit local SSA office 1010– Write SSAWrite SSA 1 1– Would not have contacted SSAWould not have contacted SSA 4 4
Custom Questions Results Reveal Custom Questions Results Reveal ROI--#2ROI--#2
After reading our answer(s), what do you plan to After reading our answer(s), what do you plan to do next? (N=487)do next? (N=487)– Nothing; question answeredNothing; question answered 36%36%– Nothing; question not answered Nothing; question not answered 4 4– Send SSA e-mailSend SSA e-mail 6 6– Call SSA 800 numberCall SSA 800 number 1414– Call SSA local officeCall SSA local office 9 9– Visit local SSA officeVisit local SSA office 2222– Try different questionTry different question 4 4– Look elsewhere on siteLook elsewhere on site 3 3
Key Comparisons for 2 QuestionsKey Comparisons for 2 Questions
Send SSA e-mailSend SSA e-mail 16%16% 6% 6% -62% -62%
Call SSA 800 numberCall SSA 800 number 3939 1414 -65 -65
Call SSA local officeCall SSA local office 2222 9 9 -59 -59
Visit local SSA officeVisit local SSA office 1010 2222 +120+120
ROI—Reduction Use of High-Cost ROI—Reduction Use of High-Cost Support ChannelsSupport Channels
Forester Research Estimates Cost of ServicesForester Research Estimates Cost of Services– Knowledge-Based Self Service: $1.17/incidentKnowledge-Based Self Service: $1.17/incident– Telephone support: $33/incidentTelephone support: $33/incident– E-mail support: $10/incidentE-mail support: $10/incident
SSA Results, shown by ACSI dataSSA Results, shown by ACSI data– 65% reduction in calls to 800 number65% reduction in calls to 800 number– 59% reduction in calls to local office59% reduction in calls to local office– 62% reduction in e-mails62% reduction in e-mails– 120% increase in local office visits120% increase in local office visits
SSA Retirement Planner Survey, SSA Retirement Planner Survey, Q3/04Q3/04
http://www.socialsecurity.gov/r&m1.htmhttp://www.socialsecurity.gov/r&m1.htm
Overall ScoreOverall Score 7777
ContentContent 8282
FunctionalityFunctionality 8080
Look & FeelLook & Feel 8181
NavigationNavigation 7777
Site PerformanceSite Performance 8585
Tasks/TransactionsTasks/Transactions 7878
SSA Retirement Planner Survey, SSA Retirement Planner Survey, Q3/04--FuturesQ3/04--Futures
Likely to ReturnLikely to Return 8585
RecommendRecommend 8383
File on-lineFile on-line 6161
What’s wrong with this picture?What’s wrong with this picture?
Is it good news or bad?Is it good news or bad?
Retirement Planner—Custom Retirement Planner—Custom QuestionQuestion
How do you plan to file for retirement benefits? How do you plan to file for retirement benefits? (7/12-8/9 data; N=1127)(7/12-8/9 data; N=1127)– On lineOn line 39%39%– Face to faceFace to face 3737– TelephoneTelephone 4 4– On line w/SSA helpOn line w/SSA help 2 2
More recent data shows ‘on line w/help’ running More recent data shows ‘on line w/help’ running ~13%, ‘on line’ ~30%--totaling ~13%, ‘on line’ ~30%--totaling more than 40%more than 40% (not enough data for conclusions)(not enough data for conclusions)
Pew Internet StudyPew Internet Studyhttp://www.pewinternet.org/PPF/r/128/report_display.asphttp://www.pewinternet.org/PPF/r/128/report_display.asp
How Americans Get in Touch with How Americans Get in Touch with GovernmentGovernment (May, 2004) (May, 2004)– TelephoneTelephone 40%40%– WebWeb 2424– In personIn person 1313– E-mailE-mail 1111– Write a letterWrite a letter 1010
Pew Internet Study—Cont’dPew Internet Study—Cont’d
Why Americans contact GovernmentWhy Americans contact Government– Conduct TransactionsConduct Transactions 30%30%– Get specific informationGet specific information 2525– Register opinionsRegister opinions 1919– Help w/specific problemsHelp w/specific problems 1111
For complex, problem-solving contacts, For complex, problem-solving contacts, 62% prefer phone/in person62% prefer phone/in person
What Does This Tell Us About ROI What Does This Tell Us About ROI for Retirement Planner?for Retirement Planner?
With ~40% interested in filing on line, SSA With ~40% interested in filing on line, SSA is doing quite well (compare Pew 30% is doing quite well (compare Pew 30% transactions figure)transactions figure)Futures Score shows opportunity to Futures Score shows opportunity to improve on-line application, as Boomers improve on-line application, as Boomers age and interest in on-line filing risesage and interest in on-line filing risesNew ‘On-line w/Help’ Numbers strong New ‘On-line w/Help’ Numbers strong indicator for future, bearing out Pew indicator for future, bearing out Pew conclusions on complex contactsconclusions on complex contacts
Internet Social Security Benefit Internet Social Security Benefit Application (ISBA)Application (ISBA)
Retirement/spouse applicants vastly more Retirement/spouse applicants vastly more satisfied (>2:1) than disability applicantssatisfied (>2:1) than disability applicantsFar more (5:1) retirement/spouse applicants Far more (5:1) retirement/spouse applicants successfully complete on-line applicationsuccessfully complete on-line applicationLowest scores in Functionality, Lowest scores in Functionality, Tasks/Transactions; cut across board, Tasks/Transactions; cut across board, especially especially among disability applicantsamong disability applicantsDisability applicants have far lower level of Disability applicants have far lower level of Internet experience (5:1 “beginner”), and may Internet experience (5:1 “beginner”), and may need additional helpneed additional helpExpectations count: level of Internet experience Expectations count: level of Internet experience inversely proportional to satisfactioninversely proportional to satisfaction
ISBA and ROI LessonsISBA and ROI Lessons
Retirement/spouse applicants >80% of ISBA Retirement/spouse applicants >80% of ISBA customers; universe will grow as Boomers age customers; universe will grow as Boomers age and demand service (Pew data cited earlier)and demand service (Pew data cited earlier)Disability applicants have lower Internet Disability applicants have lower Internet experience, need more support via high-cost experience, need more support via high-cost channelschannelsThe whole disability application process differs The whole disability application process differs substantially from retirement application processsubstantially from retirement application processWith resources limited, best use lies in improving With resources limited, best use lies in improving overall Functionality, Tasks/Transactionsoverall Functionality, Tasks/TransactionsAll applicants will benefit from general All applicants will benefit from general improvements in the on-line applicationimprovements in the on-line application
WrapWrap
ROI on your surveys not directly ROI on your surveys not directly attributable to your survey expendituresattributable to your survey expenditures
Ways of using ACSI data for ROI Ways of using ACSI data for ROI purposespurposes– Identify user behavior that explicitly shows Identify user behavior that explicitly shows
value of your sitevalue of your site– Identify data that shows how you can improve Identify data that shows how you can improve
your site to increase its valueyour site to increase its value– Identify and focus your resource prioritiesIdentify and focus your resource priorities
SSA ConclusionsSSA Conclusions
Main survey shows high value + return on 2003 Main survey shows high value + return on 2003 site redesignsite redesign
FAQ survey shows clear reduction in use of FAQ survey shows clear reduction in use of high-cost support channels as direct result of high-cost support channels as direct result of FAQ’s availabilityFAQ’s availability
Retirement Planner survey shows high value of Retirement Planner survey shows high value of Planner + the opportunity to focus our resourcesPlanner + the opportunity to focus our resources
ISBA Survey helps clarify resource priorities in ISBA Survey helps clarify resource priorities in limited-budget environmentlimited-budget environment