Incentive Prizes:Designing and Measuring for

Impact

Cristin Dorgelo

Assistant Director for Grand Challenges

Office of Science and Technology Policy

Executive Office of the President

• Designing for higher impact, broader scope– Portfolio/laddering approach– Similar mechanisms– Legacy planning– Solution sharing

• Resources– Assets– Partners– Vendors

• Communications– Capturing impact (focus of today!)– Community stewardship

2013 Priorities for Public Sector Prizes

1. Exemplar (recognition)

2. Point-solution

3. Exposition

4. Participation

5. Network

6. Market stimulation

Prize Types from the McKinsey Report

1. Exemplar (recognition)

2. Point-solution• Apps challenges, often built on “open data”

• Algorithm and coding challenges

• Design/plan/proposal competitions (intersecting with Exposition & Participation)

3. Exposition• Ideation challenges

• “Plan + implementation” challenges (intersecting with Participation)

4. Participation• Identifying and developing promising talent/workforce

• Community-focused/behavior change (intersecting with Exposition & Network)

5. Network

6. Market stimulation• Complex technology demonstrations (intersecting with Point-solution)

• New markets/industries vs. existing markets (intersecting with Network)

Evolving Understanding of Prize Types

1. Ideation (internal/external; online/offline)

2. Collaborative innovation (without competition)• Hackathons/Codeathons

• Idea “Boot Camps”

3. Crowd-sourcing platforms

4. Crowd-funding

5. Open data

6. Grand Challenges

Intersection with Other Trends

New IdeasProof of

Concept/ Prototype

Technology Developme

nt

Application Developme

nt

Commercial Scale-Up

Adoption / Market

Development

Planning Prizes Along the Innovation Chain

• Ideation• Exposition• Participation

• Point Solution• Apps, Algorithm, & Coding

Challenges• Design/Plan/Policy Challenges

• Market Stimulation (existing markets)

• Participation

• Exposition• Network

• Market Stimulation (new markets)

Wide Variety of Possible Success Criteria

1. Return on investment (ROI) – Cost-effectiveness of competition– Financial leverage during competition– Follow-on investment post-competition

2. Quantity/quality/viability/diversity of submissions

3. Quantity/diversity of participants

4. Performance improvement compared to current solutions

5. Public awareness– Community engagement beyond “solvers”– Media impressions (traditional/earned media; social media)– Survey results

6. Sponsor/donor/partner satisfaction/engagement

7. Behavior change, adoption, and decay rate

• Target: 2500 gpm at 70% efficiency, 2x today’s standard oil recovery rate

• 350+ narrowed down to 10 finalist teams

• $1,000,000 First Place Winner Elastec (Illinois)

• Achieved 4x today’s standard oil recovery rate in <6 months product development

• 4670 gallons per minute at 89.5% efficiency

Wendy Schmidt Oil Cleanup X Challenge

Impact of the Oil Cleanup Challenge (1 of 2)

1. Return on investment (ROI) – Cost-effectiveness of competition– What did teams spend to compete?– Tracking industry investments in winning/qualified technology– Tracking job creation and mergers & acquisitions

2. Quantity/quality/viability/diversity of submissions– 10 finalist teams had widely diverse approaches– 7 met 1 of the 2 minimum performance thresholds (rate or

efficiency)– 2 of 3 prize purses awarded for meeting thresholds for both

recovery rate and efficiency

3. Quantity/diversity of participants– 300+ initial teams>10 finalists>2 winners– 5 domestic finalists; 5 international– 5 known industry players; 5 new to industry

Impact of the Oil Cleanup Challenge (2 of 2)

4. Performance improvement compared to current solutions

– 4x improvement over existing solutions– Multiple options optimized for diverse environments

5. Public awareness– Significant media impressions tracked through PR firm– Key stakeholder awareness

6. Sponsor/donor/partner satisfaction/engagement– Donor agreed to support follow-on prize– Sponsor participated in post-prize briefings to Federal agencies

Impact on Prize Design

1. Submission templates

2. Judging protocol

3. Tracking public relations/social media data

4. Post-competition surveys, interviews, and anecdotes

– Participants

– Followers

– Stakeholders

5. Research partners

6. Required reporting from solvers/winners

7. Legacy plan for post-competition tracking

Annual Report to OSTP/Congress

1. Due 12/31 (LAST MONTH!)

2. Need an extension?

3. Reporting on impact…consider all of the above

4. Dealing with and reporting failures & lessons learned

Thank You

cdorgelo@ostp.eop.gov