WWW.TOPCODER.COM

Enterprise Open Innovation

AGENDA

Theme: Community, by Community

• A little about TopCoder/Appirio• Crowdsourcing• Contests• TopCoder Crowdsourcing• When Communities Build Communities• Contributing to the Asteroids Mission

TOPCODER

ENGAGING TOP MINDS TO SOLVE ENTERPRISE NEEDS

ENTERPRISECUSTOMERS

FRONT-END DESIGN & INNOVATION

SOFTWAREDEVELOPMENT

ALGORITHM/ANALYTICS

592K COMMUNITYTOP DESIGNERS, DEVELOPERS

AND ALGORITHM / ANALYTIC EXPERTS

KEY AREAS:IA/UX Design, Development& Algorithms / Analytics

Development

Assembly

Testing

Bug Races

Architecture

Design Concepts

User Interface Design

Wireframes

Storyboards

Prototype

Marathon Matches

Single-Round Match

TOPCODER

SOFTWAREDEVELOPMENT

FRONT-ENDDESIGN & INNOVATION

ALGORITHMANALYTICS

Subjective Objective PerformanceRigorous Scorecards

Decide WinnerClient Decides Winner Best solution against

pre-determined metrics wins

Web & Mobile Apps

Websites, E-CommercePlatform & Strategic Assets

Legacy System Integration& Cloud Utility Tools

Analytics

What Customers are Building

Lots accomplished with Crowdsourcing:

• Linux• Java• Wikipedia• fold.It• NatGeo – Field Expedition: Mongolia

CROWDSOURCING

CONTESTS

Lots accomplished with Contests:

• Longitude Contest• White House Portico Design• Canned food!• Lindbergh’s Flight• Ansari X-Prize• And a few things on TopCoder…

SELECT ACADEMIC RESEARCH ON COMMUNITIES

• Lakhani, Karim, David Garvin, and Eric Lonstein. "Topcoder (a): Developing software through crowdsourcing." Harvard Business School General Management Unit case 610-032 (2010).

• Malone, Thomas W., Robert J. Laubacher, and Tammy Johns. "The age of hyperspecialization." Harvard Business Review 89.7-8 (2011): 56-+.

• Wu, Wenjun, Wei-Tek Tsai, and Wei Li. "An evaluation framework for software crowdsourcing." Frontiers of Computer Science: 1-16.

• Archak, Nikolay. "Money, glory and cheap talk: analyzing strategic behavior of contestants in simultaneous crowdsourcing contests on TopCoder. com."Proceedings of the 19th international conference on World wide web. ACM, 2010.

• Begel, Andrew, Jan Bosch, and Margaret-Anne Storey. "Social Networking Meets Software Development: Perspectives from GitHub, MSDN, Stack Exchange, and TopCoder." Software, IEEE 30.1 (2013): 52-66.

• Boudreau, Kevin J., Nicola Lacetera, and Karim R. Lakhani. "Incentives and problem uncertainty in innovation contests: An empirical analysis."Management Science 57.5 (2011): 843-863.

• Kaganer, Evgeny, et al. "Managing the Human Cloud." MIT SLOAN MANAGEMENT REVIEW 54.2 (2013): 23-32.

• Eisenstein, Michael. "Crowdsourced contest identifies best-in-class breast cancer prognostic." Nature biotechnology 31.7 (2013): 578-580.

• Boudreau, Kevin J., and Karim R. Lakhani. "How to Manage Outside Innovation." MIT SLOAN MANAGEMENT REVIEW

• Lakhani, Karim R., et al. "Prize-based contests can provide solutions to computational biology problems." Nature biotechnology 31.2 (2013): 108-111.

• Zhang, Yin, and Ruoyu Lu. "The case study on innovation contest modes of Internet companies." Service Systems and Service Management (ICSSSM), 2011 8th International Conference on. IEEE, 2011.

600+ academic articles (source: Google Scholar)Nearly two dozen+ patents on crowdsourcing and software development

TopCoder Crowdsourcing

• NASA Tournament Lab• MegaBLAST• Medicaid!• Community Sites

CROWDSOURCING WITH CONTESTS

TopCoder CROWDSOURCING WITH CONTESTS

NASA and Harvard University have established the NASA Tournament Lab (NTL), to leverage the TopCoder community and allow for competitions to create the most innovative, most efficient, and most optimized solutions for specific, real-world challenges being faced by NASA researchers

FIT (Food Intake Tracker)iPad ApplicationThe application can identify the Astronaut, track all dietary intake (food and beverages), and provide a timestamp of when the food was consumed on space.

Pipeline ProtectionDetection AlgorithmA detection algorithm to spot the presence of wheeled vehicles in proximity to a pipeline with the ability to distinguish between a security threat an un-authorized vehicle and a non-threat such as wild-life or a downed tree limb.

Planetary Data SystemsWeb API, iOS and Android AppsDevelop and deploy an API to homogenize search results across a federation of PDS nodes. Demonstrate the value of the API with a suite of mobile consumer apps.

International Space Station: Solar Panel Alignment

4000+ Registrants, $30,000 TotalThe goal of the challenge is to position the solar collectors on the International Space Station to generate as much power as possible during the most difficult orbital positions.

NASA Tournament Lab (NTL)

CODERS SUBMITTED SOLUTIONS DIFFERENT APPROACHES TOSOLVE PROBLEM IDENTIFIED

WINNING COUNTRIESRUSSIA, FRANCE, EGYPT, BELGIUM & US

122 654 89 5

Improve on NIH MegaBlast algorithmfor nucleotide sequence alignment

Winning solution performs 120x faster

ANTIBODY SEQUENCE ANNOTATION

TOPCODER CROWDSOURCING

Modernizing Medicaid and Medicare

• Implement with > 130 individual contests• Design and build a new single Medicaid module for screening providers for use by all

states• Single proposal was submitted for RFP for 3 years $120M+ for 80 modules for 1 state• Game change outcome for US with $100Ms in savings possible

• Allows for late-entry stakeholder participation• States can engage at any time through conceptualization contests

• TopCoder analyst competitors review state requirements and wish-lists• TopCoder competitors organize results for changes, additions, priorities• Stakeholder steering committee can assign requirements to later phases

• Contests allow all voices to be heard and managed

• Directly tests HHS Architecture Compliance• Specification contests will explore and document requirements• “Bug Hunts” will interrogate designs and code for government compliance

TOPCODER CROWDSOURCING

Portal: Innovation and Cost Savings

Development Contractor Federal Development Project Support

Federal Infrastructure Support Total Costs $-

$400,000

$800,000

$1,200,000

$1,600,000

$2,000,000

$2,400,000

$2,800,000

$3,200,000

$3,600,000

$4,000,000

$4,400,000

$4,800,000

$5,200,000

$5,600,000

$6,000,000

$6,400,000

$6,800,000

$7,200,000

$7,600,000

$6,004,924

$735,549 $717,540

$7,458,013

$1,433,806

$59,220 $27,050

$1,520,076

TRADITIONAL MODEL CROWD MODEL

COMMUNITIES

When Communities build Communities

• The Pilgrims• Linux • Java• Ghost

Contribution Communities

By WordPress engineers who wanted to focus on Blogs+ Part-time devs they trusted+ Kickstarter funding!

COMMUNITIES: @TOPCODER

When Communities build Communities(with TopCoder & others)

• TopCoder.com• NoNameSite.com• ZeroRobotics• Verigames.com• Expedition Mongolia

Responsive Collaboration Communities

• Online Collaboration Centers• With Infrastructure and Instrumentation• For Incentives & Innovation

COMMUNITIES: @TOPCODER

TopCoder.com

COMMUNITIES: @TOPCODER

COMMUNITIES: @TOPCODER

ZeroRobotics.com• Spherical satellites move with independent control of

all six degrees of freedom in zero gravity (inside the international space station)

• These “spheres” travel within a defined area to complete tasks in a game scenario developed by researchers at MIT

• High School students compete against one another to earn the most points using a high-level language environment

• Virtual zero-gravity and visual simulation is used to allow online competition before the finals are executed on the space station

• Zero Robotics infrastructure and web site was built by TopCoder

COMMUNITIES: @TOPCODER

COMMUNITIES: @TOPCODER

Verigames.com

Perform formal software verification through games accessible to unspecialized players.

• A community platform• Provides profiles, badges, forums• Hosts 3rd Party Games• Provides storage & display for:

• Scores• Achievements• History

COMMUNITIES: @TOPCODER

Verigames.com

COMMUNITIES: @NATGEO

National Geographic Explorer

COMMUNITIES: ASTEROIDS

Advancing the Asteroid Initiative

• Build communities around specialized challenges• Encourage collaboration, and fun competition• Connect the “Carbon Crowd” to the “Silicon Cloud”?

Human Crowd Perception

Algorithmic Automation

Can an algorithm rank image analysis tasks in order of potential?

SIMILAR PROMISING RESULTS: Collective Minds Challenge

Approaches:1. Random forest approach to a reasonably

short list of features, mostly concentrating on edges and circles. Features have been collected on areas of different sizes. Performance continues to improve with more training data.

2. Random forest approach to large number of features, mostly based on color-gradient analysis. Features have been analyzed on areas of different sizes. Performance saturates after a few thousand training samples. 500 1000 2000 4000 6000 8000

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Algo 1

Algo 2

Algo 3

Algo 4

Illustration 3: Test1 Ancient Structures

1st and 2nd place submissions were good-quality generic-solution algorithms. These algorithms are comparable in scoring, and clearly outscore the third and fourth places.

510 Registrants 394 Solutions (~10 per competitor)4 Winners9 Submissions in Idea Generation Contest

21 day challenge $15k Prize Purse

Credit: Dr. Rinat Sergeev, NASA Tournament Lab, Harvard IQSS

Concentrate on the real goals- What are you trying to do?- What problem are you trying to solve?- What do you want your audience to be able to accomplish?- Stay focused – do not get technical

Learn to let go, allow creativity to flourish- Define goals but don’t define how!- Communicate! – be honest with your opinion- Allow yourself to be surprised and open to new ideas

Iteration- Fail! Be prepared to fail and learn from it.- Plan to review multiple submissions, look for “gems” within - Provide feedback!

CAPSTONE THOUGHTS

Open Innovation via Communities is the practice of finding a way to do something new through access to people you normally wouldn’t be engaged with.

Thank you!