Enlightened experimentation the new imperative for innovation by s. thomke february 2001

66 HARVARD BUSINESS REVIEW

The high cost qf experimentation has long put

a damper on companies'attempts to create

great new products. But new technologies are

making it easier than ever to conduct complex

experiments quickly and cheaply Companies

now have an opportunity to take innovation to

a whole new level-if they're willing to rethink

their R&D from the ground up.

EnlightenedExperimentation

The New Imperative for

nnovation

by Stefan ThomkeE XPERIMENTATION LIES AT THE HEART OF EVERY

company's ability to innovate. In other words, thesystematic testing of ideas is what enables companies

to create and refine their products. In fact, no product canbe a product without having first been an idea that wasshaped, to one degree or another, tbrough the process ofexperimentation. Today, a major development project canrequire literally thousands of experiments, all with thesame objective; to leam whether the product concept orproposed technical solution holds promise for address-ing a new need or problem, then incorporating that in-formation in the next round of tests so that the bestproduct ultimately results.

FEBRUARY 2001 67

Enlightened Experimentation

In the past, testing was relatively expensive, so compa-nies had to be parsimonious with the number of experi-mental iterations. Today, however, new technologies sucbas computer simulation, rapid prototyping, and combina-torial chemistry allow companies to create more learningmore rapidly, and that knowledge, in tum, can be incor-porated in more experiments at less expense. Indeed, newinformation-based technologies have driven down themarginal costs of experimentation, just as they have de-creased the marginal costs in some production and distri-bution systems. Moreover, an experimental system thatintegrates new information-based technologies doesmore than lower costs; it also increases the opportunitiesfor innovation. That is, some technologies can make ex-isting experimental activities more efficient, while othersintroduce entirely new ways of discovering novel con-cepts and solutions.

Millennium Pharmaceuticals in Cambridge, Massachu-setts, for instance, incorporates new technologies such asgenomics, bioinformatics, and combinatorial chemistryin its technology platform for conducting experiments.The platform enables factory-like automation that cangenerate and test drug candidates in minutes or seconds,compared with the days or more tbat traditional methodsrequire. Gaining information early on about, say, tbe tox-icological profile of a drug candidate significantly im-proves Millennium's ability to predict the drug's success inclinical testing and, ultimately, in the marketplace. Un-promising candidates are eliminated before hundreds ofmillions of dollars are invested in their development. Inaddition to reducing the cost and time of traditional drugdevelopment, tbe new technologies also enhance Millen-nium's ability to innovate, according to Cbief TechnologyOfficer Michael Pavia. Specifically, the company hasgreater opportunities to experiment with more diversepotential drugs, including those tbat may initially seemimprobable but might eventually lead to breakthroughdiscoveries.

This era of "enlightened experimentation" has thus faraffected businesses with higb costs of product develop-ment, sucb as the pharmaceutical, automotive, and soft-ware industries. By studying them, I have learned severalvaluable lessons tbat I believe have broad applicability toother industries. As the cost of computing continues tofall, making all sorts of complex calculations faster andcheaper, and as new technologies like combinatorialchemistry emerge, virtually all companies will discoverthat they have a greater capacity for rapid experimenta-tion to investigate diverse concepts. Financial institutions,for example, now use computer simulations to test newfinancial instruments. In fact, the development of spread-

Stefan Thomke is an associate professor of technology andoperations management at Harvard Business School inBoston.

sheet software has forever changed financial modeling;even novices can perform many sophisticated what-if ex-periments that were once prohibitively expensive.

A System for ExperimentationUnderstanding enlightened experimentation requires anappreciation of tbe process of innovation. Namely, prod-uct and tecbnology innovations don't drop from tbe skj^they are nurtured in laboratories and development orga-nizations, passing througb a system for experimentation.All development organizations have such a system inplace to belp them narrow the number of ideas to pursueand tben refine that group into what can become viableproducts. A critical stage of the process occurs when anidea or concept becomes a working artifact, or prototype,which can tben be tested, discussed, shown to customers,and learned from.

Perhaps the most famous example of the experimentalsystem at work comes from the laboratories of ThomasAlva Edison. When Edison noted that inventive genius is"99% perspiration and i% inspiration," be was well awareof tbe importance of an organization's capability and ca-pacity to experiment. That's why he designed his opera-tions in Menlo Park, New Jersey, to allow for efficient andrapid experimental iterations.

Edison knew that the various components of a systemfor experimentation-including personnel, equipment,libraries, and so on-al l function interde pendent I y. Assuch, they need to be jointly optimized, for together theydefine tbe system's performance: its speed (the timeneeded to design, build, test, and analyze an experiment),cost, fidelity (the accuracy of the experiment and the con-ditions under which it is conducted), capacity (the num-ber of experiments that can be performed in a given rimeperiod), and the learning gained (the amount of new in-formation generated by the experiment and an organiza-tion's ability to benefit from it). Tbus, for example, bighiyskilled machinists worked in close proximity to lab per-sonnel at Menlo Park so tbey could quickly make im-provements when researchers had new ideas or learnedsomething new from previous experiments. This systemled to landmark inventions, including the electric light-bulb, which required more than i,ooo complex experi-ments with filament materials and shapes, electromechan-ical regulators, and vacuum technologies.

Edison's objective of achieving great innovationthrough rapid and frequent experimentation is especiallypertinent today as the costs (both financial and time) ofexperimentation plunge. Yet many companies mistakenlyview new technologies solely in terms of cost cutting,overlooking tbeir vast potential for innovation. Worse,companies witb that limited view get bogged down in theconfusion that occurs when they try to incorporate newtechnologies. For instance, computer simulation doesn't



simply replace physical prototypes as a cost-saving measure; it introduces an entirelydifferent way of experimenting that invitesinnovation. Just as the Intemet offers enor-mous opportunities for innovation-far sur-passing its use as a low-cost substitute forphone or catalog transactions - so doesstate-of-the-art experimentation. But realiz-ing that potential requires companies toadopt a different mind-set.

Indeed, new technologies affect every-thing, from the development process itself,including the way an R&D organization isstructured, to how new knowledge-andhence learning-is created. Thus, for com-panies to be more innovative, the challengesare managerial as well as technical, as thesefour rules for enlightened experimentationsuggest:

I.Organizefor rapid experimentation.The ability to experiment quickly is integralto innovation: as developers conceive of amultitude of diverse ideas, experimentscan provide the rapid feedback necessaryto shape those ideas by reinforcing, modi-fying, or complementing existing knowl-edge. Rapid experimentation, however,often requires the complete revamping ofentrenched routines. When, for example,certain classes of experiments become anorder of magnitude cheaper or faster, orga-nizational incentives may suddenly becomemisaligned, and the activities and routinesthat were once successful might becomehindrances. (See the sidebar "The PotentialPitfalls of New Technologies.")

Consider the major changes that BMWrecently underwent. Only a few years ago,experimenting with novel design concepts -to make cars withstand crashes better, forinstance - required expensive physical pro-totypes to be built. Because that processtook months, it acted as a barrier to innova-tion because engineers could not get timelyfeedback on their ideas. Furthermore, datafrom crash tests arrived too late to signifi-cantly influence decisions in the early stagesof product development. So BMW had toincorporate the information far down-stream, incurring greater costs. Neverthe-less, BMW's R&D organization, structuredaround this traditional system, developedaward-winning automobiles, cementing thecompany's reputation as an industry leader.But its success also made change difficult.

The Essentials forEnlightened Experimentation

New technologies such as computer simulations not only make experi-mentation faster and cheaper, they also enable companies to be moreinnovative. But achieving that requires a thorough understanding ofthe link between experimentation and learning. Briefly stated, innova-tion requiresthe right R&D systems for performing experiments thatwill generate the information needed to develop and refine productsquickly. The challenges are managerial as well as technical:

1) Organize for rapid experimentation• Examine and, if necessary, revamp entrenched routines,organizational boundaries, and incentives to encourage rapidexperimentation.

•Consider using small development groups that contain keypeople (designers, test engineers, manufacturing engineers)with all the knowledge required to iterate rapidly.

• Determine what experiments can be performed in parallelinstead of sequentially. Parallel experiments are most effectivewhen time matters most, cost is not an overriding factor, anddevelopers expect to learn little that would guide them inplanning the next round of experiments.

2) Fail early and often, but avoid mistakes• Embrace failures that occur early in the development processand advance knowledge significantly.

• Don't forget the basics of experi mentation. Well-designed testshave clear objectives (what do you anticipate learning?) andhypotheses (what do you expect to happen?). Also, mistakes oftenoccur when you don't control variables that could diminish yourability to learn from the experiments. When variability can't becontrolled, allow for multiple, repeated trials.

3) Anticipate and exploit early information• Recognize the full value of front-loading: identifying problems

upstream, where they are easier and cheaper to solve.

• Acknowledge the trade-off between cost and fideliiy. Experimentsof lower fidelity (generally costing less) are best suited in theearly exploratory stages of developing a product High-fidelityexperiments (typically more expensive) are best suited later toverify the product.

4) Combine new and traditional technologies• Do not assume that a new technology will necessarily replace

an established one. Usually, new and traditional technologiesare best used in concert.

• Remember that new technologies emerge and evolve continu-ally. Today's new technology might eventually replace itstraditional counterpart, but it could then be challenged bytomorrow's new technology.

FEBRUARY 2001 69


Today, thanks to virtual experiments-crashes simu-lated by a high-performance computer rather thanthrough physical prototypes-some of the informationarrives very early, before BMW has made major resourcedecisions. The costs of experimentation (both financialand time) are therefore lower because BMW eliminatesthe creation of physical prototypes as well as the expenseof potentially reworking bad designs after the companyhas committed itself to them. (Physical prototypes arestill required much further downstream to verify the finaldesigns and meet safety regulations.) In addition, the

rapid feedback and the ability to see and manipulate high-quality computer images spur greater innovation: manydesign possibilities can be explored in "real time" yet vir-tually, in rapid iterations.

To study this new technology's impact on innovation,BMW performed the following experiment. Several de-signers, a simulation engineer, and a test engineer formeda team to improve the side-impact safety of cars. Primar-ily using computer simulations, the team developed andtested new ideas that resulted from their frequent brain-storming meetings.

The Potential Pitfalls of New TechnologiesNew technologies can stash the costs(both financial and time) of experimen-tation and dramatically increase acompany's ability to develop innova-tive products. To reap those benefits,though, organizations must preparethemselves for the full effects of suchtechnologies.

Computer simulations and rapidprototyping, for example, increasenot only a company's capacity to con-duct experiments but also the wealthof information generated by thosetests. That, however, can easily over-load an organization that lacksthe capability to process infor-mation from each round of ex-periments quickly enough to in-corporate it into the next round.In such cases, the result is waste,confusion, and frustration. Inother words, without carefuland thorough planning, a newtechnology might not onlyfail to deliver on its promiseof lower cost, increased speed,and greater innovation, itcouidactually decrease the overallperformance of an R&D organi-zation, or at a minimum dis-rupt its operations.

Misaligned objectives areanother common problem.Specifically, some managers donot fully appreciate the trade-off

between response time and resourceutilization. Consider what happenswhen companies establish centraldepartments to oversee computingresources for performing simulations.Clearly, testing ideas and concepts vir-tually can provide developers with therapid feedback they need to shape newproducts. At the same time, computersare costly, so people managing themas cost centers are evaluated by howmuch those resources are being used.

The busier a central computer is,however, the longer it takes for devel-

Waitingfor a Resource

According to queuing theory, the waiting time fora resource such as a central mainframe computerincreases gradually as more of the resource isused. But wben the utilization passes 70%, delaysincrease dramatically

40 50 60 70 80 90 100

Percent of Resource Utilization

opers to get the feedback they need.In fact, the relationship between wait-ing time and utilization is not linear-queuing theory has shown that thewaiting time typically increases gradu-ally until a resource is utilized around70%, and then the length of the delayssurge. (See the exhibit "Waiting for aResource.") An organization tryingto shave costs may become a victimof its own myopic objective. That is,an annual savings of perhaps a fewhundred thousand dollars achievedthrough increasing utilization from

70% to 90% may lead to verylong delays for dozens ofdevelopment engineers wait-ing for critical feedback fromtheir tests.

A huge negative conse-quence is that the excessivedelays not only affect develop-ment schedules but also dis-courage people from experi-menting,thus squelching theirability to innovate. So in thelong term, running additionalcomputer equipment at a lowerutilization level might wellbe worth the investment. Analternative solution is to movethose resources away from costcenters and under the controlof developers, who have strongincentives for fast feedback.



Because all the knowledge required about safety, de-sign, simulation, and testing resided within a small group,the team was able to iterate experiments and developsolutions rapidly. After each round of simulated crashes,the team analyzed the results and developed new ideasfor the next round of experiments. As expected, the teambenefited greatly from the rapid feedback: it took themonly a few days to accept, refine, or reject new design so-lutions-something that had once taken months.

As the trials accrued, the group members greatly in-creased their knowledge of the underlying mechanics,which enabled them to design previously unimaginableexperiments. In fact, one test completely changed theirknowledge about the complex relationship between ma-terial strength and safety. Specifically, BMW's engineershad assumed that the stronger the area next to the bot-tom of a car's pillars (the structures that connect the roofof an auto to its chassis), the better the vehicle wouldbe able to withstand crashes. But one member of the de-velopment team insisted on verifying this assumptionthrough an inexpensive computer simulation.

The results shocked the team: strengthening a particu-lar area below one of the pillars substantially decreasedthe vehicle's crashworthiness. After more experiments andcareful analysis, the engineers discovered that strength-ening the lower part of the center pillar would make thepillar prone to folding higher up, above tbe strengthenedarea. Thus, the passenger compartment would be morepenetrable at the part of the car closer to the midsection,chest, and head of passengers. The solution was toweaken, not strengthen, the lower area. This counterin-tuitive knowledge-that purposely weakening a part ofa car's structure could increase the vehicle's safety-hasled BMW to reevaluate ail the reinforced areas of itsvehicles.

In summary, this small team increased the side-impactcrash safety by about 30%. It is worth noting that two crashtests of physical prototypes at the end of the project con-firmed the simulation results. It should also be noted thatthe physical prototypes cost a total of about $3(Xt,ooo,which was more tban the cost of all 91 virtual crashescombined. Furthermore, the physical prototypes tooklonger to build, prepare, and test than the entire series ofvirtual crashes.

But to obtain the full benefits of simulation technolo-gies, BMW had to undertake sweeping changes in process,organization, and attitude-changes that took severalyears to accomplish. Not only did the company have to re-organize the way different groups worked together; it alsohad to change habits that had worked so well in the oldsequential development process.

Previously, for example, engineers were often loath torelease less-than-perfect data. To some extent, it was ineach group's interest to hold back and monitor the outputfrom other groups. After all, the group that submitted its

information to a central database first would quite likelyhave to make the most changes because it would havegoften tbe least feedback from other areas. So, for in-stance, the door development team at BMW was accus-tomed to-and rewarded for-releasing nearly flawlessdata (details about the material strength of a proposeddoor, for example), which could take many months togenerate. Tbe idea of releasing rough information veryearly, an integral part of a rapid and parallel experimen-tation process, was unthinkable-and not built into theincentive system. Yet a six-month delay while data werebeing perfected could derail a development programpredicated on rapid iterations.

Tbus, to encourage the early sharing of information,BMW's managers had to ensure that each group under-stood and appreciated the needs of other teams. Thecrash simulation group, for example, needed to make thedoor designers aware of the information it required inorder to build rough models for early-stage crash simula-tions. That transfer of knowledge had a ripple effect,changing how the door designers worked because someof the requested information demanded that they payclose attention to the needs of other groups as well. Theystarted to understand that withholding information aslong as possible was counterproductive. By making thesekinds of organizational changes, BMW in Germany sig-nificantly slashed development time and costs and boostedinnovation.

2. Fail early and often, but avoid mistakes. Experi-menting with many diverse-and sometimes seeminglyabsurd - ideas is crucial to innovation. When a novel con-cept fails in an experiment, the failure can expose impor-tant gaps in knowledge. Such experiments are particu-larly desirable when they are performed early on so thatunfavorable options can be eliminated quickly and peo-ple can refocus their efforts on more promising alterna-tives. Building the capacity for rapid experimentation inearly development means rethinking the role of failurein organizations. Positive failure requires having a thickskin, says David Kelley, founder of IDEO, a leading designfirm in Palo Alto, California.

IDEO encourages its designers "to fail often to succeedsooner," and the company understands that more radicalexperiments frequently lead to more spectacular failures.Indeed, IDEO has developed numerous prototypes thathave bordered on the ridiculous (and were later rejected),such as shoes with toy figurines on the shoelaces. At thesame time, IDEO's approach has led to a host of best-sellers, such as the Palm V handheld computer, which hasmade the company tbe subject of intense media interest,including a Nightline segment with Ted Koppel and cov-erage in Serious Play, a book by Michael Schrage, a co-director of the e-markets initiative at the MIT Media Lab,that describes the crucial importance of allowing innova-tors to play with prototypes.

FEBRUARY 2001 71


Removing the stigma of failure, though, usually re-quires overcoming ingrained attitudes. People who fall inexperiments are often viewed as incompetent, and thatattitude can lead to counterproductive behavior. As Kelleypoints out, developers who are afraid of failing and look-ing bad to management will sometimes huild expensive,sleek prototypes that they become committed to beforethey know any of the answers. In other words, the sleekprototype might look impressive, but it presents the falseimpression that the product is farther along than it reallyis, and that perception subtly discourages people fromchanging the design even though better alternativesmight exist. That's why IDEO advocates the developmentof cheap, rough prototypes that people are invited to crit-icize-a process that eventually leads to better products."You have to have the guts to create a straw man," assertsKelley.

To foster a culture in which people aren't afraid of fail-ing, IDEO has created a playroomlike atmosphere. OnMondays, the different branches hold show-and-tells inwhich employees display and talkabout their latest ideas and products.IDEO also maintains a giant "tech box"of hundreds of gadgets and curiositiesthat designers routinely rummagethrough, seeking inspiration amongthe switches, buttons, and various oddmaterials and objects. And brain-storming sessions, in which wild ideasare encouraged and participants deferjudgment to avoid damping the dis-cussion, are a staple of the differentproject groups.

3M is another company with ahealthy attitude toward failure. 3M'sproduct groups often have skunk-works teams that investigate the op-portunities (or difficulties) that a po-tential product might pose. The teams,consisting primarily of technical peo-ple, including manufacturing engi-neers, face little repercussion if an ideaflops-indeed, sometimes a failure iscause for celebration. When a teamdiscovers that a potential productdoesn't work, the group quickly dis-bands and its members move on toother projects.

Failures, however, should not beconfused with mistakes. Mistakes pro-duce little new or useful informationand are therefore without value. Apoorly planned or badly conducted ex-periment, for instance, might result inambiguous data, forcing researchers

to repeat the experiment. Another common mistake is re-peating a prior failure or being unable to learn from thatexperience. Unfortunately, even the best organizationsoften lack the management systems necessary to care-fully distinguish between failures and mistakes.

3. Anticipate and exploit early information. Whenimportant projects fail late in the game, the consequencescan be devastating. In the pharmaceutical industry, for ex-ample, more than 80% of drug candidates are discontin-ued during the clinical development phases, where morethan half of total project expenses can be incurred. Yet al-though companies are often forced to spend millions ofdollars to correct problems in the later stages of productdevelopment, they generally underestimate the cost sav-ings of early prohlem solving. Studies of software devel-opment, for instance, have shown that late-stage prob-lems are more than loo times as costly as early-stage ones.For other environments that involve large capital invest-ments in production equipment, the increase in cost canbe orders of magnitude higher.

The Benefits ofFront-Loaded DevelopmentIn the 1990S,Toyota made a major push to accelerate its product developnnent

cycle. The objective was to shorten the time from the approval of a body style to

the first retail sales, thereby increasing the likelihood that Toyota kept up with

the rapidly changing tastes of consumers.

Toyota made a concerted effort to identify and solve design-related problems

earlier in product development-a concept known affront-loading. To accom-

plish that, the company implemented a numberof initiatives, such as involving

more manufacturing engineers during the product-engineering stage, increas-

ing the transfer of knowledge between projects, investing substantially in com-

puter-aided design and engineering tools, and developing rapid-prototyping

capabilities.

To measure the benefits of these initlatives-and to monitor the company's

evolving capabilities for early problem solving-Toyota tracked problems over

multiple development projects. (See the exhibit "Solving Problems Earlier")

The knowledge that a higher percentageof problems were being solved at

earlier stages reassured Toyota's managers that they could aggressively reduce

both development time and cost without risking product quality. In particular,

between the first and third front-loading initiatives, Toyota slashed the cost

{including the number of full physical prototypes needed) and time of develop-

ment by between 30% and 40%.

It should be noted that in the early 1990s Toyota substantially reorganized

its development activities, resulting in more effective communication and

coordination between the different groups. This change most likely accounted

for some of the performance improvements observed, particularly during the

first front-loading initiatives.



In addition to financial costs, companies need to con-sider the value of time when those late-stage problems areon a project's critical path-as they often are. In pharma-ceuticals, shaving six months off drug development meanseffectively extending patent protection when the producthits the market. Similarly, an electronics company mighteasily find that six months account for a quarter of a prod-uct's life cycle and a third of all profits.

New technologies, then, can provide some of theirgreatest leverage by identifying and solving problems up-stream -best described asfront-baded development. In theautomotive industry, for example,"quick-and-dirty"crashsimulations on a computer can help companies avoidpotential safety problems downstream. Such simulationsmay not be as complete or as perfect as late-stage proto-types will be, but they can force organizational problemsolving and communication at a time when many down-stream groups are not participating directly in devel-opment. (See the sidebar "The Benefits of Front-LoadedDevelopment.")

Several years ago, Chrysler (now DaimlerChrysler) dis-covered the power of three-dimensional computer mod-els, known internally as digital mock-ups, for identifyingcertain problems in early development stages. WhenChrysler developed the 1993 Concorde and Dodge In-trepid models, the process of decking - placing the powertrain and related components like the exhaust and sus-pension in the prototype automobile-took more thanthree weeks and required many attempts before the pow-ertrain could be inserted successfully. By contrast, theearly use of digital mock-ups in the 1998 Concorde and In-trepid models allowed the company to simulate deckingto identify (and solve) numerous interference problemsbefore the physical decking took place. Instead of takingweeks, decking was completed in 15 minutes because allobstruction problems had been resolved earlier-when itwas relatively inexpensive and fast to do so.

Of course, it is neither pragmatic nor economically fea-sible for companies to obtain all the early informationthey would like. So IDEO follows the principle of three

Solving Problems Earlier

01

0^ 6 0E<u 40

* 0

- ^

S2 S3 S4 S5

Stages of Development Process

S2 S3 S4 S5


S8

S8

o

EV 40

e 20

S3 S4 SS


Source: Stefan Thomke and Takahiro Fujimoto, "The Effect of Front-Loading'Problem-Solving on ProductDevelopment Performance," Jhe journal of Product Innovation Management, Vol. 17, No. 2, March 2000

As Toyota intensified its front-hading efforts, it wasable to identify and solve problems much earlier inthe development process.

In the early 1990s (see top graph), the first initia-tives for front-load ing began. Formal, systematicefforts to improve face-to-face communication andjoint problem solving between the prototype shopsand production engineers resulted in a higherrelative percentage of problems found with the aidof first prototypes. Communication between differentengineering sections (for instance, between body,engine, and electrical) also improved.

In the mid-1990s (see middle graph), the secondfront-loading Initiatives called for three-dimensionalcomputer-aided design, resulting in a significantincrease of problem identificatior) and solving priorto stage 3 (first prototypes).

In the ongoing third front-loading initiatives(see bottom graph), Toyota is using computer-aidedengineering to identify functional problems earlierIn the development process, and the company istransferring problem and solution information fromprevious projects to the front end of new projects.As a result, Toyota expects to solve at least 80% of allproblems by stage 2 - that Is, before the first proto-types are made. And because the second-generationprototypes (stage 5) are now less important to over-all problem solving, Toyota will be able to eliminateparts of that process, thereby furtber reducing timeand cost without affecting product quality.

FEBRUARY 2001 73


R's: rough, rapid, and right. The final R recognizes thatearly prototypes may be incomplete but can still get spe-cific aspects of a product right. For example, to design atelephone receiver, an IDEO team carved dozens of piecesof foam and cradled them between their heads and shoul-ders to find the best possible shape for a handset. Whileincomplete as a telephone, the model focused on gettingioo% of the shape right. Perhaps the main advantage ofthis approach is that it forces people to decide judiciouslywhich factors can initially be rough and which must beright. With its three R's, IDEO has established a processthat generates important information when it is mostvaluable: the early stages of development.

In addition to saving time and money, exploiting earlyinformation helps product developers keep up with cus-tomer preferences that might evolve over the course ofa project. As many companies can attest, customers willoften say about a finished product: "This is exactly whatI asked you to develop, but it is not what I want." Leadingsoftware businesses typically show incomplete prototypesto customers in so-called beta tests, and through that pro-cess they often discover changes and problems when theyare still fairly inexpensive to handle.

4. Combine new and traditional technologies. Newtechnologies that are used in the innovationprocess itself are designed to help solve prob-lems as part of an experimentation system. Acompany must therefore understand how touse and manage new and traditional tech-nologies together so that they complementeach other. In fact, research by Marco Iansitiof Harvard Business School has found that, inmany industries, the ability to integrate tech-nologies is crucial to developing superiorproducts.

A new technology often reaches the samegeneral performance of its traditional coun-terpart much more quickly and at a lowercost. But the new technology usually per-forms at only 70% to 80% of the establishedtechnology. For example, a new chemical syn-thesis process might be able to obtain a puritylevel that is just three-quarters that of a ma-ture technique. Thus, by combining new andestablished technologies, organizations canavoid the performance gap while also enjoy-ing the benefits of cheaper and faster experi-mentation. (See the exhibit "Combining theNew with the Traditional.")

Indeed, the true potential of new tech-nologies lies in a company's ability to recon-figure its processes and organization to usethem in concert with traditional technolo-gies. Eventually, a new technology can re-place its traditional counterpart, but it then

might be challenged by a newer technology that must beintegrated. To understand this complex evolution, con-sider what has happened in the pharmaceutical industry.

In the late nineteenth century and for much of thetwentieth century, drug development occurred through aprocess of systematic trial-and-error experiments. Scien-tists would start witb little or no knowledge about a par-ticular disease and try out numerous molecules, manyfrom their company's chemical libraries, until they foundone that happened to work. Drugs can be likened to keysthat need to fit the locks of targets, such as the specificnerve cell receptors associated with central nervous dis-eases. Metaphorically, then, chemists were once blind, orat least semiblind, locksmiths who have had to make upthousands of different keys to find the one that matched.Doing so entailed synthesizing compounds, one at a time,each of which usually required several days at a cost from$5,000 to $10,000.

Typically, for each successful drug that makes it to mar-ket, a company investigates roughly 10,000 starting can-didates. Of those, only 1,000 compounds make it to moreextensive trials in vitro (that is, outside living organisms insettings such as test tubes), 20 of which are tested evenmore extensively in vivo (that is, in the body of a living or-

Combining the New with the Traditional

A new technology (blue curve) wilt reach perhaps just 70% to 80%of the performance of an established technology (red curve). A newcomputer model, for instance, might be able to represent real-worldfunctionality that is just three-quarters that of an advanced prototypemodel. To avoid this performance gap - and potentially create newopportunities for innovation - companies can use tbe new and tradi-tional technologies in concert (dotted red curve). Tbe optimal timefor switching between tbe two occurs wben tbe rates of Improvementbetween tbe new and mature technologies are about tbe same - thatis, when the slopes of tbe two curves are equal.

a V;-'

CombinedTraditional and New Potentialfor

. _ •Increased Innovation

4 PerformanceiCap

Savings from CombiningTraditional and New Technologies

Effort (Elapsed Time, Cost)



ganism such as a mouse), and ten of which make it to clin-ical trials with humans. The entire process represents along and costly commitment.

But in the last ten years, new technologies have signif-icantly increased the efficiency and speed at which com-panies can generate and screen chemical compounds. Re-searchers no longer need to painstakingly create onecompound at a time. Instead, they can use combinatorialchemistry, quickly generating numerous variations si-multaneously around a few building blocks, just as today'slocksmiths can make thousands of keys from a dozenbasic shapes, tbereby reducing the cost of a compoundfrom thousands of dollars to a few dollars or less.

In practice, however, combinatorial chemistry has dis-rupted well-established routines in laboratories. For onething, the rapid synthesis of drugs has led to a new prob-lem: how to screen those compounds quickly. Tradition-ally, potential drugs were tested in live animals-an ac-tivity fi-aught with logistical difficulties, high expense, andconsiderable statistical variation.

So laboratories developed test-tube-based screeningmethodologies that could be automated. Called high-throughput screening, this technology requires significantinnovations in equipment (such as high-speed precisionrobotics) and in the screening process itself to let re-searchers conduct a series of biological tests, or assays, onmembers of a chemical library virtually simultaneously.

The large pharmaceutical corporations and academicchemistry departments initially greeted such "combi-chem" technologies (combinatorial chemistry and high-throughput screening) with skepticism. Among tbe rea-sons cited was that the purity of compounds generated viacombichem was relatively poor compared to traditionalsynthetic chemistry. As a result, many advances in thetechnology were made by small biotechnology companies.

But as the technology matured, it caught the interest oflarge corporations like Eli Lilly, which in 1994 acquiredSphinx Pharmaceuticals, one of the start-ups developingcombichem. Eli Lilly took a few years to transfer the newtechnologies to its drug discovery division, which usedtraditional synthesis. To overcome the internal resistance,senior management implemented various mechanismsto control how the new technologies were being adopted.For example, it temporarily limited the in-house screen-ing available to chemists, leaving them no choice but touse some of the high-throughput screening capabilitiesat the Sphinx subsidiary and interact with the staff there.

Until now, pharmaceutical giants like EH Lilly haveused combinatorial chemistry primarily to optimizepromising new drug candidates that resulted from an ex-haustive search through chemical libraries and other tra-ditional sources. But as combinatorial chemistry itselfadvances and achieves levels of purity and diversity com-parable to the compounds in a library, companies will in-creasingly use it at the earlier phases of drug discovery. In

fact, ail major pharmaceutical companies have had to usecombichem and traditional synthesis in concert, and thecompanies that are best able to manage the new and ma-ture technologies together so that they fully complementeach other will have the greatest opportunity to achievethe highest gains in productivity and innovation.

Enlightened ImplicationsNew technologies reduce the cost and time of experi-mentation, allowing companies to be more innovative.Automotive companies, for example, are currently ad-vancing the performance of sophisticated safety systemsthat measure a passenger's position, weight, and height toadjust the force and speed at which airbags deploy. Theavailability of fast and inexpensive simulation enables themassive and rapid experimentation necessary to developsuch complex safety devices.

But it is important to note that the increased automa-tion of routine experiments will not remove the humanelement in innovation. On the contrary, it will allow peo-ple to focus on areas where their value is greatest: gener-ating novel ideas and concepts, learning from experi-ments, and ultimately making decisions that requirejudgment. For example, although Millennium's R&D fa-cilities look more and more like factories, the value ofknowledge workers has actually increased. Instead of car-rying out routine laboratory experiments, they now focuson the early stages (determining which experiments toconduct, for instance) and making sense of the informa-tion generated by the experimentation.

The implications for industries are enormous. The elec-tronic spreadsheet has already revolutionized financialproblem solving by driving down the marginal cost of fi-nancial experimentation to nearly zero; even a small start-up can perform complex cash-fiow analyses on an inex-pensive PC. Similarly, computer simulation and othertechnologies have enabled small businesses and individ-uals to rapidly experiment with novel designs of cus-tomized integrated circuits. The result has been a massivewave of innovation, ranging from smart toys to electronicdevices. Previously, the high cost of integrated-circuit cus-tomization made such experimentation economical toonly the largest companies.

Perhaps, though, this era of enlightened experimenta-tion is still in its bare infancy. Indeed, the ultimate tech-nology for rapid experimentation might turn out to bethe Intemet, which is already turning countless users intofervent innovators. 9

Reprint R0102DTo order reprints, see the last page of Executive Summaries.

To further explore the topic of this article, go towww.hbr.org/explore.

FEBRUARY 2001