J. Eng. Technol. Manage. 21 (2004) 237–244

Book reviews

Experimentation matters: unlocking the potential of new technologies for innovationStefan H. Thomke, Harvard Business School Press, Boston, 2003, 307 pp.

While there are many recent texts covering the management of innovation, Stefan H. Thomkeengages the topic with a sharp focus on one particular innovation practice: experimentation.In setting out the place of experimentation, he states that “. . . this book starts from theperspective that experimentation is part and parcel of innovation—it matters” (p. 4). Butwhile experimentation provides opportunities for innovation, we are also forewarned that“ . . . the book is also cautionary, to the extent that new technologies for experimentationpost challenges for the organization of innovation” (p. 4). This balanced perspective ofboth opportunity and challenge serves as a consistent theme throughoutExperimentationMattersand informs the innovation practices presented.

A focus on experimentation defines the scope of this text within the wider spectrum ofstudies of the exploratory search processes central to innovation. As such, it adds to priorinvestigations on the role of prototyping and simulation (Schrage, 1999), and it comple-ments and acknowledges other work within the broader field of innovation management,such as research on search and recombination (Hargadon, 2003), managing technologi-cal change (Christensen, 1997), and differentiating between routine and innovative work(Sutton, 2002). In its initial cautionary note and subsequent treatment of the impact of newtechnologies within organizations, the text follows in the tradition ofBurns and Stalker(2001)who noted that “Technical progress and organizational development are aspects ofone and the same trend in human affairs; and the persons who work to make these processesactual are also their victims” (p. 19). While employing new technologies for experimenta-tion is indeed fraught with challenges, the book sets out to help managers avoid becomingsuch victims by providing principles through which they may be able make best use of theopportunity provided by these methods.

The book is aimed primarily at practicing managers. Indeed, with the use ofrich examples from diverse industries—including banking, automotive, semiconductor,and yacht design—the lessons will be accessible to professionals from a variety of fields.But as the author is the contributor of many of the scholarly articles that make up theunderlying fabric of this book, and with its comprehensive endnotes and associated ref-erences, the text will be of interest to academics as well. The organization of the bookitself is not prone to experimentation; instead, it effectively follows a decomposition ofthe two-part title, with the first half exploring experimentation itself—its opportunities andchallenges—and the second half describing principles for managing experimentation usingnew technologies.

0923-4748/$ – see front matter © 2004 Elsevier B.V. All rights reserved.

238 Book reviews

Experimentation in various forms has always mattered, but we now have the opportunity todo so much more of it. As Thomke describes, advances in such areas as computer simulation,rapid prototyping, and—as in the pharmaceutical industry—combinatorial technologies,have led to order of magnitude increases in experimental results. For example, advances invehicle crash simulation at BMW have improved model complexity by over 100 times whilereducing run-times from 3 months to 30 h. In this context, over ninety crash simulationscan be performed for less than the cost of two physical crashes. But new tools not onlyallow the same things to be done more quickly and cheaply but also allow new ways ofworking by permitting “what if” scenarios not feasible by other means. For example, throughsuch methods, BMW engineers made a counterintuitive discovery that they could increaseoverall crash protection by actually weakening certain body structures. New opportunitiesfor learning and innovation are thus opened up through such tools.

The ability to run fast and frequent experiments is not particularly helpful unless some-thing can be learned from outcomes. But the ability to learn within organizations is achallenge, and Thomke identifies various factors affecting learning by experimentation.Edison, for example, understood the importance of rapid feedback for ongoing learning,designing the layout of his laboratory expressly to facilitate quick information flow betweenthe various sectors of his laboratory. In addition to the challenge of efficient learning, theintroduction of technology brings with it new issues for organizations, including limita-tions in how the technology is used in practice, the complexity of organizational interfaces,and the slower pace of change of individual behaviors. Managing in the context of suchchallenges is the focus of the second half of the book.

Six principles for managing experimentation are divided between two chapters under thetwo themes “experimenting early and often” and “organizing for rapid iteration.” By exper-imenting early and often, managers attempt to gather information when it is least expensive,while increasing the total amount of information to aid decision-making. The first principlewithin this theme is to anticipate and exploit early information through front-loaded innova-tion processes, employing more resources earlier. Examples include Toyota’s front-loadeddevelopment process, with its focus on early manufacturing involvement and rapid proto-typing practices, and Boeing’s development of software to allow early interference-testingbetween design groups.

But what happens under increased frequency of experimentation? More data are gener-ated, so much so that if the organization is not prepared, productivity could actually decrease.An example provided is that of a pharmaceutical firm that was faced with the challenge ofadapting to combinatorial-chemistry methods which enabled far more tests to be conductedthan through traditional methods. The second principle, to experiment frequently withoutoverloading the organization, advises adopting practices to aid in information absorption,such as rapid information transfer, quick-decision making, and the development of newtools to manage data.

It may be tempting to cast off old experimentation technologies completely, but lessonsfrom the automotive, semiconductor, and pharmaceutical industries suggest otherwise. It isfound that those companies that manage to integrate new and traditional technologies, thefocus of the third principle, often exhibited the most efficient development processes. Suchintegration can take the form of what Thomke refers to as a technology platform, a suite ofnew and existing experimentation practices viewed as a complete system.

Book reviews 239

The chapter on organizing for rapid iteration focuses on rethinking the role of incen-tives and rewards, encouraging practices associated with whatSchrage (1999)referred toas a “prototype culture”—through a tolerance for failure and the promotion of values thatencourage experimentation. Traditional incentives that reward precision and punish failurediscourage the behavior needed to exploit opportunities available from rough and rapid earlysimulation. Two principles—to organize for rapid innovation; and to fail early and oftenwhile avoiding mistakes—focus on aligning incentives to support ongoing experimentation.Furthermore, an organization adept at experimentation for product development may thenconsider a final principle, to manage whole projects as experiments. Thomke offers an inte-grating example in the case of IDEO Product Development, where not only product devel-opment but also the process by which products are developed are managed as experiments.

Experimentation is typically associated with activity wholly within an organization, butthe final chapter investigates “shifting the locus of experimentation,” where aspects of theexperimentation cycle—design, build, run, and analyze—are executed outside the firm.Drawing from examples in the semiconductor and flavoring industries, Thomke demon-strates how the development of customer “toolkits” for experimentation not only enablenew means of customer-driven experimentation but also create challenging new questionsfor firms. If customers are enabled to experiment to determine the exact product they want,what value does the firm provide in such a distributed development process? Not all prod-ucts will lend themselves to the toolkit approach. For those that do, this section will promptmanagers to consider the opportunities and implications of such shifts. I had at first ex-pected this section might also deal with topics related to a broader exploration outside thefirm, such as learning from competitor design failures or from market probing. But as thedegree of new technology application is more limited in these domains, Thomke is to beapplauded for navigating a tight course while acknowledging the wider issues of learningthrough exploration and failure.

The book benefits from a wealth of illustrative examples, many of which are based onpublished case studies. But there is naturally the danger of focusing on selective caseswith idiosyncratic (and mostly positive) outcomes. In one example, BMW is cited as usingthe product development of its flagship 7-Series car as a test project for an entirely newdevelopment process as well. While the project, viewed as dangerous and high-stakes, wasconsidered a success for both product and process, it is not clear that such an approach wouldbe advised across contexts. Indeed, the practice appears to violate the concept of focusingexperimentation where costs (and risks) are lowest and feedback is early. Significantly, suchexamples are augmented throughout the text with comprehensive results from industry-wideresearch, most notably global automotive studies and the author’s extensive investigationswithin the semiconductor industry.

Is increased experimentation for everyone? Studies within the computer industry demon-strated how effective strategies for accelerating product development differ within variousmarket environments (Eisenhardt and Tabrizi, 1995), andExperimentation Mattersshouldbe read with an appreciation that the level of experimentation will have to take into accountspecific firm, industry, and market context. We are provided in the text with some heuristicsto guide the limits of experimentation, but, in the end, each organization will have to treatthe level of its experimentation as an experiment itself—something to be planned, analyzed,and learned from.

240 Book reviews

“A relentless organizational focus on success makes true experimentation all too rare”(p. 2). By elucidating in detail what experimentation entails and how it may best be usedby organizations to promote innovation, Thomke has provided a thorough and thoughtfulguide to correct such imbalance. The book is particularly effective in its finely-honed focus,in the scholarly and well-referenced means by which it covers the topic, and in drawingfrom rich examples across a wide spectrum of industries. In its introduction, we are toldthat: “Like any effective experiment, this book will reveal what does and does not work”(p. 2). And in doing so, it is a great success.

References

Burns, T., Stalker, G.M., 1961/2001. The Management of Innovation, third ed., Oxford University Press, Oxford.Eisenhardt, K.M., Tabrizi, M., 1995. Accelerating adaptive processes: product innovation in the global computer

industry. Administr. Sci. Quart. 40, 84–110.Hargadon, A., 2003. How Breakthroughs Happen: the Surprising Truth About How Companies Innovate. Harvard

Business School Press, Boston.Schrage, M., 1999. Serious Play: How the World’s Best Companies Simulate to Innovate. Harvard Business School

Press, Boston.Sutton, R.I., 2002. Weird Ideas That Work: 11 1/2 Practices for Promoting, Managing, and Sustaining Innovation.

The Free Press, New York.

Victor SeidelFaculty of Management, Sa¨ıd Business School

University of Oxford, Oxford OX1 1HP, UKE-mail address:victor.seidel@sbs.ox.ac.uk (V. Seidel)

doi: 10.1016/j.jengtecman.2004.05.002

Open Innovation: The New Imperative for Creating and Profiting from TechnologyHenry W. Chesbrough, Harvard Business School Press, Boston, MA, 2003, 227 pages

The objective of this book is to change the thinking of major companies that run researchand development (R&D) centers by “innovating innovation.” The author states that in thepast, R&D used to be an activity that was conducted in a corporate research lab with littlerelation to the overall business needs of the company. Researchers were given a long leashwith which to pursue their own interests and invent things that might benefit the company.If something was invented that could be developed into a salable product, the researchdepartment was successful. However, if the output of the research department is never usedby the company it has one of two fates: it becomes warehoused in the Intellectual Propertyportfolio of the company, or it dies and benefits nobody.

The target audiences ofOpen Innovationare high-level corporate executives, heads of theresearch departments in the type of companies profiled in the book and small technologycompanies that might possibly have something to offer to larger companies. The latter,probably wouldn’t get as much out of the book unless they were savvy about marketingtheir company to a larger firm that would be interested in acquisition of technology. The