The mixed effects of inconsistency on experimentation in organizations

7/27/2019 The mixed effects of inconsistency on experimentation in organizations

1/17

Organization ScienceVol. 15, No. 3, MayJune 2004, pp. 310326

issn1047-7039 eissn1526-5455 04 1503 0310

informs

doi 10.1287/orsc.1040.0076

2004 INFORMS

The Mixed Effects of Inconsistency onExperimentation in Organizations

Fiona LeeDepartment of Psychology, and School of Business, University of Michigan, 525 East University,

Ann Arbor, Michigan 48109-1109, [email protected]

Amy C. Edmondson, Stefan ThomkeHarvard Business School, Boston, Massachusetts 02163 {[email protected], [email protected]}

Monica WorlineGoizueta Business School, Emory University, Atlanta, Georgia 30322, monica [email protected]

This paper examines how the inconsistency of organizational conditions affects peoples willingness to engage in exper-imentation, a behavior integral to innovation. Because failures are inevitable in the experimentation process, we arguethat conditions giving rise to psychological safety reduce fear of failure and promote experimentation. Based on this reason-ing, we suggest that inconsistent organizational conditionswhen some support experimentation and others do notinhibitexperimentation behaviors. An exploratory study in the field, followed by a laboratory experiment, found that individu-als under high evaluative pressure were less likely to experiment when normative values and instrumental rewards wereinconsistent in supporting experimentation. In contrast, individuals under low evaluative pressure responded to inconsistentconditions with increased experimentation. Our results suggest that evaluative pressure fundamentally alters an individualsexperience of and response to uncertainty and that understanding experimentation behavior requires examining effects ofmultiple organizational conditions in combination.

Key words: experimentation; inconsistency; evaluative pressure

Innovation is a kind of Holy Grail in managementan elusive goal to be pursued continuously. Yet, encour-aging innovation and superb execution of routine workat the same time can be difficult. For example, in2000, Bank of America decided to become an indus-try leader in innovation and established a program topromote innovation in two-dozen real-life laborato-ries (Thomke and Nimgade 2002). Laboratories, in thiscase, referred to fully operating retail bank branches inwhich employees were to experiment with new productand service concepts, such as virtual tellers. Successfulexperimentsdetermined on the basis of consumer sat-isfaction or revenue growthwere to be recommendedfor a national rollout.

Bank senior management voiced strong support forinnovation and explicitly recognized and communicatedthat experimentation with new ideas necessarily pro-duced failures along the way. Indeed, a failure rate of30% was targeted as indicative of sufficient risk takingand novelty. Initially, however, employee compensationcontinued to be based on measures of routine perfor-mance (such as opening new customer accounts). Theespoused goal of increasing innovation thus was incon-sistent with the reward system; individuals compensa-tion could suffer from time spent experimenting withnew ideas or from failed experiments. Understandably,many employees were reluctant to experiment much

until management made changes to align reward systemswith the organizations new value of experimentation.

The aim of this paper is to examine the effects ofinconsistency in organizational conditions, such as whatoccurred at Bank of America, on the individual-levelbehaviors necessary for innovation. Current researchexamining antecedents of behaviors such as learning,creativity, and experimentation has emphasized maineffects of single organizational variables. Implicit in thisapproach is the idea that changing one organizationalcondition can lead to improvement in behaviors integralto innovation. We argue, in contrast, that changing asingle organizational condition without changing otherscreates inconsistency that instead may inhibit innova-tion behaviors. This paper theorizes and reports on two

studies that explore effects of inconsistent organizationalconditions on experimentation behaviors.

Experimentation: Processes and Antecedents

Defining Experimentation. Experimentation is a trial-and-error process in which each trial generates newinsights on a problem (Allen 1977, Thomke 1998).Learning by experimentation is fundamental to solv-ing problems for which outcomes are uncertain andwhere critical sources of information are nonexistent orunavailable. Imagine trying to unlock a door with a set

310


2/17

Lee et al.: The Mixed Effects of Inconsistency on Experimentation in OrganizationsOrganization Science 15(3), pp. 310326, 2004 INFORMS 311

of unfamiliar keys. Putting one key into the lock to seeif the lock will turn is experimentation (Rosenthal andRosnow 1992); even if the experiment fails, new knowl-edge is created that narrows the scope of subsequenttrials.

Each trial in experimentation generates information

about a solution that the experimenter could not know inadvance. Information learned in a previous trial can beused to modify subsequent experimental designs, condi-tions, or even the nature of the desired solution (Thomkeet al. 1998). Tasks that are conducive to effective experi-mentation are those that allow multiple problem-solvingtrials and present opportunities to use knowledge gainedfrom earlier trials to enhance learning in subsequenttrials.

Experimentation behavior is critical to organizationalinnovation. Important discoveries in science (such asartificial vaccines) and technology (such as the electriclightbulb) resulted from constant trial-and-error experi-

mentation through which inventors systematically builtup a knowledge base (Thomke 2003). Allen (1977)found that R&D teams spent 77% of their time on exper-imentation and that the analyses of such experimentsconstituted an important source of technical information.Experimentation advances an engineers understandingof new analytical concepts, promotes new ways of think-ing, and creates new engineering knowledge (Vincente1990). More broadly, individuals who constantly impro-vise, tinker, and experiment are able to remain adaptivein fast-paced industries where new ideas and innovationsare constantly in demand (Ciborra 1996).

The Role of Failure in Experimentation. Failures areunavoidable outcomes of experimentation because theoutcome of any single experiment or trial is uncertainin advance. For example, when selecting an unknownkey out of many to unlock a door, one does not knowin advance whether or not the key will work; risk offailure is thus unavoidable. Such failures can be bene-ficial because they provide the experimenter with newknowledge about the solution and thereby facilitate inno-vation and performance in the long run (Sitkin 1992).As Thomke (2003) states, When pharmaceutical com-panies such as Eli Lilly launch new drugs or automotivefirms like BMW introduce new cars, the products are

the result of as many failed experiments as successfulones. An innovation process is at least partially basedon accumulated failure that has been carefully under-stood (p. 27). Consistent with this description, individ-uals who select tasks in which failures are likely (ratherthan safe tasks in which they know they can performwell) tend to persevere in the midst of hardship and per-form better in the long run than others (Dweck 1986).

Despite its benefits, failure has costs and is oftenavoided by both organizations and their members(Michael 1976). Clearly, failures can alienate customers,

reduce business, and lead to dissatisfaction amongemployees. At the extreme, failures can harm employ-ees or customers, financially undermine the organization,and lead to the organizations demise. Yet, even whenthese costs of failure are greatly reduced, people are stillreluctant to experiment. Thomke (1998) found that when

new technologies dramatically reduced the economiccosts, time, and effort associated with experimentationsuch that incurring failures would not harm the orga-nizations budget, deadlines, cost structure, employees,or customersindividuals still seemed to avoid experi-ments in which failures were likely.

This avoidance can be explained by the interpersonalor social costs of failure. Specifically, failures makeones gaps in expertise and knowledge salient to oth-ers (Lee 1997), and avoiding failure helps to maintainones image and professional standing among colleagues(Wolfe et al. 1986). Interpersonal costs of failure areexaggerated when people lack psychological safety.

Psychological safety refers to a belief that a group ororganization would not hold a persons mistakes, errors,and failures against him or her (Edmondson 1999, 2003).Without psychological safety, members of organizationsare likely to be concerned with the interpersonal risks offailure and to be reluctant to engage in experimentation.

While psychological safety research has primarilyinvestigated behavior in work groupslinking psycho-logical safety to willingness to take risks, admittingerror, and asking for help in teamsother researchhas found that individuals and organizations also dif-fer in psychological safety (Edmondson and Mogelofforthcoming). For example, differences across organi-zations in psychological safety have been shown toaffect the level of anxiety people feel when confrontingambiguity and uncertainty (Schein 1985). Organizationaldifferences in psychological safety can be created bysupportive structures such as information and rewardsystems (Edmondson and Mogelof forthcoming) and bythe words and actions of high-level management; in par-ticular, messages that indicate supportiveness, openness,and tolerance for error affect beliefs about the level ofpsychological safety throughout an organization (Detert2003).

Antecedents of Experimentation Behavior. Although

little research has examined organizational conditionsthat promote experimentation, many studies identify pre-dictors of similar behaviors such as learning, creativ-ity, information seeking, and other interpersonally riskybut organizationally desirable behaviors. This work hasfound that creativity is related to organizational cul-ture, reward systems, supervisory encouragement, trust,and resources (Amabile et al. 1996). Feedback, infor-mation, help-seeking, and issue-selling behaviors are allpredicted by supportive organizational norms, leader-ship openness, and trust (Ashford and Northcraft 1992,


3/17

Lee et al.: The Mixed Effects of Inconsistency on Experimentation in Organizations312 Organization Science 15(3), pp. 310326, 2004 INFORMS

Ashford et al. 1998, Lee 1997, Morrison 1993). Proac-tive learning behaviors are related to supportive orga-nizational contexts (access to resources, information,training, and supportive reward systems), leader coach-ing (Edmondson 2003), and routines that encourageexchange of relevant information, reduce sensitivity to

feedback, decrease defensiveness, and increase trust(Argyris 1994).

Organizational variables that affect innovation behav-iors include both normative and instrumental influences.Normative influences, such as organizational cultureand espoused values, influence employee beliefs andbehaviors by establishing norms and standards thatdefine appropriate and inappropriate forms of behav-ior (Hackman 1992). Normative values can be explic-itly stated by leaders (e.g., through speeches, signs, ormemos) or tacitly communicated in features of the orga-nizational environment (e.g., organizational routines)

(Amabile et al. forthcoming, Schein 1983). Normativevalues can inform individuals that failures are accept-able, thereby creating psychological safety and enablingexperimentation behaviors. There is evidence that whennormative values state that failures are expected andacceptable as part of learning, people are less hesitantto discuss mistakes (Edmondson 1996) and more willingto try novel tasks, even at the expense of incurring morefailures (Dweck and Leggett 1988).

The second categoryinstrumental influencespertains largely to formal reward systems and incentives.Instrumental rewards influence the instrumentalities, orcosts and benefits, of experimentation behaviors. For

example, when employees do not have easy access totime, materials, or information to experiment with theirideas, experimentation can be too costly to be prac-tical (Edmondson 1999). Rewards systems that pun-ish failures increase the costs of experimentation, andmay make individuals reluctant to experiment (Thomke2001). A study of airline employees showed that rewardsystems that punished individuals when problems arosereduced employees willingness to adopt new routines(Gittell 2000).

Managers seeking to change employees behavior canchoose to alter either normative or instrumental factors,

as we saw in the Bank of America example, or to changeboth at the same time. Unlike most research on inno-vation behaviors, this paper examines combinations ofnormative and instrumental influences to study effects onexperimentation, rather than examining one or the otherseparately. We also take into consideration a critical dif-ference that exists across individuals in organizationsthe degree to which they are being closely evaluated ontheir performancebecause evaluative pressure is likelyto affect psychological safety and willingness to riskfailure.

Evaluative Pressure. Evaluative pressure refers to thedegree to which salient others are seen as judging ratherthan enabling ones performance. Although most indi-viduals in organizations are being evaluated to someextent, some face more evaluative pressure than oth-ers. Individuals under high evaluative pressure receive

intense scrutiny directed at rating their performancerather than at providing helpful information or feedback.In contrast, those under low evaluative pressure eitherreceive helpful information and support from supervisorsor other observers, or else simply face a lack of intensescrutiny.

Evaluative pressure is distinct from coaching, in whichclose attention or monitoring is provided to facilitaterather than evaluate performance. Indeed, monitoring inthe context of supportive coaching can actually enableinterpersonal risk taking (Edmondson 1999, 2002), whileclose and constant evaluation intended to identify andexpose failures has been shown to inhibit creativity

(Amabile et al. forthcoming) and make novel or unfa-miliar tasks more difficult (Zajonc 1965). Building onthis work, we argue that evaluative pressure makes fail-ures especially salientinhibiting admission of error(Edmondson 1996) and help seeking (Lee 1997)andis thus likely to inhibit experimentation.

Consistent and Inconsistent Organizational

Conditions

Inconsistency and Experimentation. Much of the res-earch noted above has focused on how single variablese.g., normative values, instrumental rewards, or evalu-

ative pressureindependently affect innovation behav-iors. For example, Amabile et al. (1996) showed thateight organizational conditions individually predictedcreative performance, and Ashford et al. (1998) exam-ined four antecedents of issue selling. These studiesassume an incremental or additive model of influenceson behavior. One implication of this componential per-spective is that improving any one of various organiza-tional factors should increase these behaviors.

We are interested instead in how combinations oforganizational variables affect innovation behaviors.A combinational perspective assumes that the combina-tion of conditions employees face may be as influen-

tial as the individual conditions themselves. The Bankof America example illustrates what can happen whennormative values are changed to explicitly encourageexperimentation and instrumental rewards discourage it.Inconsistency in organizational conditions may actuallydo more harm than good, because it creates uncertaintyin which individuals do not know which factor (e.g.,normative values or instrumental rewards) will shape theorganizations response to their actions.

Consistency, or lack thereof, has implications forpsychological safety. First, psychological safety should


4/17


be greatest when organizational conditions such asinstrumental rewards, normative values, and evaluativepressure are aligned, consistently encouraging experi-mentation. Under these conditions, the message that fail-ure is an acceptable element of the innovation processis powerful and unambiguous. Thus, when organiza-

tional conditions consistently encourage experimenta-tion, we expect more experimentation behaviors thanwhen organizational conditions consistently discourageexperimentation.

In contrast, inconsistency may reduce psychologicalsafety and thus experimentation. First, inconsistent con-ditions make the rules unpredictable and ambiguous.The uncertainty about whether one will be punished cre-ates a state of mild fear, which is antithetical to feel-ings of psychological safety. Second, facing the needto simultaneously serve contradictory aims itself maycreate anxiety, lowering psychological safety. Inconsis-tent messages place people in a bind (Argyris 1982)

because they communicate two incompatible goals (e.g.,experiment with new ideas, but dont fail). Facingthis, people may experience emotions of fear or anxietythat make taking action and not taking action equallyunpleasant alternatives (Argyris 1990). Third, inconsis-tency has been shown to create cognitive and emotionalresponses such as suspicion, mistrust, and confusion,leading to threat rigidity, a tendency towards riskaversion, behavioral inhibition, suppression of activity,avoidance, lack of openness, and an inability to try novelbehaviors (Masserman 1971, Staw et al. 1981).

Because inconsistency lowers psychological safety,increases fear, and makes failures social costs salient, it

in turn decreases experimentation behaviors. This sug-gests that inconsistent organizational conditions wouldlead to less experimentation than consistent organiza-tional conditions. While this argument leads to thesomewhat intuitive idea that consistently encouragingorganizational conditions would lead to more experi-mentation behaviors than inconsistent conditions, it alsosuggests a less intuitive scenario. Specifically, it is possi-ble that individuals will engage in more experimentationbehavior when organizational conditions consistentlydiscourage experimentation than when some conditionsencourage experimentation and others do not. In the con-sistently discouraging situation, individuals are clear

about the rules and constraints they encounter, and there-fore may experience more psychological safety than theywould when facing the uncertainty created by inconsis-tent conditions. If so, they may experiment more.

Further, in consistently discouraging conditions, peo-ple working closely together can experience a sense ofsolidarity based on shared perceptions of negative workconditions (Edmondson 1999, George and Zhou 2002),whereas inconsistent conditions may lead to mistrust andsuspicion that undermine psychological safety. Whenan organizational context discourages experimentation,

the immediate interpersonal context in a specific workgroup can still be characterized by psychological safety(Edmondson 1999), encouraging experimentation, albeitat a smaller scale. It is thus conceivable that experimen-tation may be higher in the consistently discouragingconditions than under inconsistent conditions, such that

changes in a single organizational attribute to encourageexperimentation while leaving others unchanged maylower experimentation. One goal of the current researchis to explore this possibility empirically.

The Combinational Perspective. Examining the ef-fects of inconsistency requires testing multiple organi-zational conditions in combination. Many organizationalscholars have advocated studying combinations of orga-nizational attributes rather than single attributes (e.g.,Meyer et al. 1993). For example, research has exam-ined how specific configurations or bundles of or-ganizational attributes, policies, and characteristics

predict organizational performance (Inchiowski et al.1997). Ideal configurations are those where various orga-nizational attributes fit well together, with fit definedas consistency among various organizational factors(Doty et al. 1993). In a study of human resources (HR)practices, MacDuffie (1995) argued, bundles of inter-related and internally consistent HR practices createmultiple, mutually reinforcing conditions that supportemployee motivation and skill acquisition (p. 198, ital-ics added). This work suggests that an organizationalcharacteristic may have beneficial effects in combina-tion with one set of organizational attributes but havean opposite effect in combination with another set of

organizational attributes. Taking this perspective meansexamining synergistic and higher-order interactionsrather than simply examining main effects or linear rela-tionships (Delery and Doty 1996). This perspective alsosuggests that changing single organizational attributes ina piecemeal or incremental fashion may be detrimentalto outcomes.

The combinational perspective has been used toexamine measures of organizational performance suchas productivity, manufacturing quality, and efficiency,but it has not been applied to the study of individ-ual behaviors within the organization. Meyer et al.(1993) found that current theories of individual and

group behavior in organizations, ranging from per-sonality, motivation, task design, work group design,and organizational demography, have largely adopted acomponential rather than combinational approach. Forexample, Hackman and Oldhams (1980) job charac-teristics model assumes that task attributes that affectmotivation are compensatorythat a high level of oneattribute compensates for a low levels of others. How-ever, one might posit a nonlinear, noncompensatorymodel wherein different configurations of task attributesare associated with different behavioral and attitudinal


5/17


responses (Meyer et al. 1993, p. 1,188). Similarly,research on work group design has not consideredhow different combinations of antecedent variablescould be associated with different levels of groupeffectiveness (p. 1,190). Assuming that inconsistencyof organizational components affects experimentation

behaviors, a combinational perspective becomes neces-sary for understanding the antecedents of experimenta-tion in organizations.

Research Question

This paper contrasts two perspectives on howorganizational conditions affect experimentation behav-iors. The componential perspective suggests thatorganizational conditions (such as normative values,instrumental rewards, or evaluative pressure) indepen-dently affect innovation behaviors. Thus normative val-ues that encourage experimentation will lead to higherlevels of experimentation behavior, regardless of instru-

mental rewards and evaluative pressure; instrumentalrewards that do not punish failures will lead to higherlevels of experimentation behavior, regardless of nor-mative values and evaluative pressure; and individualsunder high evaluative pressure will experiment less thanindividuals under low evaluative pressure, regardless ofnormative values and instrumental rewards.

The combinational perspective suggests that the inter-action between organizational conditions is also impor-tant. This perspective suggests that when organizationalconditions are consistentfor example, when norma-tive values, instrumental rewards, and evaluative pres-sure all encourage experimentationthere will be more

experimentation than when organizational conditionsare inconsistentwhen some encourage experimentationand some discourage experimentation. This perspectivealso allows the possibility that experimentation will begreater when organizational conditions consistently dis-courage it than when they are inconsistent.

These perspectives are not mutually exclusive. It ispossible to find that overall, each attribute has dis-cernible main effects on experimentation and that com-binations yield interaction effects that cannot be detectedby linear main effects alone. Our first study is anexploratory pilot study to see whether interaction effectsbetween normative values, instrumental rewards, and

evaluative pressure predict experimentation. Becausefew studies have empirically measured experimentation,we do not make specific hypotheses about how thesethree variables would interact to predict experimenta-tion, but will use the preliminary findings that emergedfrom Study 1 to develop more specific hypotheses forStudy 2. We also use this exploratory study to investigatedifferent measures of experimentation, normative values,instrumental rewards, and evaluative pressure, which weuse to develop more focused and well-defined opera-tionalizations in Study 2.

Study 1: Exploratory Research in the FieldStudy 1 explored organizational antecedents of exper-imentation behavior in a hospital that had recentlyimplemented a Web-based clinical information system.Study 1 was part of a larger survey study examining theimplementation of this new system. Although the larger

study was not originally designed to examine experi-mentation or the research question posed in this paper,the survey provided meaningful approximate measuresof key variables for this research.

This site had several advantages for examining ourresearch question. First, past research in hospitals hasshown that there are naturally occurring, realistic, andmeaningful differences in normative values, instrumen-tal rewards, and evaluative pressure (Edmondson 1996).Second, the medical profession exemplifies a workplacewhere competence is highly emphasized and where fail-ures are evaluated harshly (Edmondson 1996, Lee 2001).Third, hospitals are extremely hierarchical organizations

where differences in evaluative pressure between occu-pations tend to be pronounced and well understoodby all members (Edmondson 1996, Lee 1997). Fourth,there was no training when this new system was imple-mented as the designers assumed that most users wouldalready be familiar with the World Wide Web interface.Therefore, to successfully learn the new system and itsfeatures, users had to engage in multiple trials of exper-imentation, clicking on different icons to find out whattypes of information was provided. Further, experimen-tation was imperative to successful learning, as infor-mation previously learned could be applied to enhanceperformance later. For example, after users learned to

successfully gather one type of information, this knowl-edge could be used to navigate other parts of the systemto gather different types of information.

Background

The study was conducted at a large Midwestern medicalcenter that consisted of 3 hospitals, 30 health centers,and 120 outpatient clinics. At the time of this study, clin-ical activity at this organization included 35,615 yearlyadmissions and 1,149,473 yearly outpatient visits. Themedical center had 872 total licensed beds and 8,321employees.

In January 1998, this organization implemented a

Web-based clinical information system. This new infor-mation system integrated data from over 20 differentsources of clinical information (such as blood testsresults and medication orders). Rather than logging onto different systems with varied interfaces and platformsto access information about patients, users of this newsystem can access up-to-date clinical information froma single location.

Employees with heavy patient contact, includingphysicians, nurses, allied health care givers (such associal workers, physical therapists, or dieticians), and


6/17


support staff (such as clinical clerks), were the pri-mary users of the system. The system could be accessedby any workstation with Web access, and health careproviders could use the system on patient floors, in theoperating rooms, their offices, or even at home. The sys-tem was introduced at the same time to all users in all

departments within the organization.

Methods

A Web-based survey measured individuals usage andattitudes regarding this new technology. About onemonth after the system was introduced, users of the newsystem were solicited to visit the survey website viae-mail messages. Because there was no single e-maildistribution list that included all the users of this system,administrators of various functional areas forwarded thee-mail to their subordinates. We were not always ableto find out what individuals or how many individualswere included in each list, and thus we did not know

how many people actually received the e-mail message.Users were also solicited when they logged on to thenew system, asking them to respond to the survey viaa link on the homepage of the system. Both methodsof soliciting respondents presented sampling biases. Forinstance, soliciting respondents on the system homepagereached existing users only. Similarly, e-mail solicitationwas unlikely to reach individuals who were not in theintended user group.

Participants

Two weeks after sending out e-mail messages andincluding a reminder in the system homepage, there

were 688 responses. Of the respondents, 18% were male.The average age was 39.8, and average employment atthe hospital was 8.4 years. Nurses made up 46% of

Table 1 Factor Loadings of Principal Components Analysis Using Varimax Rotation: Study 1

Factors

NVa IRa Expa

In this department:

1. Current ITb systems work and will remain unchanged (R). 066 005 034

2. It is not worth the trouble to question the current IT system (R). 065 028 004

3. Leaders emphasize being on the cutting edge of IT even if it is not perfect. 063 031 005

4. If I come across a new but imperfect IT idea, I can still influence others. 054 015 032

5. I can be critical of IT ideas, even if they come from leaders. 053 017 0096. Traditional IT systems should be upheld because they work (R). 052 016 001

7. It is okay to try new IT without negative repercussions. 035 072 010

8. Leaders reward IT innovators, even if they fail. 037 072 013

9. If I make a mistake with IT, it will be held against me (R). 010 066 021

10. I avoid IT systems with uncertain potential (R). 020 007 081

11. I avoid making errors with IT as much as possible (R). 023 016 078

12. There is one best IT to achieve desired outcomes (R). 021 028 067

13. It is dangerous to experiment with not tried-and true IT (R). 028 018 063

14. Errors on IT are signs of failure (R). 002 018 055

Notes. (R) indicates items that have been reversed scored;a NV=Normative values, IR= Instrumental rewards, Exp=Self-report experimentation with the new technology;b IT is short for information technologies.

those surveyed; 13% were physicians; 13% were alliedhealth care givers; 12% were clinical support; 10% wereadministrative personnel; 4% were medical students; and2% were unknown.

Because we had no accurate number as to how manyindividuals read the e-mail message or saw the link on

the system homepage, we were not able to calculatean accurate response rate. To check for sampling bias,the responses of the 688 respondents were comparedwith 57 nonvoluntary respondents (46% physicians,54% nurses). These nonvoluntary respondents wererecruited in various departmental meetings or roundswhere the surveys were handed out and collected bysenior administrators. There were no significant differ-ences between these two groups in the key variables usedin this study.

Measurement of Key Variables

Normative Values and Instrumental Rewards. A largerconcurrent study on overall system implementationincluded survey items that measured various attributesof the organizational contextsuch as empowerment,resources, supportiveness, and culture (Edmondson1999, Lee et al. 1996, Spreitzer 1996). From thesesurvey items, we extracted items that tapped into:(a) participants perceptions of whether the organiza-tions normative values encouraged experimentation withnew technologies (Items 1 to 6 in Table 1) and (b) partic-ipants perceptions of whether the organizations instru-mental rewards encouraged experimentation with newtechnologies (by not penalizing failures; Items 7 to 9 in

Table 1). All surveys items were formatted in a Likertscale from 1 (strongly disagree) to 7 (strongly agree).Although these items were used in past research to


7/17


measure general norms and rewards around learning, wemodified them to focus more narrowly on learning anew information technology. Given that these items werenot originally designed to measure normative values orinstrumental rewards exclusively, questions can be raisedabout their validity. However, as an exploratory analysis,

we felt that these items could reveal useful trends thatcould be more systematically tested in Study 2.

Experimentation Behaviors. Experimentation wasmeasured in three ways. First, in addition to items aboutnormative values and instrumental rewards, the surveyalso included items asking participants to indicate howoften they experimented around new technologies (Items10 to 14 in Table 1). Second, 29 features of the newsystemfor example, displaying specific types of infor-mation, customizing clinical information, or creating spe-cific short cuts in the system, etc.were listed on thesurvey, and respondents indicated whether they had usedeach feature. We reasoned that respondents who exper-imented more would have used more features. Third,respondents rated how often they used various strategiesto solve system problems, including asking informationtechnology personnel, sending e-mail, asking colleaguesfor help, referring to system documentation, giving up,or trial and error. We reasoned that those who experi-mented more would indicate higher levels of trial-and-error behavior.

In the survey, respondents also indicated their age,gender, departmental affiliation, profession, and yearsemployed at the hospital. Respondents also indicatedtheir prior experience with computers on a scale of 1

(never used a computer before) to 7 (a regular and expertuser).

Evaluative Pressure. We inferred evaluative pressurefrom the occupation of our respondents. The differentlevels of evaluative pressure faced by those in differ-ent occupations within medicine are well established inthe health care literature (Jones et al. 1997). Medicalstudents, for example, are constantly under the super-vision of interns, residents, and attending physicians,who closely evaluate the students every act in an effortto identify and correct for errors. Attending physicians,though not immune from evaluative pressure, are lessclosely evaluated in their day-to-day activities. In fact,

senior attending physicians are not always able to evalu-ate the competence of more junior attending physiciansas they seldom observe each other at work. Further,there is typically less span of control at lower levelsof the hospital hierarchyfor example, an intern mightevaluate only one student at a time, but the departmentchair of a clinical area might evaluate up to 60 attendingphysicians. As noted by Konners (1987) ethnographicstudy of lower-level members of medical teams, thesmall span of control at lower levels in the medical hier-archy is almost always used for purposes of evaluation

rather than to provide coaching or mentoring. In sum,individuals at the lower levels of the hierarchy moreclosely evaluate the activities of each of their subordi-nates, contributing to higher evaluative pressure.

We asked one physician and one nurse to rate theevaluative pressure associated with the various occupa-

tions of our respondents. This process yielded five levelsof evaluative pressure, including (in ascending order):physicians, medical students, nurses, allied health caregivers, and secretarial/administrative staff. A variable(1 to 5) was created in which higher values indicatedoccupations with higher evaluative pressure. Note thatthese various occupations differ along many dimensionsbesides evaluative pressure, and we must be cautiouswhen inferring evaluative pressure from occupationalstatus alone. However, as an exploratory analysis, thisapproach yields useful insights that could be more sys-tematically tested in Study 2.

Results

Preliminary Analyses. Survey items were subjectedto a principal components analysis. Using a step-up-by-one approach with varimax rotation, a three-factorsolution was extracted. The factor loadings are listedin Table 1. The three factors are: (a) normative val-ues that encourage experimentation with new technolo-gies, (b) instrumental rewards that do not penalizeexperimentation with new technologies, and (c) self-report experimentation with new technologies. This fac-tor structure conforms closely to the variables each itemwas designed to measure. We formed three composite

variables by averaging across all items within each fac-tor (after reverse scoring appropriate items). The inter-nal reliabilities of the composites for normative values,instrumental rewards, and experimentation were 0.87,0.74, and 0.71, respectively.

Relationships Between Organizational Antecedents

and Experimentation. Correlations between key vari-ables are listed in Table 2. The three experimentationmeasuresself-report experimentation, usage of newfeatures, and trial-and-error problem solving were posi-tively correlated with each other, although the size of thecorrelation coefficients were moderate. Thus, separateregression analyses were conducted using each of the

three experimentation ratings as the outcome variable;with normative values, instrumental rewards, evaluativepressure, and the interaction terms as predictors. Con-trol variables included years employed at the hospital,age, gender, and prior computer experience. All predic-tor variables were centered and entered into the regres-sion model simultaneously (Aiken and West 1991).

The results are shown in Table 3. Men reported moreexperimentation, but we found no gender differences intrial-and-error problem solving or feature usage. Priorexperience with computers was positively related to all


8/17


Table 2 Correlations Between Key Variables: Study 1

Correlation coefficients

1 2 3 4 5 6 7 8 9 10

Means 87 398 018 55 52 41 31 55 41 132

Std. dev. 75 96 039 15 16 10 15 11 12 77

1. Yrs. employed 102. Age 057 10

3. Gendera 018 008 10

4. Experienceb 015 015 017 10

5. Instr. rewards 015 012 020 023 10

6. Norm. values 007 004 008 006 021 10

7. Eval. pressurec 006 006 040 004 004 013 10

8. Self-report exp. 004 003 004 016 025 032 006 10

9. Problem solve 011 003 001 022 017 026 003 036 10

10. Feature usaged 005 005 014 018 008 012 012 012 045 10

p< 005, p< 0005, p< 00005, N= 662.a 1=male, 0= female;b prior experience with computers (17 scale);c evaluative pressure: 1(low) to 5 (high);d # of system feature used (29 total). All other variables (besides year employed and age) are on a scale of 1 to 7.

three measures of experimentation. Normative valuessignificantly predicted all measures of experimentation.Individuals in departments with normative values moreencouraging of experimentation reported more exper-imentation, more trial-and-error problem solving, andmore feature usage. The relationships between instru-mental rewards and experimentation were mixed. Indi-viduals who perceived that failures were not penalizedreported more experimentation in the survey. However,instrumental rewards did not significantly predict trial-and-error problem solving or feature usage. The

relationships between evaluative pressure and experi-mentation were also mixed. Individuals in occupationswith higher evaluative pressure used fewer features, butthe effect was not significant for the other measures of

Table 3 Unstandardized Regression Coefficients: Study 1

Unstandardized regression coefficients (standard errors)

Experimentation Self-report exp. Prob. solve Feature usage

Intercept 482 024 297 027 488 187

Control variables

Yr. employed 000 001 001 001 002 005

Age 000 000 000 001 007 004+

Gender 029 011 006 012 075 087Experience 011 003 019 003 106 021

Predictors(centered)

Instr. reward (IR) 011 003 002 003 004 021

Norm. values (NV) 020 004 017 005 098 034

Eval. pressure (EP) 003 004 006 004 10 031

Interactions terms

IR NV 005 003+ 007 003 003 021

IR EP 003 003 002 003 010 019

NVEP 005 003 003 004 008 025

NV IR EP 003 001 004 001 017 010+

p< 005, p< 0005, p< 00005.

experimentation. The two-way interaction of normativevalues and instrumental reward was mixed. The inter-action was significant only for trial-and-error problemsolving. Normative values positively predicted trial-and-error problem solving when instrumental rewards werehigh, but negatively predicted trial-and-error problemsolving when instrumental rewards were low.

Most interestingly, we found significant three-wayinteractions between normative reward, instrumental val-ues, and evaluative pressure for all three measures ofexperimentation. Figure 1 illustrates the interactions. Forindividuals in high evaluative pressure occupations, nor-mative values positively predicted experimentation onlywhen instrumental rewards were also high (or support-ive of experimentation). However, when instrumental


9/17


Figure 1 The Interaction of Normative Values, Instrumental Rewards, and Evaluative Pressure on Experimentation: Study 1

Trial-and-Error Problem Solving

2.00

2.25

2.50

2.75

3.00

3.25

3.50

3.75

Features Used

4.50

5.00

5.50

6.00

6.50

7.00

7.50

8.00

Willingness to Experiment

Values

4.20

4.40

4.60

4.80

5.00

5.20

5.40

4.00

Values Values

2.00

2.25

2.50

2.75

3.00

3.25

3.50

3.75

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

Encouraging Rewards

Discouraging Rewards

4.20

4.40

4.60

4.80

5.00

5.20

5.40

4.00

Values Values Values

Encouraging Rewards


Encouraging

Rewards

Discouraging Encouraging

LowE

valuativePressure

HighEvaluativePr

essure

Discouraging

Rewards

Discouraging EncouragingDiscouraging Encouraging

Discouraging Encouraging Discouraging Encouraging Discouraging Encouraging

Encouraging

Rewards

Discouraging

Rewards

Encouraging Rewards


Encouraging Rewards


rewards were low, normative values negatively predictedexperimentation. In short, when normative values andinstrumental rewards were inconsistent, experimentationbehaviors decreased. As Figure 1 shows, this trendwas reversed for individuals in low evaluative pressure

occupations. Here, normative values positively predictedexperimentation when instrumental rewards were low,and normative values negatively predicted experimen-tation when instrumental rewards were high. In otherwords, experimentation increased when normative val-ues and instrumental rewards were inconsistent with oneanother.

Discussion

This exploratory study examined how normative values,instrumental rewards, and evaluative pressures, singly orin combination with one another, related to experimen-tation. Consistent with many past studies on innovation

behaviors, we found some support for the componentialperspective. The results showed that normative valuessupportive of experimentation predicted more experi-mentation behaviors. However, instrumental rewards andevaluative pressure did not show consistent relationshipswith experimentation.

However, a componential perspective does not tellthe whole story. The three-way interactions showed thatindividuals in occupations with high evaluative pressurewere relatively less likely to experiment when norma-tive values and instrumental rewards were inconsistent

with one another (that is, when one supported exper-imentation and the other did not). This is consistentwith our argument that inconsistency lowers psycholog-ical safety, increasing the costs of failures, and lead-ing to less experimentation. It is possible that, being

under close and constant evaluation, these individualsare already feeling less trusted and more defensive,and thus already have a reduced sense of psychologicalsafety (Edmondson 1999). When inconsistent organiza-tional conditions further reduced perceptions of safety,experimentation suffered.

In contrast, individuals in low evaluative pressureoccupations experimented more when one condition sup-ported experimentation and the other did not. Under theuncertainty created by inconsistent organizational condi-tions, individuals under low evaluative pressure may per-ceive more leverage to experiment with their own ideas.

Indeed, Keltner et al. (2003) argue that individuals underlow evaluative pressure typically have more resourcesat their disposal, are less likely to be sanctioned fordeviating behaviors, and more likely to approach theirown goals in uninhibited ways. These tendencies maybe enhanced when the organization does not have a sin-gle, clear, and compelling direction to guide and con-tain their behaviors. Unlike people under high evaluativepressure who tend to become avoident and inhibited,people under low evaluative pressure are more likely toexhibit behavioral approach tendencies, or attempt risky


10/17


and novel courses of action during times of uncertainty(Carver and White 1994).

Limitations of Study 1. Overall, the results of theexploratory study provide preliminary support forthe combinational perspective. Particularly, evaluativepressure moderates how inconsistent organizational

conditions predict experimentation. However, as anexploratory study, Study 1 presents several limitationsthat require further testing and replication in a morecontrolled setting. For example, the results are correla-tional, and do not allow us to make causal inferences orrule out alternate explanations for the findings. We reliedon self-reports of experimentation that could be sus-ceptible to demand characteristics (for example, respon-dents might have reported using more system featuresthan they actually did). We asked respondents abouttheir perceptions of normative values and instrumen-tal rewards, but did not collect other data to test that

these perceptions reflected actual differences in howdepartments were structured and managed. Due to thenumerous and varied departments within the hospital(physicians listed 29 different departments, nurses listedmore than 15, ancillary staff listed 6), we were unable toaccurately determine respondents departmental affilia-tions, and therefore unable to compare normative valuesand instrumental rewards across departments. Norma-tive values, instrumental rewards, and self-report exper-imentation were all measured using the same surveyinstrument, creating the potential for bias due to com-mon method variance.

As noted above, it is also problematic that evaluative

pressure was inferred from the respondents occupationrather than measured directly. Indeed, evaluative pres-sure and occupation are distinct concepts. It is conceiv-able for two individuals to have the same occupation,but for one to experience more evaluative pressure thananother. Also, different occupations have different taskdemands, which can lead to different needs for usingthe system. A doctor might need more types of clinicalinformation, and thus be more likely to use a variety ofsystem features than a clerk with a narrow job function,who might need to use only one or two system featuresregularly. Thus, experimentation might be affected by

job demands rather than status or the evaluative pressure

associated with the occupation.Finally, Study 1 focused on experimentation with a

new information system that was arguably peripheralto respondents core competence. Health care work-ers might not be evaluated based on their competencewith information technology, but rather on competencewith clinical procedures. Further, failures associated withusing the new system could be made privately, as thesystem could be used in the privacy of the users ownoffice or home. Both these factors can reduce the inter-personal costs of failure (Lee 2002).

Study 2: A Laboratory ExperimentStudy 2 tests specific hypotheses that emerged fromStudy 1. First, according to the componential perspec-tive, we expect to find main effects for normative values,instrumental rewards, and evaluative pressure. In short,normative values suggesting that failures are acceptable

will lead to more experimentation behaviors than nor-mative values suggesting that failures are not acceptable;instrumental rewards that do not penalize individualsfor failures will lead to more experimentation behaviorsthan instrumental rewards that penalize individuals forfailures; and individuals under high evaluative pressurewill experiment less than those under lower evaluativepressure.

More interestingly, the combinational perspective sug-gests that we will be able to replicate the three-wayinteraction that emerged in Study 1. Specifically, we pre-dict that individuals under high evaluative pressure areless likely to experiment when normative values and

instrumental rewards are inconsistent than when they areconsistent; we expect these trends to be reversed forindividuals under lower evaluative pressure.

Study 2 was a laboratory experiment where partici-pants experimentation behaviors were measured whilethey solved a maze with a confederate. Three variableswere manipulated in a 2 2 2 design: (1) normativevaluesencouraged or discouraged experimentation;(2) instrumental rewardfailures were penalized or not;and (3) evaluative pressurethe confederate closelyevaluated the participants performance during the entirecourse of the experiment (high evaluative pressure forthe participant), or the participant closely evaluated theconfederates performance during the entire course ofthe experiment (low evaluative pressure for the partic-ipant). Participants performance and experimentationbehaviors during the task were measured.

We used this design to complement and offset the lim-itations of Study 1. We directly measured experimentalbehavior rather than relying on self-reports. By exper-imentally manipulating normative values, instrumentalrewards, and evaluative pressure, we can make causalinferences about the effects of these variables on experi-mentation behaviors, and eliminate confounds with othervariables. Solving the maze was the only task partici-

pants had to complete during the session, and all failureswere visible to both the experimenter and confederate.

Methods

Participants. There were 185 undergraduate students(52% males; average age 18.7) from a large universitywho participated in the study. Participants received par-tial course credit. Participants arrived at the laboratoryat the same time as a same-sex confederate. The confed-erates were undergraduate research assistants; the con-federates and participants had never met each other until


11/17


the start of the experiment. Both the participant and theconfederate filled out a questionnaire providing demo-graphic data and past working experiences. The partic-ipant and confederate were then told that they wouldwork as a team to solve a maze. To motivate partici-pants, they were also told that the top performers would

receive a monetary prize.Materials. Participants were told to solve an elec-

tronic maze originally designed for management simu-lation exercises (see Senge et al. 1993). The maze wasa 6 9 carpet laid out in a grid pattern9 rows of6 squares eachwith some squares emitting an elec-tronic beeping signal when stepped on and other squaresremaining silent when stepped on. We programmed asingle path of contiguous nonbeeping or silent squaresthrough the maze. The path could not be detected visu-ally; to identify it, participants stepped on one square ata time and would learn immediately if that square waspart of the path by whether or not it emitted an audiblesignal. Participants were told that the goal of the taskwas to find the nonbeeping path through the maze.Anytime they stepped on a beeping square (a failure tostay on the nonbeeping path), they had to return to thebeginning of the maze and start again. This task pre-sented multiple trials of problem solving (each of whichended as soon as a failure was encountered) where infor-mation learned on an earlier trial could be used to informsubsequent trials.

Procedure. Participants were randomly assigned tonormative value conditions that either discouragedor encouraged experimentation. The instructions were

adopted from past research examining individualsvalues about learning (Dweck and Leggett 1988). Inconditions where normative values discouraged exper-imentation, the experimenter emphasized that beingaccurate (or avoiding beeps) was instrumental to solv-ing the maze, and told participants to keep on thenonbeeping path and avoid mistakes. In conditionswhere normative values encouraged experimentation,the experimenter emphasized that failures (or beeps)were instrumental to solving the maze, and told partici-pants to learn as you go to find alternative solutions.(Actual scripts for manipulating values and evaluativepressure are available from the authors).

Participants were also randomly assigned to instru-mental reward conditions that either discouragedor encouraged experimentation. In conditions whereinstrumental rewards discouraged experimentation, allbeeps were penalized. Participants were told that everytime they stepped on a beeping square, 30 secondswould be added to their overall time. In conditions whereinstrumental rewards encouraged experimentation, par-ticipants were told that 30 seconds would be added totheir overall time only if they stepped on a beepingsquare that they had stepped on previously. Thus, beeps

that resulted from original experiments or trials were notpenalized, although beeps that resulted from errors inmemory or executionsuch as stepping on a same beep-ing square twicewere penalized. (In reality, we did notadd time to participants final time in either case).

Participants were further randomly assigned into high

or low evaluative pressure conditions. In the high evalua-tive pressure condition, the participant was told that afterthe task was completed, the confederate would evaluatethe participants performance on the experimental task.In the low evaluative pressure condition, the participantwas told that he or she would evaluate the confeder-ates performance on the experimental task. Regardlessof the evaluation conditions, all participants were giventhe task of walking on the maze and all confederateswere given the task of keeping track of the beepingsquares. Note that in both high and low evaluative pres-sure conditions, the participant was not immune fromevaluative pressure. The performance of all the partici-

pants was being monitored and measured, and significantmonetary rewards were contingent on performance. Par-ticipants in the high evaluative pressure condition weresimply under closer and more explicit evaluation thanparticipants in the low evaluative pressure condition.

The experiment ended when the participant success-fully walked from one end of the maze to the otherwithout incurring beeps. The participant then filledout a survey with manipulation-check items and wasdebriefed.

Coding. All the experimental sessions were video-taped. From the videotapes, a coder measured the length

of time each participant took to solve the maze (exclud-ing the time participants took to retrace their steps to thebeginning of the maze after stepping on a beep). Codersalso recorded participants behaviors at a particularlydifficult part of the experimental task. The electronicmaze was programmed such that there was a dead endwithout any viable outlets (that is, without any contigu-ous nonbeeping squares going in the forward direction)from which the route might continue; 97.7% of the par-ticipants encountered this dead end during the exper-imental task, at which point the route they had beenpursuing had to be abandoned. We measured the fol-lowing variables from the videotapes: (a) the trial num-

bers in which the participant entered and left the deadend and (b) the number of errors (number of beepingsquares that had been stepped on previously) made in thedead end.

Measuring Experimentation. We used three variablesto measure the effectiveness of participants experimen-tation. First, we used total time to solve the maze to mea-sure trial-and-error problem-solving behavior. Becausetrial-and-error was necessary to complete the maze (theonly way for participants to find the path was to incurfailures or beeps), participants who engaged in more


12/17


trials more quickly would have a lower solution timethan participants who tried to avoid beeps by hesitatingbetween trials or minimizing error. Second, we measuredthe behavior of trying new courses of action by subtract-ing the trial number participants first entered the deadend from the trial number they left the dead end (trials).

More trials in the dead end indicated less willingnessto abandon a familiar path to experiment with a newpath. Third, we measured errors, or previously stepped-on beeping squares in the dead end (errors). More errorsin the dead end also indicated a stronger commitmentto the familiar path and less willingness to experimentwith a new path as participants kept stepping on beep-ing squares they had stepped on already. Overall, higherlevels of solution time, trials, and errors were indica-tive of less effective experimentation, and lower levels ofsolution time, trials, and errors were indicative of moreeffective experimentation.

ResultsFirst, we examined the effectiveness of our manipu-lations of normative values and instrumental rewards.Using a Likert scale from 1 (strongly disagree) to7 (strongly agree), participants rated their perceptionsof normative values and instrumental rewards. Partic-ipants in the condition where normative values dis-couraged experimentation were more likely to say thatit was important to get the right answers in the taskM= 59 than participants in the condition where nor-mative values encouraged experimentation M= 28t183= 501 p < 00001. Participants in the conditionwhere instrumental rewards discouraged experimenta-

tion were more likely to indicate that they made toomany mistakes (M= 49) than participants in the condi-tion where instrumental rewards encouraged experimen-tation M= 37 t183= 237 p= 001.

The three experimentation measures were significantlyand positively correlated with one another, although thesize of the correlation coefficients were not uniformlyhigh rsolution time trials = 031, p < 0001; rsolution timeerrors =022, p = 0002; rtrialserrors = 083, p < 0001. Sepa-rate analyses of variance were conducted using each ofthe three experimentation measures as dependent vari-ables, and normative values, instrumental rewards, eval-uative pressure, and their interactions as independent

variables. Normative values had no significant effecton any of the measures of experimentation behaviorp > 010. As predicted, participants in the conditionwhere instrumental rewards encouraged experimentationhad fewer trials F1170= 385p= 0051 and fewererrors F1174 = 995p = 0002 in the dead endthan those in the condition where instrumental rewardsdiscouraged experimentation, but the same effect wasnot significant for solution time p > 010. There wereno main effects of evaluative pressure, and none of thetwo-way interactions were significant (ps > 010.

The three-way interaction of normative values, instru-mental rewards, and evaluative pressure was significantfor all three analyses (F1173= 935p = 00026 forsolution time; F1170 = 504, p = 003 for trials;F1174 = 351, p = 006 for errors). The means areshown in Figure 2, where higher values on the y-axis

(higher solution time, more trials in the dead end, andmore errors in the dead end) indicated less effectiveexperimentation. The shaded bars indicate conditionswhere normative values and instrumental rewards wereconsistenti.e., both were high or both were low. Thenonshaded bars indicated conditions where normativevalues and instrumental rewards were inconsistenti.e.,one was high but the other was low. These results showthat when participants were in the high evaluative pres-sure condition, they experimented more effectively whennormative values and instrumental rewards were consis-tent (shaded) than inconsistent (nonshaded). Experimen-tation was lower in the inconsistent conditions than when

both normative values and instrumental rewards consis-tently discouraged experimentation. This trend was notapparent in the low evaluative pressure conditions. Here,individuals were more likely to experiment effectively(i.e., lower solution times, fewer trials and errors) whennormative values and instrumental rewards were incon-sistent than when they were consistent.

Discussion

Study 2 used a radically different methodology thanStudy 1 by experimentally manipulating normativevalues, instrumental rewards, and evaluative pressure.

Unlike Study 1, we failed to find support for the predic-tion that normative values affect experimentation. Oneexplanation for this finding might be the difficulty ofcreating meaningful and realistic normative values ina short laboratory task with verbal instructions (Schein1983). We found that instrumental rewards had the pre-dicted effect on experimentation for two of the threemeasures; people experimented more in the dead endwhen instrumental rewards did not penalize failures (ornew beeps). It is possible that instrumental rewards hada stronger effect on experimentation in especially chal-lenging situations (such as the dead end), than in moreroutine trial-and-error contexts. In the dead end, most

participants expressed frustration as they gradually dis-covered that the previously tried-and-true path had noviable outlets, and every attempt to proceed forwardfrom the dead end was met with a beep, requiring themto begin again before making new progress. In this diffi-cult situation, individuals might be more sensitive to theimplications of instrumental rewards (or penalties) forfailure than when there was no excess difficulty.

Most importantly, the significant three-way interac-tions showed that individuals under high evaluative pres-sure experimented most when normative values and


13/17


Figure 2 The Interaction of Normative Values, Instrumental Rewards, and Evaluative Pressure on Experimentation: Study 2

Trials * Errors**Lower levels are

indicative of higher

levels of

experimentation

3.0

4.0

5.0

6.0

7.0

8.0

9.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

LowEv

aluativePressure

3.0

4.0

5.0

6.0

7.0

8.0

9.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Disc. Enc.Discouraging Encouraging

HighEvaluativePre

ssure

Instr. Reward:Norm. Value:

Inconsistent

conditions

Consistent

conditions

Solution Time *

6.0

6.5

7.0

7.5

8.0

8.5

9.0

9.5

6.0

6.5

7.0

7.5

8.0

8.5

9.0

Instr. Reward:Norm. Value:


Discouraging Encouraging Discouraging Encouraging



Disc. Enc. Disc. Enc. Disc. Enc. Disc. Enc. Disc. Enc.

Disc. E nc. Disc. Enc. Disc. Enc. Disc. Enc. Disc. Enc. Disc. Enc.

instrumental rewards consistently supported experimen-tation, and experimented least when normative valuesand instrumental rewards were inconsistent. The resultsshow that individuals experimented less in the inconsis-tent conditions than when the two factors consistentlydiscouraged experimentation. Conversely, individuals

under lower evaluative pressure experimented mostwhen normative values and instrumental rewards pro-vided inconsistent support for experimentation, andexperimented least when normative values and instru-mental rewards were consistent. These results mirroredthose from Study 1. Below, we discuss these results andtheir implications in detail.

General DiscussionAlthough the literature on innovation has emphasizedorganizational-level structures and processes, an orga-nizations ability to introduce a new product, develop

unique processes, and leverage new technologies beginswith individuals coming up with new ideas and try-ing these ideas out to assess their feasibility (Argoteand Ingram 2000). Understanding conditions that enableindividuals to engage in experimentation behavior is thusan important element of understanding organizationalinnovation (Thomke 2003).

Summary of StudiesWhile previous research has primarily investigated howsingle organizational attributes influence experimenta-tion behaviors, this paper suggests that a more holistic

approach is needed to supplement this work. We arguedthat inconsistency in organizational conditionswhensome encourage experimentation but others do notmight reduce experimentation. We conducted two stud-ies with complementary methodologies to examine thispossibility. Study 1 was an exploratory pilot study in thefield that examined the nature of synergistic interactioneffects between organizational conditions on experimen-tation. Study 2, a laboratory experiment, was conductedto test trends observed in Study 1. While Study 1examined experimentation in a realistic work setting,Study 2s participants worked on a laboratory task witha confederate. Study 1 relied on self-reports of exper-imentation, while Study 2 examined actual experimen-tation behavior. Study 1 inferred evaluative pressurefrom occupational role, and Study 2 directly manipu-lated whether or not participants were explicitly evalu-ated. The results in Study 1 were correlational, while

Study 2s experimental methodology allowed us to makecausal inferences.

Taken together, the studies provided mixed support forthe componential perspective. Normative values did notaffect experimentation in the laboratory, but showed asignificant relationship with self-reported experimenta-tion in the field, where normative values were more real-istic and meaningful. Both studies found partial supportfor the notion that individuals experimented more underinstrumental rewards that did not penalize individualsfor failures. Study 1 found partial support that evaluative


14/17


pressure was related to experimentation, though noeffects of evaluative pressure emerged in Study 2.

By comparison, the combinational perspective receivedmoresupport in these studies. Both found that under highevaluative pressure individuals experimented most whenorganizational conditions consistently supported exper-

imentation and experimented least when organizationalconditions were inconsistent. Under low evaluative pres-sure, individuals experimented most when organizationalconditions inconsistently supported experimentation, andexperimented least when organizational conditions wereconsistent.

Evaluative Pressure as a Moderator of

Inconsistency Effects

The finding that inconsistency leads to less experimen-tation among those under high evaluative pressure isconsistent with the idea that multiple organizational con-ditions should be aligned in the same direction to support

desired behaviors (Doty et al. 1993, MacDuffie 1995).According to this view, when individuals are exposedto consistent messages, each is more likely to be seenas credible and thereby has a stronger effect on behav-ior (MacDuffie 1995). Inconsistency in organizationalconditionswhen some encourage and others discour-age experimentationmay undermine experimentationbehavior, with one factor rendering the other ineffec-tive. Further, inconsistency creates suspicion, mistrust,fear, confusion, and risk aversion (Lerner and Keltner2001, Staw et al. 1981). Yet, our results showed thatthese disabling effects of inconsistency only occurredfor individuals under high evaluative pressure. Evalua-

tive pressure might decrease psychological safety andmight make individuals more vulnerable to the uncer-tainty inconsistent conditions create. Counterintuitively,both studies found that individuals under low evaluativepressure experimented more when organizational condi-tions were inconsistent than consistent. In Study 2, thosewith low evaluative pressure experimented more underinconsistent conditions than when both normative val-ues and instrumental rewards consistently encouraged ordiscouraged experimentation.

The observed differences between individuals underhigh and low evaluative pressure can be explained withresearch findings on the effects of evaluative pressure on

cognitive, emotional, and behavioral outcomes (Lee andTiedens 2001). First, when facing ambiguity and uncer-tainty, people under high evaluative pressure tend to bemore aware of potential punishments and thus be riskaverse, leading to behavioral inhibition and less experi-mentation. In contrast, people under low evaluative pres-sure facing uncertainty tend to be more aware of thepotential benefits from the situation and thus are willingto take risks, experience greater behavioral activation,and engage in more experimentation (Carver and White1994, Keltner et al. 2003).

Second, being evaluated creates the psychological bur-den of being constantly aware of and thinking aboutones performance and of the impression one is makingon the evaluator (Lerner and Tetlock 1999). This psycho-logical burden can tax an individuals mental energy andattention, which in turn can prevent the type of in-depth

processing that is essential for contemplative, strategic,and effective experimentation (Muraven and Baumeis-ter 2000). Our data suggest that these negative conse-quences of evaluative pressure may be exaggerated whenthe external environment is uncertain and ambiguous.

Third, evaluative pressure could alter the emotionspeople experience facing uncertainty. For example,Tiedens et al. (2000) found that, in response to uncer-tainty, individuals under high evaluative pressure weremore likely to exhibit guilt, while individuals under lowevaluative pressure were more likely to exhibit anger.High arousal emotions such as anger might producemore proactive behaviors directed towards change and

innovationthus responding to the uncertainty throughsearchas compared to the low arousal emotions ofguilt, which instead may inhibit proactivity.

Fourth, it is possible that individuals under high andlow evaluative pressures experience the same emotionfacing uncertainty but react to the emotion differently.Specifically, uncertainty may produce fear, which hasbeen shown to produce two somewhat contradictoryresponses: one is an automatic or instinctive responseof behavioral inhibition (or inaction) that requires noexpenditure of mental or cognitive resources; the other isor a more controlled response of behavioral activationthat requires one to exert some level of in-depth process-

ing of the situation (Quirk et al. 1997). If, as mentionedearlier, being under high evaluative pressure taxes onesmental energy and attention such that fewer cognitiveresources are available, then the fear that results fromuncertainty in the environment would more likely pro-duce the automatic response of behavioral inhibition.In contrast, if those under low evaluative pressure do notshare the salience of failure and punishment, nor expe-rience the same drain on their mental energy and atten-tion, they are more likely to have the cognitive resourcesavailable to react to fear in a more controlled fashion,characterized by higher levels of thinking, informationprocessing, and action.

Fifth, evaluative pressure has implications for agencyand control that could affect experimentation behavior.Specifically, individuals under low evaluative pressureare more likely to perceive themselves as having con-trol over external environments (Fiske et al. 1996), beingable to influence others to see things their way (Leeand Ofshe 1981), more able to effect change (Schminke1993), and more likely to have an internal locus ofcontrol (Porter et al. 1981). Thus, when organizationalconditions are inconsistent and the environment appearsuncertain and unpredictable, individuals under lower


15/17


evaluative pressure might be able to draw on these inter-nal, psychological resources to support experimentingwith different ways to effect change and exert controlover this uncertain environment.

All of these explanations have in common the obser-vation that, facing unpredictability, individuals under

high evaluative pressure are more likely to becomeinhibited, fearful, narrowly focused, and rigid, whileindividuals under less evaluative pressure are more likelyto become proactive, optimistic, thoughtful, and riskseeking. The proposition that individuals experience dif-ferent responses to combinations of organizational con-ditions depending on their levels of evaluative pressurehas far reaching implications for behavior in organiza-tions. However, these explanations and results are specu-lative, and future research is needed to better understandhow individuals under different levels of evaluative pres-sure react to inconsistent conditions.

Future research is also needed to more clearly

delineate the difference between status and evaluativepressure. In Study 1, we inferred evaluative pressurefrom occupational status. In Study 2, we manipulatedevaluative pressure directly, but participants may stillhave inferred status differences from our manipula-tions (manipulation checks revealed that the evaluator isindeed perceived as having higher status than the evalu-atee). In organizations, it is common for both high andlow status individuals to feel evaluative pressure, albeitin different forms. High status individuals in organi-zations, such as CEOs, often face evaluative pressure(from shareholders, boardmembers, and employees),although it usually does not take the form of close moni-

toring and constant, day-to-day scrutiny. Future researchis needed to more carefully separate out the effectsbetween status and evaluative pressure and to examinewhat specific attribute of statusaccess to resources,networks, prestige, self-concept, psychological safety, orevaluative pressuremight influence experimentation.

Combinational Approaches to Organizational

Behavior

Unlike previous research on organizational antecedentsof innovative behaviors, we examined the effects of com-binations of organizational conditions in addition to theeffects of single conditions. Our findings suggest that the

combinational perspective can add to our understandingof experimentation behaviors, and proactive behaviors inorganizations generally. This perspective also suggestsseveral important directions for research.

First, more research is needed to examine the mecha-nisms underlying the link between inconsistency of orga-nizational conditions and experimentation. We suggestthat inconsistency has effects on subjective experiencessuch as psychological safety, experienced emotions, riskperceptions, cognitive processing, and mental resourcessuch as energy and attention; however, we did not

measure these variables directly. Understanding thesevarious effects of inconsistency, and how they oper-ate differently for organizational members facing dif-ferent role and situational demands, is critical to ourunderstanding of experimentation and of learning moregenerally.

Second, the combinational perspective stems from theassumption that organizational attributes or systems areinterdependent, such that certain configurations or bun-dles of organizational attributes tend to be more commonthan others (Meyer et al. 1993). For example, discrep-ancies between espoused values and actual managerialpractices are common, and managers are often unawareof these discrepancies (Argyris 1982). It is easier toimagine organizations espousing values that encourageexperimentation but not changing compensation sys-tems, resources, or task structures in ways that are con-sistent with the values than the reverse scenario. Wethus suspect that the type of inconsistency illustrated by

the Bank of America examplewhere espoused valuesencouraged experimentation and instrumental rewardsdid notwould be more common than other combina-tions that create uncertainty (Lee 2001). In a relatedvein, while organizations can be consistent or incon-sistent across various policies, structures, or conditions,they can also be consistent or inconsistent across time.For example, an organization might create instrumentalreward systems that do not penalize failures but laterchange them such that failures are indeed monitored andpenalized. This paper focused on inconsistency acrossorganizational conditions, but inconsistency across time

also may create suspicion and distrust, lowering psycho-logical safety and willingness to experiment. Examiningdifferent types of inconsistencies, and how they mightaffect innovation behaviors such as experimentation, isanother important direction for future research using acombinational perspective.

Third, although the combinational approach to under-standing organizational-level outcomes has generallyshown that consistency between organizational attributespositively predicts performance (Delery and Doty 1996),our research suggests that fit or consistency is not alwaysadvantageous at the individual level. Our findings showthat inconsistency in organizational conditions led to

higher levels of experimentation for individuals underlower evaluative pressure. Lacking evaluative pressure,they were apparently able to react to the inconsistencyby engaging and experimenting with the task at hand.

Conclusion

Much organizational research has examined effects ofsingle conditions on individual behavior without consid-ering interaction effects. This research can mistakenlyimply that innovation, creativity, or learning behaviorswill increase as supportive conditions are added one at a


16/17


time. Although the combinational approach, which sug-gests otherwise, is not new to organizational research,using this approach to examine individual experimenta-tion behaviors is new and opens up some new possi-bilities. In two studies, we found significant interactioneffects that went beyond the main effects of the orga-

nizational variables on experimentation behavior. Theresults suggested that although adding a single support-ive condition to otherwise unsupportive conditions mayfacilitate experimentation for individuals working underlow evaluative pressure, it may decrease experimentationfor those working under high evaluative pressure. Morebroadly, it is easy to imagine two studies of individualbehavior in organizations generating directly contradic-tory results if evaluative pressure (or other importantlatent variables) were not manipulated or measured. Thestudies reported here strongly suggest that particularcombinations of factors can have significant, if at timespuzzling, effects. Our intention in this paper is thus to

spur additional discussion and research on how inves-tigative strategies may affect efforts to understand orga-nizational behavior.

AcknowledgmentsPortions of this paper were presented at the 2000 Academy ofManagement Meetings in Toronto and the Managing Knowl-edge in Organizations Conference at Carnegie Mellon Uni-versity. Sim Sitkin and three anonymous reviewers providedinvaluable feedback that improved the paper greatly. Theauthors want to thank Summer Berman for assistance in datacollection.

ReferencesAiken, L. S., S. G. West. 1991. Multiple Regressions: Testing and

Interpreting Interactions. Sage Publications, Thousand Oaks,CA.

Allen, T. J. 1977. Managing the Flow of Technology. MIT Press,Cambridge, MA.

Amabile, T. M., E. A. Schatzel, G. B. Moneta, S. J. Kramer. Leaderbehaviors and the work environment for creativity: Perceivedleader support. Leadership Quart. Forthcoming.

Amabile, T. M., R. Conti, H. Coon, J. Lazenby, M. Herron. 1996.Assessing the work environment for creativity. Acad. Manage-ment J. 39(5) 11541184.

Argote, L., P. Ingram. 2000. Knowledge transfer: A basis for com-petitive advantage in firms. Organ. Behavior Human DecisionProcesses 82(1) 150169.

Argyris, C. 1982. Reasoning, Learning, and Action: Individual andOrganizational. Jossey-Bass, San Francisco, CA.

Argyris, C. 1990. Knowledge for Action: Changing the Status Quo.Jossey-Bass, San Francisco, CA.

Argyris, C. 1994. Good communication that blocks learning. HarvardBus. Rev. 74(4) 7785.

Ashford, S., G. Northcraft. 1992. Conveying more (or less) thanwe realize: The role of impression-management in feedback-seeking. Organ. Behavior Human Decision Processes 53(3)310334.

Ashford, S., N. Rothbard, S. Piderit, J. Dutton. 1998. Out on a limb:The role of context and impressions management in sellinggender-equity issues. Admin. Sci. Quart. 43 2357.

Carver, C., T. White. 1994. Behavioral inhibition, behavioral acti-vation, and affective responses to impending reward and pun-ishment: The BIS/BAS scales. J. Personality Soc. Psych. 67319333.

Ciborra, C. 1996. The platform organization: Recombining strategies,structures, and surprises. Organ. Sci. 7(2) 103118.

Delery, J. E., H. Doty. 1996. Modes of theorizing in strategic humanresource management: Tests of universalistic, contingency, andconfigurational performance predictions. Acad. Management J.39(4) 802835.

Detert, J. R. 2003. To speak or not to speak: The multi-level influenceson voice and silence in organizations. Unpublished dissertation,Harvard University, Cambridge, MA.

Doty, H., W. Glick, G. Huber. 1993. Fit, equifinality, and organiza-tional effectiveness: A test of two configurational theories. Acad.Management J. 36(6) 11961250.

Dweck, C. S. 1986. Motivational processes affecting learning. Amer.

Psychologist 41(10) 10401048.Dweck, C. S., E. L. Leggett. 1988. A social-cognitive approach to

motivation and personality. Psych. Rev. 95(2) 256273.

Edmondson, A. C. 1996. Learning from errors is easier said thandone: Group and organizational influences on the detectionand correction of human error. J. Appl. Behavioral Sci. 32(1)532.

Edmondson, A. C. 1999. Psychological safety and learning behaviorin work teams. Admin. Sci. Quart. 44 350383.

Edmondson, A. C. 2002. The local and variegated nature of learningin organizations: A group level perspective. Organ. Sci. 13(2)128146.

Edmondson, A. C. 2003. Speaking up in the operating room: Howteam leaders promote learning in interdisciplinary action teams.J. Management Stud. 40(6) 14191452.

Edmondson, A. C., P. Mogelof. Explaining psychological safetyin innovation teams. L. Thompson, H. Choi, eds. Creativityand Innovation in Organizations. Lawrence Erlbaum AssociatesPress, Mahwah, NJ. Forthcoming.

Fiske, S. T., B. Morling, L. E. Stevens. 1996. Controlling self andothers: A theory of anxiety, mental control, and social control.Personality Soc. Psych. Bull. 22(2) 115123.

George, J., J. Zhou. 2002. Understanding when bad moods fostercreativity and good ones dont: The role of context and clarityof feelings. J. Appl. Psych. 87(4) 687697.

Gittell, J. H. 2000. Organizing work to support relational coordination.Internat. Human Resource Management 11(3).

Hackman, J. R. 1992. Group influences on individuals in organiza-tions. M. D. Dunnette, L. M. Hough, eds. Handbook of Indust.Organ. Psych., Vol. 3. Consulting Psychologists Press, PaloAlto, CA, 199267.

Hackman, J. R., G. R. Oldham. 1980. Work Redesign. Addision-Wesley, Reading, MA.

Inchiowski, C., K. Shaw, G. Prennushi. 1997. The effects of humanresource management practices on productivity: A study of steelfinishing lines. The Amer. Econom. Rev. 87(3) 291313.

Jones, K. R., V. DeBaca, M. Yarbrough. 1997. Organizational cultureassessment before and after implementing patient-focused care.Nursing Econom. 15(2) 7380.


17/17


Keltner, D., D. H. Gruenfeld, C. Anderson. 2003. Power, approachand inhibition. Psych. Rev. 110(2) 265284.

Konner, M. 1987. Becoming a Doctor: A Journey of Initiation inMedical School. Viking, New York.

Lee, F. 1997. When the going gets tough, do the

Documents

The mixed effects of inconsistency on experimentation in organizations