Principles and Practices of Outcomes Based Training & Education

gary

Text Box

Riccio, G., & Darwin, M. (2010). Principles and Practices of Outcomes Based Training & Education. In: Riccio, G., Diedrich, F., & Cortes, M. (Eds.). An Initiative in Outcomes-Based Training and Education: Implications for an Integrated Approach to Values-Based Requirements (Chapter 3). Fort Meade, MD: U.S. Army Asymmetric Warfare Group. [Cover art by Wordle.net represents word frequency in text.]

Principles and Practices of OBTE 50

Asymmetric Warfare Group

Chapter 3. Principles and Practices of Outcomes Based Training & Education

Gary Riccio and Morgan Darwin The Wexford Group International

The approach of the U.S. Army Asymmetric Warfare Group (AWG) to Outcomes-Based Training and Education (OBTE) assumes that teaching in the information age must be different from predominant approaches in the industrial age (Darwin, 2008a, b). Whether or not education takes place in an institutional setting, the responsibility for both teaching and learning must become considerably more decentralized than in the 20th century. OBTE further assumes that educational practice in the Army must accommodate the fact that teaching and learning already is more decentralized, and is becoming increasingly so, because of the social and semantic web (Mikroyannidis, 2007; Paavola & Hakkarainen, 2005; Scardamalia & Bereiter, 2006). Accordingly, the Institutional Army should be proactive in assessing its preparedness for the demographic fact that the next generation of Soldiers will be comprised of “millennials” for whom the ubiquitous decentralization offered by the World Wide Web is a basic fact of life (Carlson, 2005; Howe, Strauss, & Matson, 2000). The initiative of the AWG in OBTE is a response to this need for responsible decentralization of teaching and learning. Responsible decentralization implies personal autonomy that must be bounded or grounded in some way. In Army training and education, the appropriate grounding is in Army Values and other cultural imperatives. OBTE helps identify how Army training and education can be grounded in this organizational culture. Partially for this reason, OBTE is what many would consider to be “good training;” however, such a characterization begs all the important questions. Thus, the intent of this chapter is to elucidate specifically what “good” means in the context of Army training and education, and how it relates to the emerging understanding of Full Spectrum Operations. The result of this review is a better understanding of the established and emerging instructional practices that should be sustained and those that should be improved to meet the needs of the future. It provides the basic facts for a deeper scientific inquiry into the social and psychological foundations of OBTE as an approach to teaching and learning.

3.1 Multifaceted Inquiry

Over the course of the investigation, a variety of methodologies were employed iteratively and concurrently to help achieve a deeper and multifaceted understanding of the principles and practices of OBTE (CMMI Product Team, 2009; Cresswell, 1998; D. Kirkpatrick, 1994; Miles & Huberman, 1994; Mâsse, Moser, Stokols, et al., 2008; Tashakkori & Teddlie, 1998; Trochim, 2004; Trochim, Marcus, Mâsse, Moser, Weld, 2008). In addition to the COMPASS process (Chapter 2), we analyzed surveys of participants in the AWG’s Combat Applications Training Course (CATC) (Chapter 6), observed CATC-trained instructors and their students in rifle marksmanship training (Chapters 8 and 9), surveyed alumni of CATC about its downstream impact in theater (Chapter 10); observed potential applications of OBTE in a classroom-based instructor education course (Chapter 13), and identified critical issues pertaining to organizational change (Chapter 14) and leader development (Chapter 15). Our development of a grounded theory for OBTE also has been informed by several other concurrent activities. In particular, we interviewed progenitors and practitioners of OBTE and discussed relevant scientific research with them, attended AWG workshops and seminars on OBTE, reviewed documents from the AWG, observed AWG instructors in CATC, participated in CATC, interviewed stakeholders in the Army Training and Doctrine Command (TRADOC), and

51 Riccio & Darwin


participated in various non-AWG conferences and workshops in which OBTE was addressed. These sources of evidence are briefly summarized below. The COMPASS process and its products guided and facilitated qualitative crystallization across these multiple methodologies (Camic et al., 2003; Denzin & Lincoln, 2003; Ellingson, 2009; Wise, 1980). The crystallization involved more than constant comparison among different kinds of evidence. There also was a balance and tension among the perspectives of a variety of experts with substantially different backgrounds in a dialectic that extended over more than a year (Figure 1). The descriptions of OBTE provided in this chapter reflect the claims, insights, and clarifications that emerged from this multifaceted inquiry.

Figure 1. Balance and tension of different perspectives in the collaborative inquiry.

3.1.1 Interaction with Progenitors of OBTE The military experts involved in our collaborative inquiry included both active duty and retired Army noncommissioned officers (NCO) and commissioned officers associated with the AWG who had been involved in the development of their initiative in OBTE. The most striking aspect of this collaboration was that it did not involve a common division of labor across topics based on assumed relevance of background. Instead, all perspectives were brought to every aspect of the inquiry on a level playing field in which there was no place for assumptions of hierarchy or dominance relations among different types of experience or expertise. The role of the technical experts was not merely to transcribe the opinions of the operational experts but, instead, to interpret their perspectives in terms of familiar scientific assumptions and paradigms. Similarly, the operational experts interpreted the putatively relevant science in terms of exigencies of the Full Spectrum Operations and preparation of Soldiers for it. In additional to seeking common ground, the intent was to influence the sense making within the multidisciplinary team in ways that were visible and traceable over an extended period of time. The elaboration of the principles, practices and measures of OBTE summarized in this chapter

Officer Perspective

NCO Perspective

Engineering Perspective

Scientific Perspective

SOF Perspective

Perspective of Conventional

Forces



(see also Chapters 2, 4, and 5) are the result of these extended and iterative deliberations. This process of crystallization is typical of the approach we have used in other research on Army training and education (e.g., Riccio, Sullivan, Klein, Salter, & Kinnison, 2004; Sidman, Riccio, Semmens, et al., 2009). Moreover, this kind of collaboration is representative of the organizational environment of the AWG from which the initiative in OBTE emerged. 3.1.2 AWG Documents on OBTE Various members of the AWG had prepared documents for internal coordination and for presentation to stakeholders outside the AWG. Four documents were particularly influential in our identification of assumptions and claims about OBTE (Darwin, 2008a, b; Cornell-d’Echert, 2009a, b). These documents have been widely distributed among stakeholders in Army training and education. Over a period of two years, one or more of these documents has been presented in numerous forums in which there was opportunity for feedback and discussion of the meaning and implications of OBTE at particular sites and in particular programs. While there were many contributors to these documents, there was a clear first author and main contributor for each set of two documents. Both of these authors are progenitors of OBTE. One of the authors of this chapter, a retired Command Sergeant Major from the U.S. Army Special Operations community, took the lead in preparing and maintaining a working document comprised of approximately 100 PowerPoint slides. Darwin (2008a) is an early version, and Darwin (2008b) is a more recent version of this document (the document is updated periodically). The intended audience is primarily senior leaders associated with specific TRADOC units and programs of instruction. The early version (Darwin, 2008a) laid out the purpose, vision, and philosophical underpinnings of CATC. The later version (Darwin, 2008b) extended the concepts more explicitly beyond marksmanship training to OBTE in general and revealed the relationship between OBTE and doctrinal requirements for adaptability in Full Spectrum Operations. A retired U.S. Army Lieutenant Colonel took the lead in preparing documents intended primarily for personnel who are responsible for design, development, and implementation of specific Army courses. One of these documents (Cornell-d’Echert, 2009a) presents twenty-six “articles” for OBTE that describe its intent and attempts to clarify misconceptions about the place of OBTE in broader Army culture and practices. The other document (Cornell-d’Echert, 2009b) is an implementation guide that begins to translate the twenty-six articles into practices and considerations for adapting the approach for existing programs of instruction. It is intended as a living document to guide peer-to-peer sharing of best practices and lessons learned. 3.1.3 Collaborative Reflection on Participant Observation in CATC Four members of the research team participated in the one-week CATC as students. One pair of researchers took the course one week, and another pair took the course another week. Each pair included one scientist and one person with military experience. The researchers in each pair were encouraged to discuss their experiences with each other as the week unfolded and to document key experiences and insights to the extent possible after each day of instruction. After completion of the course, another member of the research team interviewed the participant-observers. The interviews were conducted with two participant observers at a time, one from each of the prior pairings; the two scientists were interviewed together, and the two retired military officers together (one had been an Army Infantry Captain, and the other was a retired U.S. Marine Major). The primary role of the interviewer was to stimulate dialog between the interviewees in which they would compare and contrast their respective experiences and the meanings they made of it. This dialog facilitated subsequent discussions with progenitors of OBTE, instructors in CATC, and participants in CATC.

53 Riccio & Darwin


3.1.4 Interaction with Stakeholders There were two main contexts in which there were opportunities for substantial and extended interaction with OBTE stakeholders in Army training and education (See Chapter 1, section 1.3.2). One context involved the AWG seminars and briefings to individuals at sites associated with particular programs of instruction. Members of the research team attended several of these briefings. Another context was our preparation for field-based observations of the implementation of OBTE at Fort Benning, Fort Jackson, and Fort Sill (see, e.g., Chapters 8, 9, and 13). We also have interacted with many other stakeholders through various AWG events (e.g., AWG, 2009; see Chapter 11, section 7.2.3).

3.2 Essential Characteristics of OBTE The most important contributions of the AWG’s initiative in Outcomes-Based Training & Education (OBTE) and the associated research are the claims and evidence that (a) the gap between Army instruction and Full Spectrum Operations can be viewed in terms of the relationship between a singular focus on tangible, near-term objectives and the resulting effect on enabling intangible attributes valued by Army leaders, (b) everything that is done in Army training and education has an impact on this relationship whether one addresses it or not, (c) this impact can be understood and described thus allowing leadership to make informed decisions based on verifiable observations and valid scientific reasoning, (d) OBTE is an approach that can be utilized to inform leaders at all levels on the direction their unit is moving in the development of leadership traits, problem solving skills and intangible attributes, (e) the approach can be verified and validated, and (f) it can influence without necessarily replacing any extant instructional methodology (M. Darwin, personal communication, June 30, 2009). 3.2.1 The Meaning of Developmental is a Critical Difference OBTE is an approach to training and education that focuses on development of the individual in relation to basic needs of the individual as well as associated cultural values and objectives of the individual’s occupation (Darwin, 2008a, b; Cornell-d’Echert, 2009a, b). The meaning of “developmental” thus is fundamentally different from its typical connotations in training and education, which emphasize individual differences and, in many cases, refer to remedial learning in specific subject matter. This is not to say that OBTE is unconcerned with Soldier’s mastery of specific knowledge and skills. An assumption of OBTE is that longer-term developmental outcomes can be pursued while Soldiers are being prepared for specific tasks (Cornell-d’Echert, 2009b). In OBTE, immediate and long-term objectives are not considered fundamentally inconsistent. There may be tradeoffs about nested instructional objectives to be made on the scale of hours in particular learning events to the scale of months or years in curriculum design, but this more likely would be a question of return on investment rather than incompatibility (see Chapter 5 and Epilogue). More generally, OBTE is an approach in which persistent awareness of long-term outcomes, including but not limited to the explicit learning objectives for an event, influences the planning and execution of instruction in important ways. Why consider Soldier development in training and education when it seems to be the purview of other programs, the responsibility of other people, or a focus for time periods outside of formal learning events? The reason is simple and inescapable. Every interaction with a student influences long-term development whether or not it is specifically considered in planning or execution of instruction. The profundity of this human factor in training and education has been recognized from antiquity to modern times (e.g., Aristotle, trans. 1925; Augustine, trans. 1953; Bandura,



1977; Dewey, 1915/2008; Guay, Ratelle, & Chanal, 2008; James, 1890/1907; Keatinge, 1896; Lave & Wenger, 1991). OBTE takes into consideration that an individual can be changed by the learning experience in ways that transcend the particular knowledge or skills acquired during a learning event. On this philosophical foundation in learning and development, OBTE stridently rejects any separation of learning and doing. This is not merely a superficial claim about experiential learning (i.e., that one learns by doing). OBTE claims that, while applying knowledge or skill in practice, individuals also can learn about other things that can be done or that can be known. Individuals weave performatory behavior with exploratory behavior to achieve a more differentiated and more refined understanding of their capabilities and of the environment in which they are situated (J. Gibson & E. Gibson, 1955; E. Gibson, 1988, 1991). This weave of performance and discovery essentially is an inter-temporal compromise between immediate and long-term goals of behavior (cf., Glaser, 2002; Riccio, 1993a). In general, inter-temporally directed behavior often involves minimal tradeoffs to the extent that exploratory behavior can be nested within performatory behavior with little or no interference (cf., Feldbaum, 1965; Filatov & Unbehauen, 2004; Riccio, 1993a, b; Riccio & McDonald, 1998). This is addressed in Chapters 4 and 5 in the context of individual adaptability and collective agility. OBTE claims that there are ubiquitous opportunities for instructors and students to engage in thinking or activity beyond the task at hand, amid the activities required by the task at hand. These opportunities exist on the scale of seconds to minutes. Moreover, OBTE claims that instructors and students often are engaged in such apparently incidental behavior, that is, behavior not prescribed by the task at hand. The less esoteric of version of this claim is that human beings are not inanimate objects or machines even when they are implicitly or explicitly treated as such in the development or execution of instruction. Our informal observations in the field revealed that, when students are engaged motivationally in a learning event, their thinking and behavior outside the task at hand relates to the task at hand. For example, under favorable conditions, students will spontaneously talk with each other about performance on a task even if they are not required to do so. Such ad hoc engagement in learning does not introduce any delays in the learning event. Under less favorable conditions, something unrelated to the learning event will capture the attention of students in the time periods between behavioral scripts. In both cases, habits of learning and engagement are nurtured, for good or ill. Rarely if ever, is each student involved in scripted behavior at every second of every learning event, especially when the student-to-instructor ratio is high. An assumption of OBTE is that, during waiting time (e.g., in the ubiquitous queues), students are making sense of their experience in a learning event. This does not necessarily involve overt or even explicit intentional analysis of the event. In the extreme, it may involve nothing more elaborate than the conclusion that there is little sense to be made of the event but the consequences for human development are no less profound. OBTE seeks to engage such metacognitive activity to have a more purposeful influence on development. As discussed below, this can be achieved in the approach to planning and execution of instruction without necessarily adding tasks for the instructor or the student. At a high level, OBTE influences metacognitive activity by addressing the reasons behind the structure of a learning event and the constraints on its validity. It also does this by providing models (i.e., real-time examples) of collaborative reflection and reinforcing it wherever appropriate. This does not require an instructor to be everywhere at all times or to help every student exploit every opportunity for useful reflection. It merely requires some examples of collaborative reflection that students can model whether they participated in the examples or experienced them vicariously.

55 Riccio & Darwin


Even occasional engagement of students on a metacognitive level is powerful because this is where ownership of their own learning and development resides. This should not be interpreted in ways that imply an infinite regress in analysis of one’s own thinking and behavior. Fundamentally, it is about the deeply personal struggle between the known and the unknown, between the controllable and the uncontrollable. It is where passion and faith intermingle with reason and belief (Figure 2) (Kierkegaard, 1843). Through development, it is where individuals internalize previously external sources of motivation and where they make meaning of the balance between what they can control and what they are either unwilling or unable to control, that is, with respect to which they yield or cede power (cf., Aristotle, trans. 1925; Augustine, trans. 1991; Hoffman, Yang, Kaklauskas, & Chan, 2009; Idel & McGinn, 1999; James, 1902/1982; Maslow, 1964; Plato, trans. 1892a; Solomon, 2004). Engaging Soldiers on this level helps them find a balance between choice and responsibility that is personally sustainable and more likely to help them be successful in the ambiguous and changing conditions of Full Spectrum Operations. Students will find their own levels but it is in the interest of future commanders, peers, and subordinates for Soldiers as students to understand and continually develop their relationships with various sources of power that influence their decisions and actions, beyond the formal or explicit lines of authority within their units. For this reason, as discussed below, OBTE motivates the development and use of formative measures that engage students and instructors on this level (see Chapter 2).

Figure 2. Intangible metacognitive and metaphysical influences on capabilities for thinking, decision-making, action, and their development.

With respect to development and assessment, OBTE is related to approaches in education that focus on progress maps, especially those that consider multidimensional constructs including latent variables (Pellegrino, Chudowsky, & R. Glaser, 2001). The latent variables of OBTE, however, are not mathematical constructs or other abstractions derived computationally from quantitative data. They are based on assumptions about shared values and qualitative evidence about an individual’s values that manifest in both mundane and extraordinary situations (cf., Aspinwall, & Staudinger, 2003; Keyes & Haidt, 2003; Peterson & Seligman, 2003; Riccio et al, 2004; Webster, 2001). OBTE considers Soldier development to be the most powerful focus for any approach to Army training and education insofar as it addresses attitudes and values that guide life-long learning and self-development. More than anything else, this focus engages instructors as leaders and leader developers. In this sense, OBTE essentially is both values-based and leadership-based training and education. It thus helps bring what the Army does best into the



realm of training and education (United States Army Training and Doctrine Command (TRADOC], 2009). It seeks to give training and education a higher status and sense of calling (Riccio et al., 2004). 3.2.2 The Definition of Outcomes is a Critical Difference The current revision of TRADOC Regulation 350-70 tries to provide guidance on the selection and use of a variety of instructional methodologies and associated theories of education. In this respect, it is important to note that OBTE is more general and inclusive than a particular method of instruction or a prescriptive set of instructional techniques. At the same time, from the perspective of OBTE, not all methods are equal. OBTE is not, for example, consistent with “outcomes based” methods of instruction that focus exclusively on course-specific objectives. OBTE should not be confused with constructs that are based on a definition of outcomes that is more like the definition of common learning objectives and that implies a different developmental approach. While the “outcomes based education” movement of the 1990s (see e.g., Spady & Marshall, 1991; Spady, 1994, 2009), for example, stressed more of a focus on the objective output than on an a priori script or input focus for an instructional event (Figure 3), it is inconsistent with OBTE on a fundamental level in that it rejects values, attitudes, internal mental processes or psychological states of mind as outcomes (Spady, 1994, 2009). While reverse planning is a common aspect of different approaches to outcomes-based instruction, OBTE starts with different kinds of outcomes, ones that relate to attitudes and attributes developed in any learning event whether or not one explicitly recognizes it. Another fundamental difference is that OBTE is developed for situations, as in the Army, in which there are practical constraints on time and resources. In the Army, there is not the luxury to allow students to achieve a pre-specified level of performance at their own pace. OBTE requires an inter-temporal focus that balances immediate learning objectives of the task at hand with longer-term habits of thinking and action that transfer to different situations. In this respect, it is not wise to give high performers short shrift so that relatively low performers can be given more assistance with their performance. All Soldiers can benefit from being engaged on the level of metacognitive development (section 3.2.1) irrespective of their level of knowledge or skill. Currently there is considerable confusion in the use of the terms objectives and outcomes. Sometimes they are used synonymously, sometimes not, and there is a conspicuous lack of consistency in the use of each of these terms. An outcome in OBTE is not the same as a “terminal learning objective” that typically is the focus of Army training and education. We equate objectives with intended results of a well-defined component of training and education, whether short-term or long-term. We refer to outcomes as all results of a well-defined component of training and education, whether intended or not. In general, intentions are specified explicitly only with respect to short-term results, at least in any way that is verifiable or measurable; thus use of the term objectives often connotes short-term results. When verifiability of objectives is considered, the focus is almost exclusively on tangible results. The use of the term outcomes in OBTE emphasizes long-term results that, of necessity, are mostly intangible but that also can become verifiable and measurable, in conjunction with traditional short-term results. One way to clarify the use of the terms, objectives and outcomes, in OBTE is to begin a rigorous and traceable inquiry into the relationship between verifiable intangible long-term results (outcomes) and verifiable tangible short-term results (a priori objectives and ensuing outcomes). The first step in this inquiry is to identify ways to talk about and measure both tangible and intangible results in which they are commensurable. Then one can begin to identify the antecedents of long-term results that can be observed at the same time short-term results are

57 Riccio & Darwin


observed (e.g., Deci & Ryan, 2008; Peterson & Seligman, 2004; Rick & Loewenstein, 2008; Riccio et al., 2004). This would enable identification of conflicts and synergies in the means to achieving both short-term and long-term results. We believe that a promising line of inquiry for OBTE is the role of motivation and emotion in tradeoffs involving intangible long-term outcomes that influence perception and action (Camber, Loewenstein, & Prelec, 2005; Loewenstein & Lerner, 2002; Loewenstein, Weber, Hsee, & Welch, 2001; cf., Deci & Ryan, 2008; Ryan & Deci, 2008; Peterson & Seligman, 2004). As demonstrated by the measures of Chapter 2, intangibles should not necessarily be interpreted as fuzzy concepts. They are only intangible in terms of their manifestations under the uncertain conditions of the future (cf., Rick & Loewenstein, 2008).

Figure 3. Highly stylized depiction of three approaches to training and education. Dotted horizontal line represents target performance associated with the terminal learning objective for a learning event. Green lines represent relatively fast learners; red lines represent relatively slow learners. Gray band indicates uncertainty about retention of skills or knowledge or transfer of learning to a new situation.

It may be helpful to consider the antecedents of long-term outcomes as analogous to “enabling learning objectives” typically associated with terminal learning objectives in Army training and education. This suggests a potential synthesis by recognizing that objectives are nested (cf., Feldbaum, 1965; Filatov & Unbehauen, 2004; Glaser, 2002; Riccio, 1993a,b; Riccio & McDonald, 1998; Riccio & Stoffregen, 1991), perhaps in ways that are directly analogous to



nested intent statements in mission command (Cornell-d’Echert, 2009b; cf., Appendix C; Freeman et al., 2008). Longer-term attitudes and values can be addressed, along with immediate learning objectives, in the context of the same learning event. This approach seems timely given the exigency in the most recent version of Army leader development strategy to integrate leader development with training and education (TRADOC, 2009). Toward this end, we believe it is essential to make more progress in identifying scientific underpinnings for OBTE. The development of measures for instructors helped us recognize some of these connections (see Chapter 2). In this chapter, we begin to describe the research that followed the development of measures, that helped us refine and organize them, and thus that helped us realize a theoretical commitment for a rigorous line of inquiry (see Chapters 4 and 5). 3.2.3 The Emphasis on Values and Causally Potent Intangibles is a Critical Difference The principles of OBTE (e.g., Darwin, 2008a, b) reveal the kinds of outcomes that can be addressed during any learning event. The principles also suggest why these outcomes are important with respect to retention and transfer of learning to operational environments (i.e., sub-bullets in list of principles below).

• Training to Grow Problem Solving o teach Soldiers to “learn for themselves” the skills necessary to the success of

their mission, within an established framework of knowledge • Training to Increase Intangibles

o develop intangible attributes like confidence, accountability, initiative, judgment and awareness; reinforce positive character traits

• Training to Increase Understanding and Awareness o teach through contextual understanding of the task and its mission application

• Training to Increase Deliberate Thought o condition Soldiers to exercise always a deliberate thought process (evaluation,

judgment and decision) while under stress • Training to Improve Combat Performance

o condition Soldiers to overcome the psychological and physiological effects, and the physical requirements of combat

A focus for our interactions with the progenitors of OBTE was to develop a better understanding of the causal potency of these principles. The most significant discussions revolved around the so-called “intangibles” of OBTE. This collaborative inquiry involved constant comparison with diverse lines of research in psychology and related disciplines that have a long history of scholarship with respect to intangible attributes and latent causal variables. While such interdisciplinary connections can be as complex as one would like them to be, we chose to identify a simple framework for sustained trans-disciplinary communication and collaboration that is well grounded in a variety of disciplines across the physical, biological, behavioral, and social sciences. In particular, we chose a triadic framework within which a balance and equilibrium of two causal factors is manifested with respect to a third factor (see e.g., Burke, 1985; Lorentz, Einstein, Minkowski, & Weyl, 1923/1952; Glass & Mackey, 1988; Goldberg, 1986; Heisenberg, 1958; Lanczos, 1970; Wolfram, 2002). The point here is not to equate our current scientific understanding of OBTE with these well-developed branches of science. Instead, the point is that considerable explanatory power with respect to complex phenomena can be achieved from relatively simple concatenation operations if one starts with primitives and

59 Riccio & Darwin


principles that are trustworthy. Such confidence typically derives from a history of careful observation of phenomena from which primitives and principles are abstracted. We believe a triadic framework is the simplest possible expression of the essential features of OBTE and thus is most useful in translating theory into practice. The primitives and principles in this framework are based on decades of experience of a variety of people who have approached Army training and education from different albeit converging perspectives (Figure 4). The nascent concatenation operations of balance and tension between primitives are based on this experience as well as on themes from a variety of philosophical perspectives that have stimulated thought and dialogue for millennia. Some of the important implications of a triadic framework are discussed further in Chapters 4 and 5 and in the Epilogue.

Figure 4. Triadic balance and equilibrium among intangible personal attributes and the associated scaffolding for capabilities of the human dimension in Full Spectrum Operations and competence in trans-extremis environments (e.g., transitions between strategically significant combat and noncombat operations).

Early on in the collaborative inquiry with progenitors of OBTE, we identified three intangible attributes as foundational for OBTE. Confidence, initiative, and accountability became the organizing framework for outcomes that are given special attention in OBTE. We believe that any two of these attributes has an impact on the third attribute but, for power and simplicity, we emphasize the balance between initiative and accountability. The power of this dyad can be understood in terms of the interplay between choice and responsibility (Aristotle, trans. 1925; Augustine, trans. 1991; Hickman, L.A. & Alexander, 1998; James, 1890; Solomon, 2004) or between agility and the variable contextual constraints of Full Spectrum Operations (HQDA, 2003; 2005). This triad emphasizes the urgency of choice within contextual understanding of military situations, with particular attention to the powerful aspect of initiative coupled to moral accountability, as compared to ethical responsibility (cf., Lévi-Valensi, 2006; Clausewitz, 1976 trans.; Kierkegaard, 1843). There are any number of motivational contrivances and tricks that can be employed to make individuals more flexible or to display more initiative but this is not what the Army or the Nation needs of its Soldiers. Soldiers need to be adaptable in initially ambiguous



and in changing conditions. Adaptability implies that their decisions and actions are appropriate given the context. At the same time, the Army needs Soldiers to exhibit cunning and not merely to be driven by context. There is a need for units to be agile, proactive rather than reactive, and to recognize the functional potency of human agency in social structures. The balance of agility and context is not merely about recognition of the limits of power. It also is about an appreciation of the consequences of exercising power (cf., Clausewitz, 1976 trans.). Consequences include longer-term effects, including unintended effects, as well as impact on the task at hand. OBTE assumes that habits of such inter-temporal awareness and thinking can be developed in any and all learning events, in formal learning events or informal life-long learning. This is an important determinant of the confidence the Army needs in its Soldiers, not a false confidence bred from ignorance, invalid assumptions, or unexamined misinterpretations. This is not to imply that Soldiers must know everything at all times. To the contrary, OBTE assumes that habitual experience with the interplay between initiative and accountability helps Soldiers come to terms with uncertainty and the application of knowledge and skill amid uncertainty. While this is a deeply personal experience, in OBTE, it is not a solitary endeavor. OBTE builds habits of collective struggling with uncertainty and collaborative experience even in the context of ostensibly individual tasks. There are other intangibles emphasized in OBTE. To maintain simplicity, such intangibles are envisioned in terms of a triadic infrastructure (Figure 4). This conceptual nesting should be viewed as analogous to the grouping of causal factors in a factor analysis. The nesting is not intended to represent all the relationships or to “explain all the variance.” It is intended to be the least arbitrary grouping that achieves the simplest possible description of a myriad of concepts. By simplest, we mean the easiest to put into practice. In this sense, every attribute can be addressed through the interplay of two other attributes. We explored a triadic infrastructure in another project to simplify inquiry into the inculcation of Warrior Ethos in Initial Entry Training (Riccio et al., 2004; Brunyé, Riccio, Sidman, Darowski, & Diedrich, 2006; Appendices D and E). We found that this framework also helped achieve a useful mapping among a variety of values-based concepts in current Army doctrine. Again, while such relationships are not intended to suggest one-to-one mappings or a full understanding of each concept, they do alleviate unnecessary complexity in important sets of Army requirements and thus suggest ways in which the requirements have actionable implications for Army training and education. Along those lines, the mapping has recently been extended to yet another set of values-based requirements in Army training and education (Sidman, Riccio, Semmens, et al., 2009). The composite triadic mapping is indicated in Table 1. Table 1. Triadic mapping among plethora of concepts in Army training and education.

OBTE Attributes of Tenets of Army Pentathlete Intangibles Warrior Ethos Warrior Ethos Values Characteristics

Confidence Perseverance Courage Warrior Leader Initiative Adaptability Respect Judgment Prioritization Mission First Duty Critical Thinker Deliberate thought Tradeoffs Never Accept Defeat Honor Creative Thinker Accountability Sense of Calling Integrity Leader Developer Discipline Responsibility Never Leave Fallen Selfless Service Resource Manager Awareness Dependence Never Quit Loyalty Ambassador

61 Riccio & Darwin


The triadic mapping between the intangibles of OBTE and the attributes of Warrior Ethos is most striking. This is important because the focus of the project that generated the attributes of Warrior Ethos was to provide valid and verifiable ways to assess progress in inculcating Warrior Ethos in Soldiers during Initial Entry Training. A promising line of inquiry is the adaptation and utilization of the attributes of Warrior Ethos as measures of long-term development, perhaps even standards, that relate to values-based requirements in current Army doctrine (see Epilogue). 3.2.4 The Meaning of Experience is a Critical Difference In OBTE, experience refers to more than just a personal history of participation in learning; that is learning by doing. To be sure, participation in one’s own learning is central to OBTE; however, participation also is the best way to experience learning and to find meaning in it. Experiencing the conditions, process, and consequences of learning is important. In other words, consistent with the philosophy of John Dewey, it matters what the student experiences during learning (Hickman & Alexander, 1998). In OBTE, it is particularly valuable if the learning experience includes reflection on the dynamic interplay of attributes such as initiative and accountability, the confidence one derives from the balancing of these attributes, the resulting capacity to navigate the unknown with what is known, and negotiation between internal and external sources of power. While this may seem like an existentially significant or cognitively intense undertaking, it need not be. That it can be more mundane is, of course, critically important in practice. That it should become somewhat routine or ordinary is the point of OBTE (cf. Riccio et al., 2004; Appendices D and E).

Figure 5. Development of thinking in action and thinking about action in outcomes-based training and education [from Darwin, 2008a].

An essential feature of experience in OBTE is that it is inter-temporal (see e.g., Figure 5). There is the student’s experience during participation in the designed learning event, and there is the student’s experience outside the context of this required participation. The latter typically is neglected. It can be neglected either unintentionally or even by design based on priorities or expectations that do not associate intangible attributes with the performance or learning of a task or subject. One challenge is that instructors typically are not involved with every student during a scripted learning event, thus it may appear that instructors cannot have an influence on it. Another



challenge is that discipline may preclude overt activity of students when they are not involved in a scripted learning event (e.g., during waiting time), or the layout of the range or the classroom may make the informal activity of students difficult to observe. In neither case would it be accurate to assume that no thinking or communication is occurring. If neglected, the thinking that occurs during such time periods may not be favorable to learning and retention of the knowledge or skills addressed in the formal learning event. In any case, there is an opportunity to stimulate reflection during these frequent periods of time, however brief, to help students make values-based connections with experience in the learning event, and thus to influence longer-term outcomes. OBTE aggressively and systematically seeks to exploit such micro-experiences to build habits of inter-temporal thinking. We believe that micro-experiences are a significant untapped or misused resource in most learning events (M. Darwin, personal communication, May 5, 2009). 3.2.5 The Emphasis on Instructor-Student Interactions is a Critical Difference The most concrete findings from our inquiry into OBTE are best practices associated with interactions between instructors and students (Chapter 2, Appendix A). These practices are not expressed as immutable scripts but as guidelines for the ongoing pursuit of mastery in the art of teaching (cf., Augustine, trans. 1953; James, 1890/1907; Keatinge, 1896). The essential aspects of this art in OBTE are depicted in Figure 6.

Figure 6. Three critical roles of an instructor as a leader, leader developer, and influencer of human development [From Darwin, 2008a].

Instructor-student interactions seem to be relatively neglected in the science of education, presumably because it is considered to be an art and something that can only be addressed in the field, or perhaps because it is deeply and broadly embedded in apparently esoteric research of scientific psychology. We agree that acumen in working with students is something that can only be learned in the field but we reject any notion of a strict separation between the art of teaching and the science of psychology (cf., James, 1890/1907). Part of the intent of OBTE, and of chapters 3 through 5 in particular, is to reorient Army training and education to theory and data from relevant subdisciplines in scientific psychology. Our position is that the design,

63 Riccio & Darwin


development, and implementation of instruction can get considerable practical guidance from a deeper understanding of recent research and historical inquiry in scientific psychology that addresses the reciprocal influence between individuals and their environment and the impact these interactions have on the development of motivation, personal identity, and values. 3.2.6 The Emphasis on Learning to Learn is a Critical Difference The emphasis of OBTE on instructor-student interactions begs questions about how to enable this through staff and faculty development (see Epilogue). We don’t believe that this implies a necessity for additional instructor education in established theories of education with which OBTE seems to have most in common, such as experiential education (e.g., Hickman & Alexander, 1998; Kolb, 1984), constructivist education (e.g., Hmelo-Silver, 2004; Tobias & Duffy, 2009), or adult education (e.g., Jarvis, 2004; Knowles, Holton, Swanson, 2005). We believe it is important to return to the foundations for educational theory in psychological science both historically and in terms of significant trends in psychological science over the last thirty years (see Chapters 4 and 5). Partly for this reason, and to avoid premature conflation of OBTE with extant theories of education, the progenitors of OBTE have chosen to orient their principles and practices with respect to a simple framework for education that has endured for millennia (Figure 7). Interestingly, this framework suggests an approach to Army training and education that is reminiscent of a liberal arts education rather than a pre-professional education.

Figure 7. Conceptualization of outcomes-based training and education in terms of the Trivium and the spiral development of individual competence based on continuing experiences [from Darwin, 2008a].

It could be argued that one of the most important objectives of a liberal arts education is to help students acquire learning skills inside the classroom that will help them learn outside the classroom. OBTE assumes that, in the Army, such skills center around learning collaboratively with others (cf., Lave & Wenger, 1991; Wenger, 1998). Collaborative experience and inter-subjectivity thus is important in OBTE, not just in transactions between instructor and student but also in interactions between peers. Peer-to-peer dialog is valuable for several reasons. It provides students with opportunities to learn with, through, and from another person who is not necessarily



in a position of authority. Thus the student can learn in a formal learning event how to learn outside the context of formal learning events, that is, in situations in which an authority figure is not available to tell individuals what to do. It also provides students with the experience of transitioning between momentary context-specific roles of leader and subordinate. This skill is invaluable in task organization of groups in ambiguous or changing conditions.

Learning to learn is directly relevant to notions of adaptability (see Chapter 4). There seems to be different meanings of adaptability in the institutional and operational Army. In an attempt to bridge this gap, we offered a more axiomatic formulation of the problem of adaptability in a workshop held by the University of Southern California and Institute for Creative Technologies, in April 2009. 1) The vast majority of learning in the Army occurs on the job or in ad hoc instructional events.

a) Instructional events designed and delivered by leaders at the company level and below. b) Instructional objectives driven by exigencies of the day and influenced by priorities

implicit in intent of higher commanders. c) The implication is that most learning happens outside the context of curricula and courses

that result from the Army's formal instruction system design (ISD) process. d) Need: What can be done in the institutional Army to develop curricula and courses that

improve the quality of instruction and learning outside this formal context? 2) The amount of operationally relevant learning occurring in formal curricula and courses in

the Army has been increasing. a) This is partly because of greater and more prevalent experience of instructors in the full

spectrum of operations (i.e., offense, defense, stability, support) b) This is partly because of the intent of leadership in Army schools to utilize experience of

instructors and close the gap with the operational Army. c) It is accepted that lessons learned from experience in Full Spectrum Operations can be

valuable in the classroom. d) Need: How can we determine what kind of experience is useful in the classroom and how

it should influence curriculum development and instructional activities? 3) The most fundamental unit of analysis in the Army is the group not the individual.

a) Adaptability and the associated agility of groups in the Army is exemplified and fostered by task direction and organization (i.e., ad hoc task-specific division of labor that takes into account what is known about group members).

b) Adaptability and the associated agility of groups in the Army is exemplified and fostered by ad hoc prioritization and tradeoffs (e.g., typically there is more to be done than can be done given time and resource limitations) that take into account understanding of one's responsibility to others in the group and one's dependence on them.

c) Need: What are the implications for curriculum development (e.g., nature and extent of instructional guidance) if we assume that Soldiers must become better at utilizing and eliciting guidance outside the classroom to achieve collective agility rather that assuming that they need to become less dependent on guidance in moving from the classroom to the job?

In OBTE, micro-experiences typically are inter-subjective as well as inter-temporal. The experiences create a habitual orientation to shared values and long-term outcomes. This has implications about the consideration of individual differences in personality or learning style. In particular, it suggests that there is some value to acquainting students with the effects of individual differences in collaborative work (see e.g., Allik & McCrae, 2004; de Raad &

65 Riccio & Darwin


Perugini, 2002; Heine & Noranzayan, 2006; Hofstede, Pedersen, & Hofstede 2002; Matsumoto &Yoo, 2006) insofar as it could help them with task organization and the collaborative agility it enables. Interestingly, the assumptions about adaptability in OBTE also suggest that too much accommodation for individual differences during a learning event may be counterproductive. The actors and trans-extremis conditions in Full Spectrum Operations are not likely to accommodate individual differences in ways that instructors can, in principle. Thus, careful thought about downstream consequences should be given to the treatment of individual differences in Army training and education. 3.2.7 The Emphasis on Collaborative Design and Development is a Critical Difference In Army training and education, historically, there has been a rather strict division of labor between analysis, design, development, implementation, and evaluation (ADDIE). OBTE argues for less separation and more collaboration across these curricular activities and responsibilities (see e.g., Chapters 14 and 15; Cornell-d’Echert, 2009a, b). In particular, we believe that instructor cadre should be included in local decisions involving prioritization and tradeoffs about the extent to which various skills and knowledge can be addressed in a program of instruction as well as when and how they should be addressed. This helps establish a command climate in which mutual trust is both highly valued and salient (cf., Bandura, 1995; Edmonson, 2003; Vaughan, 1997; Rasmussen, 1997). It helps an instructor take ownership of the instructional environment; and it helps them take responsibility for instructional effectiveness rather than merely nominal efficiency. The instructor is able to prioritize on the task set forth by the Army, to the standard set forth by the Army, but in a manner that permits adaptation of the instructional environment to the uniqueness of the instructor’s situation. OBTE maintains that this results in a positive, long-term outcome for instructor and student. Figure 8 depicts a process that has emerged in the adoption of OBTE in various programs and at various sites in the Army (e.g., Appendix C; Perry & McEnery, 2009). Commanders in instructional programs have established a command climate and communicated an intent that fosters collaborative discussion and decision-making by instructional cadre and their chain of command to achieve a balance of near-term learning objectives and the long-term developmental outcomes emphasized by OBTE. These commanders are influenced heavily and self-consciously by a belief that there is a gap between the historical priorities of institutional training and emerging operational needs such as preparation for the ambiguous and changing trans-extremis conditions of Full Spectrum Operations. With the participation of their instructional cadre, they establish priorities based on emerging needs of Full Spectrum Operations as defined by the Army, and make tradeoffs based on limited time and resources while still achieving the intent of higher commanders. An important aspect of the decentralized programmatic decision-making occurring in the adoption of OBTE is the identification of outcomes that are observable during training and that have a plausible causal relationship with downstream outcomes in the operational environment (see section 3.2.2). The principles and intangibles of OBTE have been helpful to commanders and their instructional cadre in their collaborative identification of outcomes. The intangibles guide thinking about plausible relationships between near-term outcomes and long-term development outcomes (see section 3.2.3). The principles of OBTE have been helpful in identifying outcomes that are most important in Full Spectrum Operations and in identification of instructional practices that help develop attributes in Soldiers on which such outcomes depend. Verification and validation is given special attention in such programs. Commanders and their instructional cadre are diligent in observing and reflecting on the developing instructional practices and outcomes in their programs (see sections 3.2.4-3.2.6). In this context, there is considerable



interest in formative measures such as those discussed in this chapter (see also Chapter 2), and there is increasing interest in measures that more directly relate to outcomes of interest in Full Spectrum Operations (see Epilogue).

Figure 8. Collaborative design and development for OBTE in a local command. Solid lines represent direct programmatic influences. Dotted lines connote informal or indirect programmatic influences. Heavy solid lines connote direct psychological causes and effects of instruction on students. For simplicity of depiction, “Command Intent & Collaborative Decision-Making” includes all activities involved in planning, executing, and assessing instruction. Compare with Figure 1 in the Epilogue.

3.3 Toward a Grounded Theory for OBTE

3.3.1 Need for an Integrated Interdisciplinary Framework As an approach, OBTE is multifaceted and multidisciplinary because it addresses (a) the various ways that an organization invests in or otherwise influences the development of individuals and (b) the long-term return on this investment for the individual and the organization. It recognizes that instruction requires careful consideration of the student, the instructor, and the nested structure of the instructional institution and the culture it serves (Bransford, et al., 2000; Dewey, 1915/2008; Hmelo-Silver, 2004; James, 1890/1907; Keatinge, 1896; Lewin & Grabbe, 1945; Magolda, 1999; Riccio, et al., 2004; Wenger, 1998). Effective instruction should have components that have an impact on all these elements and levels of an instructional system. For these reasons, OBTE includes interventions that address individual motivation and well-being, interpersonal interaction and influence, the ecology and development of perception and action, and reciprocal means-end relations within adaptable organizations. We use Bransford et al. (2000) as a framework within which to organize and describe the multidisciplinary considerations implicit in the set of formative measures for OBTE (Chapter 2;

67 Riccio & Darwin


Appendix A). This framework addresses learner-centered, knowledge-centered, and assessment-centered needs of a learning environment that are grounded in a deep understanding of community-centered needs (Figure 9). Bransford et al. (2000) argue that learning systems will be more effective to the extent that they meet these needs.

Figure 9. Exigencies for design of a learning environment that is effective with respect to human development and far transfer [after Bransford et al., 2000].

We believe that this framework facilitates understanding of the capability gaps to which OBTE is a response as well as the way in which it addresses those gaps. It is a broad and integrated view of educational programs that also has implications for program evaluation (see Chapters 1 and 11). In particular, we believe Bransford’s four essential characteristics of a good learning environment ground the development of measures in a way that helps achieve an integration of Kirkpatrick’s four levels of evaluation (D. Kirkpatrick, 1994), integration envisioned by Kirkpatrick but that has proved to be elusive in most implementations of his system (J. Kirkpatrick & W. Kirkpatrick, 2009). At the same time, a concrete integrated approach to program evaluation helps those involved in various phases of the ADDIE process understand specifically what they can do to address the needs of a learning environment as articulated by Bransford et al. One of the most important results of the methodological crystallization in our inquiry about OBTE was an appreciation of the actionable implications of principles and practices of OBTE (Darwin, 2008a, b; Cornell-d’Echert, 2009a, b) for a learning environment consistent with the framework of Bransford et al. The following sections address these opportunities in the learning environment in OBTE. 3.3.2 Formative Measures of Instructor Behavior as Evolving Best Practices of OBTE The product of our COMPASS effort with instructional designers and cadre of OBTE yielded a set of 65 operationally relevant and observable measures (see Chapter 2; Appendix A). These measures define OBTE at a level sufficient for rigorous verification of instructional practices of OBTE and, thus, they are useful for both program evaluation and formative feedback to instructors. It is important to note that the set of 65 measures are intended as a menu from which



one can chose a manageably small subset for a particular learning event. The measures codify the best practices of OBTE that, in essence, are perspectives on how to translate theory into practice. The measures are not specific to one application, such as marksmanship training, and they can be employed to assess instructor behavior in virtually any training domain for quality assurance or improvement. In this respect, there is a value to measures such as these beyond the particular relevance they have to OBTE. It is critically important in quality assurance to have some method of verifying what instructors actually are doing. Absent that, it would be difficult if not impossible to justify conclusions about what to sustain or what to improve based merely on measures of learning or student performance. Validation is useless without verification (CMMI Product Team, 2009); dependent variables are problematic without independent variables. A fundamental premise of OBTE is that instructor-student interactions are the most important factor affecting the learning and development of Soldiers. An enhanced capability for verification of this aspect of instruction is the most important contribution of the formative measures developed in our research on OBTE. The essential characteristics of a good learning environment for OBTE were used to identify categories for the measures of instructor behavior that facilitate selection and use of the measures (see Appendix A). This was accomplished by ranking the principles in terms of their relevance to each of the 65 measures. The measures were then grouped into categories of five to nine items based on similarity (cf., Tversky, 1977) with respect to the four characteristics of a good learning environment. Ten high-level categories emerged including three for planning, six for execution, and one for collaborative reflection. Collaborative reflection currently has three small subcategories that we expect to expand as experience is gained with OBTE in a wider range of applications.

3.4 Emerging Best Practices in OBTE for a Community-Centered Environment To be community-centered, Bransford, et al. (2000) argue that learning systems must encourage exploration in an environment that is relevant to and reflects the community in which learning will be applied. Consistent with this, OBTE strives to emphasize the relationship between what is learned early in training and education and what will be necessary in order to function appropriately in Full Spectrum Operations. Also consistent with Bransford’s framework, OBTE strives to create an open and supportive learning environment that encourages the manifestation and development of leadership. This is important for the community that OBTE serves, given that these are characteristics of a learning environment that reflects Army Values. We believe that the characteristics of a community-centered environment relate directly to leadership and enculturation of Soldiers (see Table 2). 3.4.1 Leadership and Enculturation of Soldiers What can be done to develop curricula and courses that improve the quality of life-long learning and development such as in an operational context where significant and critical learning occurs? One answer to this question follows from the realization that a collective, not an individual, is the most fundamental unit of analysis in the Army. Task organization and division of labor in the Army both requires and provides opportunities for momentary leadership that is not necessarily formal as designated by rank, position or specialty. Leadership and guidance that individuals provide to each other is not limited to formal learning events. The goal should not necessarily be to wean Soldier-students from guidance so that they are competent on their own when they leave formal instruction. It should be to make them more competent in eliciting, utilizing, and

69 Riccio & Darwin


providing guidance within a task-organized unit. Learning events can be designed to develop this competency in individuals and thus to prepare Soldiers for success in Full Spectrum Operations. Table 2: Exigencies of the learning environment in OBTE Leadership and enculturation of Soldiers

• Inculcation of Warrior Ethos and Army Values • Instructor is a leader, facilitator, advisor, mentor, and role model • Instructor gives purpose & vision, allows passion & risk, fosters growth • What is taught and how it is taught reflects necessity of Full Spectrum Operations • Community-centered learning environment

Integrated understanding of basic Soldier skills in Full Spectrum Operations

• Condition Soldiers to overcome the psychological and physiological effects of combat • Condition Soldiers always to exercise deliberate thought under stress • Demonstrate the linking of tasks in a military situation • Understand relationships between what is taught and why, when, and how it is taught • Knowledge-centered learning environment

Collaborative reflection and problem solving

• Training to grow problem solving • Teach Soldiers to “learn for themselves” within an established framework of knowledge • Teach through contextual understanding of the task such as its mission application • Draw out of the Soldier a critique of performance during the process • Assessment-centered learning environment

Soldier motivation and development of intangibles

• Training to develop intangibles such as confidence, initiative, and accountability. • Assist the Soldier to understand the situation and desired result • Assist the Soldier in identifying obstacles to the desired result • Allow the Soldier to work towards a solution within defined principles • Learner-centered learning environment

Given that there is an abundance of Soldiers with recent and relevant experience in Full Spectrum Operations, how can we determine what kind of experience is useful in the classroom? OBTE assumes that the most valuable experience is in the development of individuals. The outcomes emphasized by OBTE are long-term developmental trajectories and milestones (B. Glaser, 2002; Rick & Loewenstein, 2008; Peterson & Seligman, 2004; Deci & Ryan, 1985, 2008) more commonly associated with leader development (TRADOC, 2009). In this sense, OBTE essentially is a leadership-based approach to training, education, and self-development. OBTE is based on the recognition that Soldiers, who are both confident and competent, are developed when leaders allow subordinates reasonable autonomy to exercise individual and small-unit initiative. This requires a climate of accountability that is fostered by mutual trust among superiors and subordinates (Riccio et al., 2004). It also requires that leaders at every level recognize their influence as role models (Bandura, 1977, 1997; Lave & Wenger, 1991) by thinking and acting flexibly based on constant awareness and adjustment to deviations from optimal or expected conditions in any situation. An instructional strategy associated with this principle is to create opportunities for Soldiers to assume responsibility for their actions, and hence, to develop a sense of accountability (Darwin, 2008a, b; Cornell-d’Echert, 2009a, b). The priority is to encourage mutual awareness among peers. Awareness of the consequences of one’s behavior with respect to one’s peers also leads to



a heightened awareness of the behavior of those peers (Bandura, 1977; Lewin, 1951). This can lead to ad hoc peer coaching that can be a valuable instructional multiplier. For example, consider what happens when a Soldier is taught the importance of pointing his weapon in a safe direction at all times. As Soldiers move around during training, they become aware of where their weapon is pointing, but also where other Soldiers’ weapons are pointing, so that they do not intentionally walk into a potential line of fire, and call out other Soldiers for a lack of such awareness. Contrast this with Soldiers who are told to “always point their weapon up and down range when on the firing line.” They follow those instructions to the letter, and ignore the purpose of pointing the weapon up and down range, for safety, and may perseverate on this rule even in situations when it results in their weapon pointing at someone who may be standing on an elevated terrain. This example illustrates that reciprocal influence and accountability need not be a complex undertaking. The foundations for collective agility and success can be developed early in a Soldier’s training and education. 3.4.2 Robust and Adaptable Plan There are eight measures in this category (Appendix A). A representative measure is “Do the instructors adapt outcome expectations to account for Soldiers who learn slower/faster than anticipated?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) cancels training or maintains focus on specific outcomes for original task; (b) chooses different outcomes but does not achieve effective training; or (c) chooses outcomes to achieve training effectiveness, leverages condition as another training opportunity. Outcomes and training events should focus on the mission relevance of knowledge, skills, and attitudes rather than strict adherence to prescribed (e.g., expository) details of a certain task. In OBTE, instructors plan to emphasize effectiveness over efficiency; that is, they focus on progress with respect to development of the Soldier as a student and development of the student as a Soldier, not merely task completion or throughput. In OBTE, an instructor is prepared with a framework—a mindset, intent, and nested objectives—that helps prioritize an inexhaustible array of prior considerations and opportunities that may arise during a learning event (Cornell-d’Echert, 2009b; Haskins, 2009a, b; see Appendix C). To ensure that training is effective, instructors should plan to frequently assess how the training is progressing. During this assessment, instructors can determine if they are achieving their intent for the day and making changes as necessary. The selection of a manageable number of formative measures for a particular learning event can facilitate assessments that can be conducted during or after a learning event (see Appendix A). They provide a theme for a learning event that is robust to uncontrolled variation in conditions. Planning flexibility is important in dealing with variables that may be out of the instructor’s control. A mature instructor should have a contingency plan or expectation to facilitate in-stride adjustments, for example, to inclement weather or limited resources. In every situation, the instructor should anticipate leveraging unforeseen conditions as a training opportunity rather than a constraint. For example, a rainy day can be leveraged to build Soldier confidence in the capabilities of their weapon to operate in inclement weather. 3.4.3 Instructors as Role Models There are six measures in this category (Appendix A). A representative measure is “Do the instructors demonstrate openness in changing their training progression?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) training will not deviate from a set schedule; (b) instructors determine when it is time to move on to a new task; or (c) Soldiers have input into the progression of training.

71 Riccio & Darwin


Instructors can monitor whether students are getting the training they need and decide when students should advance to the next event or revisit a learning objective within the parameters of established time resources. Instructors, as mentors, exhibit behavior associated with the same attributes that they are trying to grow in their students throughout training (e.g., confidence, initiative, accountability). In OBTE, this means that instructors demonstrate that they are open to deviations from the plan and that they are open to insights and guidance from others such as their peers in the cadre. A central premise of OBTE is that all instructors are influential as role models, all the time, whether or not they intend to be role models and irrespective of the extent to which they are prepared to act as role models (Bandura, 1977, 1997; Lave & Wenger, 1991). Instructors may have admired status, or otherwise may have high credibility, whether or not they deserve it. The vastly important implication of this assumption is that instructors may actually do more harm than good if they are not educated about the potential sources and manifestations of such influence. The measures for this category are a simple mechanism to help instructors achieve awareness of their influence and a framework for instructor education. 3.4.4 Collaborative Identification of Outcomes and Measures There currently are no measures in this category. The need for this category is emphasized by the importance of collaborative discussion among instructional cadre and their chain of command to achieve a balance of near-term learning objectives and the long-term developmental outcomes emphasized by OBTE. Such collaborative decision-making is verifiable and best practices can be identified (Appendix C; Haskins, 2009a, b; Perry & McEnery, 2009). Development of formative measures for this activity of an instructional organization is one way to focus attention on a change initiative and to track its progress within the organization (cf., Chapter 14). Measures of this kind have not yet been developed for OBTE because addressing organizational change and associated frictions was not part of the original intent of the investigation. Nevertheless, organizational change has emerged as a critical element of OBTE. Interestingly, the process used to develop measures for OBTE (e.g., Chapters 2, 7, and 12) is a model for how outcomes and measures can be developed collaboratively by an instructional cadre and their chain of command in a way that is scientifically rigorous and traceable (see section 3.2.7). In principle, one also can observe and measure implementation of this model as a critical element of organizational behavior and come to a better understanding of the factors affecting its implementation (cf., Bandura, 1995; Edmonson, 2003; Vaughan, 1997; Rasmussen, 1997). Formative measures should facilitate post hoc internal or external review of the collaborative conduct and performance of instructional cadre and their chain of command in identifying: (a) the psychological and physiological effects and the physical requirements of combat; (b) outcomes associated with adaptability amid the trans-extremis conditions of Full Spectrum operations; (c) outcomes associated with inculcation of Warrior Ethos and Army Values; (d) observable instructor behavior associated with being a leader, facilitator, advisor, mentor, and role model; and (e) instructional strategies that introduce and control passion and risk.

3.5 Emerging Best Practices in OBTE for a Knowledge-Centered Environment Knowledge-centered environments foster learning in ways that lead to an integrated understanding of a discipline and to subsequent transfer (Bransford et al., 2000). There is an emphasis on a network of connections within learning events, between learning events, and with both prior and subsequent learning. A student becomes oriented within a curriculum-level landscape instead of being lost in a disconnected complex of knowledge and skills acquired over an extended period of time. We believe that the characteristics of a knowledge-centered



environment relate most directly to integrated understanding of basic Soldier skills in Full Spectrum Operations (see Table 2). 3.5.1 Integrated Understanding of Basic Soldier Skills in Full Spectrum Operations OBTE emphasizes that instruction is most effective when it reflects and responds to the most urgent requirements of Full Spectrum Operations. These requirements are considered in terms of basic Soldier skills such as move, shoot, and communicate but at a layer or two deeper than what those words commonly connote (Flanagan, 2005). These operational requirements can be addressed at a level that relates to individual adaptability and collective agility and that addresses the why, what, when, where, and how of actions that can have irrevocable consequences in operational settings and missions (Darwin, 2008a, b). Thus, they can be addressed at a level that benefits all Soldiers and their units. Many of these requirements relate to stress, psychological and physiological effects of stress, performance given these effects, and strategies that are resistant to stress (Lerner & Keltner, 2001; Lerner & Tiedens, 2006; Lukey & Tepe, 2008). Experience with these relationships and effects across disparate learning events helps a Soldier overcome stress by avoiding distraction, maintaining an outward orientation, and always exercising deliberate thought (Darwin, 2008a,b; cf., Kolditz, 2007; Lukey & Tepe, 2008). It helps students make sense of the relationships between what is taught and why, when, and how it is taught. It provides them with a path toward ever deepening understanding of their capabilities in context (Hmelo-Silver, 2004; Magolda, 1999; Riccio et al., 2004). An instructional strategy within the framework of OBTE is for instructors to communicate with Soldiers and guide them to discover the purpose and intent of the training. Purpose generally illuminates a broader context within which tasks are interrelated and the role of an individual is related to the mission of the unit (J. Gibson, 1977; Lewin, 1951). A focus on purpose reveals the relevance of the instructional task to an operational mission. More deeply, it reveals the aspects of the surroundings about which a Soldier should be aware in a collective task, whether in training or in an operational mission. This lays the foundation for education of the attention, habitual orientation to the surroundings in any and all situations, and an integrated understanding of a situation (Endsley, 1995; E. Gibson, 1988; J. Gibson, 1977). 3.5.2 Task Relevance of Planned Instructional Events There are five measures in this category (Appendix A). A representative measure is “Do the instructors build time for this discussion [about tactical relevance of the task] into the schedule?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) does not build in time for discussion into the schedule; (b) builds in time for short discussions after the day is complete; or (c) builds in time for short discussions between tasks. It cannot be assumed that the students will understand why a training exercise is relevant to their mission. To ensure a knowledge-centered environment, it is important to design exercises that aid the student in making these connections. For example, understanding the design of a learning event and the reasons behind it will contextualize students’ learning, and presumably lead to better skill retention and transfer to the operational environment. A learning environment that is both community-centered and knowledge-centered should reflect and develop values of the Army culture (see Chapters 1-5 and the Epilogue; Riccio et al., 2004). The training environment does not have to look like the operational environment to develop values, basic Soldier skills, and

73 Riccio & Darwin


resilience to stress. Operational fidelity of a training event can be considered, instead, in the context of the conditions that foster development of intangible attributes that are critically important in an operational context or mission. In OBTE, instructors focus on setting up conditions that foster fidelity of interpersonal interactions and task linkage that, in turn, provide students with an appreciation of the context in which knowledge, skills, and attitudes make a difference (cf., Neisser, 1976; Shaw & Bransford, 1977; Winograd, Fivush, & Hirst, 1999). In this approach, variation in conditions is an enabler for the development of competencies that can transfer to ambiguous or changing environments. The most robust competencies are those defined in terms of the relations or interactions between the individual and the environment (Bandura, 1977; E. Gibson, 1988; J. Gibson, 1977; Lave & Wenger, 1991; Lewin, 1951). 3.5.3 Reveal Operational Relevance of Training There are seven measures in this category (Appendix A). A representative measure is “Do the instructors group tasks into collective behavior?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) does not group tasks into collective behavior, results in incorrect performance of linked tasks; (b) does not group tasks into collective behavior (no observable negative consequences); or (c) groups task in a way that simulates the combat application and reinforces correct performance of linked tasks. OBTE seeks to lay the foundation for training, education, and self-development (Darwin, 2008a, b). Soldiers, for example, should come to understand the reasons behind a training event, especially if its operational relevance is not readily apparent. The purpose is not for the instructor to seek the approval or buy-in of students. Instead, it is to help students understand that the operational relevance of learning events is not necessarily manifested in tangible ways or through superficial similarities to an operational scenario. In addition to the interpersonal interactions and task linkages mentioned above in the context of planning for operationally relevant instruction, students should gain experience in collaborative problem solving and the collective agility it enables. They should gain experience dealing with the unknown and the learning in real time that it necessitates. They should be given a rich variety of experiences that gives them practice in making connections among disparate events and in extrapolating to new situations (Tobias & Duffy, 2009; Van Merrienboer & Kirschner, 2007). Finally, they should understand that, in these ways, training is like the operational environment. 3.5.4 Incorporate Stress into Instructional Events There are six measures in this category (Appendix A). A representative measure is “Do the instructors encourage deliberate thought in stressful situations?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) rushes or encourages Soldier to complete the task without deliberate thought, focuses on speed; (b) encourages Soldier to think but fails to establish effective training conditions to reinforce thinking; or (c) focuses Soldier on thinking through situation regardless of the stress level, focuses on accountability. As indicated above, stress is not inconsistent with a positive learning environment if the source and manifestations of stress are representative of demanding operational environments such as combat (see, e.g., Lukey & Tepe, 2008). Complexity introduced through scenarios is one way to create stress that is relevant and that potentially allows Soldiers to develop strategies that reduce stress or mitigate the effects of stress. Training scenarios need not look exactly like operational



scenarios to be useful in this way. In general, scenarios can reveal complexity in linkage among tasks, for example, and the related implications for interdependence of individuals in a task-organized unit. Systematic manipulation of such complexity has instructional utility because it is relevant to the understanding and capabilities for action that Soldiers acquire during training. Experience and development of self-efficacy in such situations helps Soldiers build habits of outward orientation amid a growing repertoire of personal capabilities. OBTE assumes that an understanding of what one can do, one’s self-efficacy, in various situations can help one avoid distractions from the physiological or psychological effects of stress (cf., Kolditz, 2007; Lerner & Keltner, 2001; Lerner & Tiedens, 2006; Lukey & Tepe, 2008). 3.5.5 Identify General Lessons Learned and Extrapolate to New Situations There currently are two measures in this evolving category (Appendix A). A representative measure is “Does the instructor ensure Soldiers can articulate how to apply concepts to new situations?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) discussion focuses on the specific task only, does not discuss combat or mission application; (b) facilitates a discussion of the specific tasks and how they apply to missions; or (c) facilitates a discussion of the intangible attributes underling the task and how it applies to ambiguous combat/mission situations. This category of measures, generally utilized after a learning event, addresses the role of inter-subjectivity and collaboration in learning and development. It is relevant to acquisition of knowledge and skill but at multiple levels. Thus it is involves refection on more than the knowledge or skill per se. Some of the measures also should reveal the development of metacognitive understanding about how the knowledge or skill is utilized in particular situations and how this is influenced by the situation. An important consideration in further development of these measures should be to help Soldiers reflect on the role of emotion in thinking and performance (Lerner & Keltner, 2001; Lerner & Tiedens, 2006; Lukey & Tepe, 2008). The assumption is that such reflection will develop metacognitive awareness can help Soldiers (a) exercise deliberate thought under stress; (b) be better prepared for transitions into, out of, and between stressful situations characteristic of Full Spectrum Operations; and (c) learn lessons from their experience in particular stressful situations and trans-extremis conditions that can be applied to different situations.

3.6 Emerging Best Practices in OBTE for an Assessment-Centered Environment Assessment-centered environments emphasize formative feedback that promotes understanding (Bransford et al., 2000). Opportunities for feedback should be ubiquitous and should be driven by noteworthy events in individual or group activities. Feedback should help a student make connections between current learning events and other parts of the curriculum and the student’s life. It should help students appreciate and become more skillful at self-assessment and peer assessment (Bransford et al., 2000). Consistent with this characteristic of a good learning environment, OBTE extensively utilizes after-action reviews and ad hoc instructor-student interactions to draw out of the Soldiers a critique of their own performance throughout the learning cycle. We believe that the characteristics of an assessment-centered environment relate most directly to collaborative reflection and problem solving (see Table 2).

75 Riccio & Darwin


3.6.1 Collaborative Reflection and Problem Solving OBTE seeks to establish a mindset of collaborative reflection and problem solving in which there is continuous vigilance for noteworthy events and lessons learned worth discussing among instructors, students, and peers. Development of mastery in assessing oneself and one’s unit is a key outcome for OBTE. It is important to understand that collaborative reflection is a ubiquitous opportunity that exists in any program of instruction. It can be done on an as-needed basis when the lessons learned justify the investment of time and resources. It can exploit naturally occurring events and utilize whatever method is appropriate at the time, such as “after action reviews,” “hot washes,” and informal conversation. This does not mean that constant discussion is a goal of OBTE or that discussion itself is a reflection that OBTE is occurring. It is as important to decide not to engage in collaborative reflection if there has not been a noteworthy event or if there is a more valuable use of time for learning. This mindset, thus, also continually reinforces the importance of prioritization and tradeoffs required in most military scenarios (Riccio et al., 2004). If a decision is made to engage in collaborative reflection (e.g., to execute a planned after-action review), the discussion should be conducted in an open and supportive manner regarding lessons learned. The purpose is not to summarize merely what happened in the learning event but to facilitate a discussion that allows students to evaluate their own performance in terms of the consequences of their actions for the rest of the team and with respect to other linked tasks (Neisser, 1996; Neisser & Jopling, 1997). Collective self-examination leads to an understanding in which the whole is greater than the sum of the parts (cf., Barnes, 1959; Cotkin, 2003; Solomon, 2004). A core strategy of OBTE is to guide Soldiers to the discovery of solutions, rather than merely telling them what to do (e.g., telling the solution prior to introduction of the problem) (Darwin, 2008a, b; Savory & Duffy, 1995; Vandergriff, 2006, 2007; see also Tobias & Duffy, 2009). This prepares Soldiers to take initiative in addressing unique problems that may be faced in a mission environment. OBTE recognizes that training design cannot predict all of the problems that Soldiers will face. Therefore, rather than try to condition a repertoire of scripted responses, OBTE attempts to develop skills that can support adaptive behavior even in unforeseen or novel environments. One of these skill sets is the ability to make decisions about collaborative reflection and to engage effectively in it. This problem solving approach helps to develop a sense of initiative that is grounded in awareness of contextual relationships and vigilance about the attendant consequences of one’s own behavior (J. Gibson, 1977; Shaw & Kinsella-Shaw, 2007). Guidance provided by instructors includes stimulating and facilitating awareness of this dynamic context rather than simply telling the Soldier about the best course of action given these considerations. 3.6.2 Communication There currently are six measures in this evolving category (Appendix A). A representative measure is “Does the instructor encourage inter-trainee communication and discussion?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) Soldiers do not have the opportunity to discuss; (b) training design provides opportunities for unguided inter-trainee communication; or (c) instructor facilitates discussions and encourages follow-on inter-trainee discussion, training design provides opportunities. This category includes measures that initially were developed outside of the COMPASS process using an approach employed in previous research on adaptive communications analysis (e.g., Entin, Diedrich, & Rubineau, 2003). The purpose of these measures was to conduct a quantitative



assessment of communication between instructors, students, and peer coaches that is emphasized in OBTE (see Chapter 2). Some of the measures (e.g., guided learning or self-discovery and constructive or diagnostic communication) also can be useful in formative assessment; thus, we expect these measures to be a priority in continuing development of the comprehensive set of measures for OBTE. 3.6.3 Nature and Extent of Guidance There are seven measures in this category (Appendix A). A representative measure of how an instructor might gauge the need for guidance to a Soldier is “Does the instructor recognize when a Soldier is too withdrawn/distracted to effectively participate in training?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) does not recognize a deeper individual problem when Soldier is struggling with a task; (b) sees that there is a deeper problem, but doesn’t do anything (doesn’t realize he can do something); or (c) recognizes there is a deeper problem, takes appropriate action to help Soldier get back in the game. OBTE occurs when instructors make principled decisions about when and how to adapt the instructional environment to achieve a positive outcome. Such adaptation includes utilization of whatever instructional methods are appropriate for the problem at hand, whether that is direct instruction, problem-centered instruction, or some other method (see e.g., Hmelo-Silver et al., 2007; Tobias & Duffy, 2009; Van Merrienboer & Kirschner, 2007). The goal is to provide learners with the right amount of guidance at the right time to achieve both the learning objective for the event and longer-term developmental outcomes such as the growth of confidence, initiative and accountability. 3.6.4 Establish a Pervasive Mindset of Collaborative Reflection There currently are two measures in this evolving category (Appendix A). A representative measure is “Does instructor conduct AAR or hot-washes as needed?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) does not conduct AAR or hot-wash, or conducts regardless of need; (b) Tells or lectures Soldiers the lessons learned in a timely manner; or (c) facilitates discussion on lessons learned amongst the Soldiers in a timely manner. This category of measures, generally utilized after a learning event, addresses the practices of inter-subjectivity and collaboration in learning and development. Further development of these measures will parallel the development of measures outlined in section 3.6.2 that generally are utilized during a learning event. The difference is that these measures should foster collaborative metacognitive reflection. Engaging others in reflection, after joint participation in an event, on the characteristics and effectiveness on communication and collaboration that had occurred during the event can help metacognitive reflection from becoming an infinite regress into the abstract. It keeps it inter-subjectively valid and grounded in a shared reality. The prior intent and interpretations of other participants can be made explicit, thus, the interpersonal causes for miscommunication and misinterpretation are more readily revealed. It is important to know whether problems were due to differences in prior experience, prior knowledge, and associated assumptions or whether they were due to personality characteristics or even momentary conditions (e.g., stressors). Differentiating these potential causes, and identifying the critical ones, can improve collaboration and performance of the group but it also potentially can improve an individual’s understanding and awareness of interpersonal factors in different groups.

77 Riccio & Darwin


3.7 Emerging Best Practices in OBTE for a Learner-Centered Environment To be learner-centered, an educational system must address what learners know, what they do not know, how they learn, and what they are motivated to learn (Bransford et al., 2000). In OBTE, the strategy is to provide learners with right amount of guidance at the right time to achieve both learning and development. As a mentor and facilitator, the instructor assists the Soldier to understand the situation, the desired result, and the obstacles to the desired result. The instructor creates an environment that encourages initiative, a broad appreciation of accountability, and the development of confidence that is grounded in demonstrable competence. We believe that the characteristics of a learner-centered environment relate most directly to Soldier motivation and development of intangibles (see Table 2). 3.7.1 Soldier Motivation and Development of Intangibles OBTE strives to provide learners with guidance that helps them achieve both the learning objective for the event and longer-term developmental outcomes such as the growth of confidence, initiative and accountability (Deci & Ryan, 1985; Guay, Ratelle, & Chanal, 2008; Peterson & Seligman, 2004). The intent is to leverage internal motivation and to internalize sources of motivation by balancing initiative with a sense of accountability based on discipline and awareness of one’s relationship to others, and by building confidence based on enhanced self-efficacy and commitment to the pursuit of mastery (Bandura, 1997; Deci & Ryan, 2008; Ryan & Deci, 2008). OBTE occurs when instructors make principled decisions about when and how to adapt the instructional environment to achieve a positive outcome for particular students in particular situations, that is, when they exemplify leadership. Such adaptation includes utilization of whatever instructional methods are appropriate for the problem at hand (Tobias & Duffy, 2009). The ways in which OBTE can benefit from methods motivated by direct instruction (e.g., Kirschner, Sweller, & Clark, 2006), constructivist instruction (e.g., Hmelo-Silver, Duncan, & Chinn, 2007), and experiential instruction (e.g., Hickman & Alexander, 1998) remain to be seen but the exigencies of OBTE can inform decisions about the nature and extent of guidance provided to students or instructors in any of these methods. For example, even in situations in which students do a "worked example" or solve a well-defined problem given by the instructor, it is beneficial for the student to understand that the assumptions of the given problem are valid and why the assumptions are necessary, even if they are not ready to be thrown into a situation in which assumptions are violated. This allows them to become ready, to become prepared to deal with such a situation. The foundation for collective agility is thus reinforced in all training and educational situations, even ones that ostensibly do not require individual adaptability. A key strategy for the continuing development of intangibles is an emphasis on mastery (Darwin, 2008a, b). From OBTE perspective, pursuit of mastery is essential for confidence that is grounded in the development of competence. In addition, mastery enables Soldiers to see beyond the instructional situation and beyond the associated standards as end states. Competence as a pursuit should motivate an interest in further learning and in obtaining enhanced depth or breadth of competence. An instructional consideration in this strategy is setting up conditions in which Soldiers are allowed to experience improvement amid awareness that they can do better (Cornell-d’Echert, 2009a, b; Bandura, 1997; Cole, John-Steiner, Scribner, & Souberman, 1978). For each of these strategies, the cornerstone of OBTE is for the instructor to set the conditions that create opportunities for learning (e.g., Dewey, 1915/2008; Bransford et al., 2000; Tobias & Duffy, 2009).



3.7.2 Plan for Development of the Individual There are nine measures in this category (Appendix A). A representative measure is “Is the training designed to develop intangible attributes (Judgment, Adaptability, Accountability, Problem Solving, Confidence, Initiative, Awareness, Thinking Skills)?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) focus of training is solely to accomplish the task; (b) instructor discusses intangibles directly related to task being trained, does not focus plan on development; or (c) instructor discusses intangibles directly related to the task being trained, plan focuses on development. Instructors should prepare training that develops Soldier understanding of a concept, and supports adaptive thinking and problem solving that is robust to changes in conditions. Instructors should not be the proximate source of stress through techniques such as intimidation and belittlement that have a counterproductive effect on Soldier confidence, initiative, and accountability. This does not imply that instructors should coddle Soldiers. There are times for a strident authoritarian style but it should be used judiciously. To utilize stress in training, instructors should prepare challenges that are more likely to carry over to realities of the operating environment. This frees the instructor to be a source of guidance and insight about how to manage stress (Darwin, 2008a, b; Cornell-d’Echert, 2009a, b). Students learn best in a relatively comfortable, low stress environment, but this does not ensure that they will learn to deal with stress. To address this tradeoff, in OBTE, instructors plan for systematic increases in stress across learning events (Darwin, 2008a, b; cf., Hmelo-Silver et al., 2007; Van Merrienboer & Kirschner, 2007). Stress does not necessarily have negative connotations in this context. One source of increasing stress or challenge is to provide students with opportunities to take ownership of their own learning. This is a powerful motivator and source of initiative (Deci & Ryan, 2008; Ryan & Deci, 2008) but it also requires increased discipline, awareness, and accountability that bounds one’s own initiative and ensures that it is suitably robust to the inevitable mistakes that are critical in learning to learn. Time should be built into the schedule for students to engage in self-discovery and collaborative reflection. Though it may be quicker to simply tell a student what the answer is, it is operationally more relevant if they are provided the tools to arrive at the solution themselves (cf., Hmelo-Silver, 2004; Tobias & Duffy, 2009). This helps students acquire the skills and internal sources of motivation that enable self-development in and outside of formal learning environments (Darwin, 2008a, b; d’Echert, 2009a, b; Guay, Ratelle, & Chanal, 2008). 3.7.3 Get Students to Take Ownership There are eight measures in this category (Appendix A). A representative measure is “Does the instructor foster self-development amongst the Soldiers?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) tells them how they did without giving Soldiers a chance to reflect; (b) asks the Soldiers how they think they did, tells them what they did wrong; or (c) asks the Soldiers how they think they did, facilitates self-discovery of mistakes through targeted questioning. A balance of power and leadership is critical in OBTE. Achieving this balance is complicated for many reasons, not the least of which is that students may be unfamiliar with this instructional style. The hurdle can be overcome more easily if the instructor acts as an advisor and mentor with a clear interest in the student’s development. In OBTE, instructors allow passion and measured risk (Darwin, 2008a, b). They encourage the development of individual initiative that is

79 Riccio & Darwin


constrained by the collective accountability implicit in interdependence and by confidence grounded in demonstrable competence. Thus, they foster self-determination in Soldiers that is consistent with military order. 3.7.4 Collaborative Reflection as a Means to Develop Self Efficacy There currently are five measures in this evolving category (Appendix A). A representative measure is “Does the instructor ask Soldiers to assess their own performance?” This measure includes the following anchors for novice, intermediate, and expert behavior respectively: (a) does not ask Soldiers to assess their own performance or is overly negative in assessment; (b) asks Soldiers to assess their own performance but does not recognize accurate self-assessments; or (c) asks Soldiers to assess their own performance but monitors to ensure they are realistic. This category of measures, generally utilized after a learning event, addresses the effects of inter-subjectivity and collaboration in learning and development. The important concept of self-efficacy, one’s perception of one’s own capabilities, is most noteworthy when there is an apparent discrepancy (Bandura, 1977, 1997). Inter-subjective influences can either increase or decrease discrepancies. Collaborative reflection is useful in this regard to the extent that others are willing and able to help inform one about one’s own capabilities, that is, to foster the veridical perception that is critical in engagements with the real world. Such guidance from more astute observers, from less biased observers, or simply from a different perspective, is even more critical amid the uncertainty of inferences about one’s zone of proximal development (cf., Cole et al., 1978). The acumen or perspective of others can help one identify paths in the pursuit of mastery that may not be obvious or that may be obscured by habits, assumptions, or unknowns about strategies for cognition and action. Collaborative reflection in groups fosters intersubjective convergence on reality (cf., Chapters 4 and 5). Collaborative reflection and inter-subjectivity also are central in the development of values implicit in the balance of choice and responsibility (cf., Aristotle, trans. 1925; Augustine, trans. 1991; Hoffman et al., 2009; Maslow, 1964; Neisser & Jopling, 1997; Solomon, 2004). Others can help reveal constraints of shared values on one’s cognition and action, they can help reveal non-obvious ways in which one’s cognition and action is situated in a shared context, and the impact of the mere existence of others in a shared context on one’s cognition and action. It both directs and educates the attention. In the context of the group, which is the fundamental unit of analysis in the Army, inter-subjectivity is a source of coupling among individuals (i.e., their respective cognition and action) that renders the notion of individual adaptability almost irrelevant. Individual adaptability is of consequence and is admissible only in the context of collective agility, in a context that bounds unplanned activity and that gives it purpose. The principles and practices of OBTE, summarized in this chapter, provide basic facts from which to build an appropriate scientific foundation for OBTE. Such scientific foundations are explored in the Chapters 4 and 5.

3.8 References

Allik, J., & McCrae, R. R. (2004). Towards a geography of personality traits: Patterns of profiles

across 36 cultures. Journal of Cross-Cultural Psychology, 35, 13-28. Augustine (1953). The teacher. In: J. Burleigh (Trans. & Ed). Augustine: earlier writings.

Philadelphia, PA: The Westminster Press. (Original work written ca. 389) Augustine (1991). Confessions. H. Chadwick (Trans). New York: Oxford University Press.

(Original work written ca. 400)



Aristotle (1977). Nicomachean ethics. In: R.M. Hutchins et al. (Eds.). Aristotle II. Chicago, IL: Encyclopedia Britannica. (Reprinted from The Works of Aristotle, by W.D. Ross Ed. & Trans., 1925, Oxford, UK: Oxford University.) (Original work written ca. 350 BCE)

Aspinwall, L. & Staudinger, U. (Eds.) (2003). A psychology of human strengths: Fundamental questions and future directions for a positive psychology. Washington, DC: American Psychological Association.

Bandura, A. (1977). Social learning theory. New York: General Learning Press. Bandura, A. (Ed.) (1995). Self efficacy in changing societies. Cambridge, UK: Cambridge

University Press. Bandura, A (1997). Self-efficacy: The exercise of control. New York, NY: Freeman and

Company. Barnes, H. (1959). Humanistic existentialism: The literature of possibility. Lincoln, NE:

University of Nebraska. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind,

experience, & school. Washington, DC: National Academy Press. Brunyé, T., Riccio, G., Sidman, J., Darowski, A., & Diedrich, F. (2006). Enhancing warrior ethos

in initial entry training. Proceedings of the 50th Annual Meeting of the Human Factors and Ergonomics Society, San Francisco, CA.

Burke, W. (1985). Applied differential geometry. Cambridge, UK: Cambridge University. Camber, C., Loewenstein, G., & Prelec, D. (2005). Neuroeconomics: How neuroscience can

inform economics. Journal of Economic Literature, 43(1), 9-64. Camic, P.M., Rhodes, J.E., & Yardley (Eds.) (2003). Qualitative research in psychology:

Expanding perspectives in methodology and design. Washington DC: APA. Carlson, S. (2005). The net generation goes to college. The Chronicle of Higher Education,

Volume 52 (7), A34. Clausewitz, C. von (1976). On war. In: M. Howard. & P. Paret (Trans. and Eds.). On war.

Princeton: NJ: Princeton University. (Original work written 1832) CMMI Product Team (2009). CMMI for services, version 1.2. (CMU/SEI-TR-2009-001; ESC-

TR-2009-001). Pittsburgh, PA: Carnegie Mellon University. Cole, M., John-Steiner, V., Scribner, S., & Souberman, E. (1978). Mind in society: The

development f higher psychological processes (L.S. Vygotsky). Cambridge, MA: Harvard University Press.

Cornell-d’Echert, B. (2009a). An introduction to outcomes-based training and education. Fort Meade, MD: Asymmetric Warfare Group.

Cornell-d’Echert, B. (2009b). Outcomes-based training and education: Implementation guide. Fort Meade, MD: Asymmetric Warfare Group.

Cotkin, G. (2003). American existentialism. Baltimore, MD: Johns Hopkins University. Cresswell, J. (1998). Qualitative inquiry and research design: Choosing among five traditions.

Thousand Oaks, CA: Sage. Darwin, M. (2008a). Asymmetric Warfare Group combat applications training course (CATC)

senior leader discussion (Briefing, January 23, 2008). Fort Meade, MD: Asymmetric Warfare Group.

Darwin, M. (2008b). Outcomes-based training and education: fostering adaptability in full spectrum operations (Briefing, December 2008). Fort Meade, MD: Asymmetric Warfare Group.

Deci, E. & Ryan, R. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum.

Deci. E. & Ryan, R. (2008). Self determination theory: a macrotheory of human motivation, development, and health. Canadian Psychology, 49, 182-185.

81 Riccio & Darwin


Denzin, N.K, & Lincoln Y.S. (Eds.) (2003). Strategies for qualitative inquiry. Thousand Oaks, CA: Sage.

De Raad, B., & Perugini, M. (2002). Big five assessment. Gottingen, NL: Hogrefe & Huber. Dewey, J. (2008). The school and society, the child and the curriculum. Chicago, IL: Centennial

Publications of The University of Chicago Press. (Original work published 1915) Ellingson, L. (2009). Engaging crystallization in qualitative research: An introduction. Los

Angeles, CA: Sage. Endsley, M.R. (1995). Toward a theory of situation awareness in dynamic systems. Human

Factors, 37, 32-64. Entin, E. E., Diedrich, F. J. & Rubineau, B. (2003). Adaptive communication patterns in different

organizational structures. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Denver, CO.

Feldbaum, A. (1965). Optimal control systems. New York: Academic Press. Filatov, N.M. & Unbehauen, H. (2004). Adaptive dual control: Theory and applications. New

York: Springer. Flanagan, S. (2005). Combat skills. Unpublished manuscript (Nov. 16, 2005). Freeman, J., Jason, J., Aten, T., Diedrich, F., Cooke, N., Winner, J., Rowe. L., & Riccio, G.

(2008). Shared Interpretation of Commander's Intent (SICI). Final Report to the Army Research Institute for the Behavior and Social Sciences, contract number W74V8H-06-C-0004.

Gibson, E. (1988). Exploratory behavior in the development of perceiving, acting, and the acquiring of knowledge. Annual Review of Psychology, 39, Pages 1-42.

Gibson, E. (1991). An odyssey in learning and perception. Cambridge, MA: MIT Press. Gibson, J. (1977). The theory of affordances. In R. Shaw & J. Bransford (eds.), Perceiving, acting

and knowing: toward an ecological psychology. Hillsdale, NJ: Erlbaum. Gibson, J. & Gibson, E. (1955). Perceptual learning: Differentiation or enrichment. Psychological

Review, 62, 32-51. Glaser, B. (2002). Efficiency versus sustainability in dynamic decision making: Advances in inter-

temporal compromising. New York: Springer. Glass, L. & Mackey, M. (1988). From clocks to chaos: The rhythms of life. Princeton, NJ:

Princeton University. Goldberg, S. (1986). Introduction to difference equations: With illustrative examples from

economics, psychology, and sociology. Guay, F., Ratelle, C. F., & Chanal, J. (2008). Optimal learning in optimal contexts: The role of

self-determination in education. Canadian Psychology, 49, 233–240. Haskins, C. (2009a, March). Development of outcomes based training. Presentation at the US

Army Asymmetric Warfare Group workshop on Outcomes-based Training and Education, Applied Physics Laboratory, Johns Hopkins University, Laurel MD.

Haskins, C. (2009b). Outcomes-based training and education. White Paper solicited by TRADOC Headquarters, August, 2009.

Headquarters Department of the Army (2003). Mission command: Command and control of Army forces. FM 6-0. Washington, DC: Department of the Army.

Headquarters Department of the Army (2005). The Army. FM 1. Washington, DC: Department of the Army.

Heine, S. J. & Norenzayan, A. (2006). Toward a psychological science for a cultural species. Perspectives on Psychological Science, 1(3), 251-269.

Heisenberg, W. (1958). Physics and philosophy: The revolution in modern science. New York: Harper.

Hickman, L.A. & Alexander, T.M. (1998). The essential Dewey, Volume 1: pragmatism, education, and democracy. Bloomington, IN: University of Indiana Press.



Hmelo-Silver, C. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, Vol. 16(3), 235-266.

Hmelo-Silver, C. Duncan, R.G., & Chinn, C. (2007). Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107.

Hoffman, L., Yang, M., Kaklauskas, F., Chan. (2009). Existential psychology East-West. University of. Rockies Press. Colorado Spring.

Hofstede, G. J., Pedersen, P., & Hofstede, G. (2002). Exploring culture: exercises, stories, and synthetic cultures. Boston, MA: Intercultural Press.

Howe, N., Strauss, W., & Matson. R.J. (2000). Millennials rising: the next great generation. New York, NY: Vintage.

Idel, M., & McGinn, B. (Eds.) (1999). Mystical union in Judaism, Christianity, and Islam: An ecumenical dialogue. New York: Continuum.

James, W. (1890). Principles of psychology. New York, NY: Henry Holt and Company. James, W. (1907). Talks to teachers on psychology: and to students on some of life's ideals. New

York, NY: Henry Holt and Company. (Original work published 1899) James, W. (1982). The varieties of religious experience: A study in human nature. New York:

Penguin. (Original work published 1902) Jarvis, P. (2004). Adult education and lifelong learning: Theory and practice. London, UK:

Routledge-Falmer. Keatinge, M. (1896). The great didactic of John Amos Comenius. Whitefish, MT: Kessinger

Publishing. Keyes, C. & Haidt, J. (2003). Flourishing: Positive psychology and the life well-lived.

Washington, DC: American Psychological Association. Kierkegaard, Soren (1843) under the pseudonym Silentio, Johannes. Fear and Trembling, edited

by Alastair Hannay. (1985): Penguin Classics Kirkpatrick, D., (1994). Evaluating training programs. San Francisco, CA: Berrett-Koehler

Publishers, Inc. Kirkpatrick, J., & Kirkpatrick, W. (2009). The Kirkpatrick four levels: A fresh look after 50 years,

1959 – 2009. Retrieved May, 2009, from http://kirkpatrickpartners.com. Kirschner, Sweller, and Clark (2006). Why minimal guidance during instruction does not work:

An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry based teaching. Educational Psychologist, 41, 75-86.

Knowles, M., Holton, E., Swanson, R. (2005). The adult learner. Burlington, MA: Elsevier. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development.

New Jersey: Prentice-Hall. Kolditz, T. (2007). In extremis leadership: Leading as if your life depended on it. New York:

Wiley. Lanczos, C. (1970). The variational principles of mechanics. New York: Dover. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.

Cambridge, UK: Cambridge University Press. Lerner, J., & Keltner, D., (2001). Fear, anger, and risk. Journal of Personality and Social

Psychology, 81(1), 146-159. Lerner, J., & Tiedens, L. (2006). Portrait of the angry decision maker: How appraisal tendencies

shape anger’s influence on cognition. Journal of Behavioral Decision Making (Special Issue on Emotion and Decision Making), 19, 115-137.

Lévi-Valensi, J. (Ed.) (2006). Camus at combat: Writing 1944-1947. Princeton, NJ: Princeton University. (Original work written 1944-1947).

Lewin, K. (1951). Field theory in the social sciences. New York, NY: Harper. Lewin, K & Grabbe, P. (1945). Conduct, knowledge, and acceptance of new values. The Journal

of Social Issues, 1 (3), 53-64.

83 Riccio & Darwin


Loewenstein, G., & Lerner, J.S. (2002). The role of affect in decision making. In: R. Davidson, K. Scherer, & H. Goldsmith (Eds.), Handbook of affective science (pp. 619–642). New York: Oxford University Press.

Loewenstein, G., Weber, E., Hsee, C., & Welch, N. (2001). Risk as feelings. Psychological Bulletin, 127 (2), 267–286.

Lorentz, H., Einstein, A., Minkowski, H., & Weyl, H. (1952). The principle of relativity: A collection of papers on the special and general theory of relativity. W. Perrett & G. Jefferey (Trans.). New York: Dover. (Original work published 1923)

Lukey, B. & Tepe, V. (Eds.) (2008). Biobehavioral resilience to stress. Boca Raton, FL: CRC Press.

Magolda, M.B.B. (1999). Creating contexts for learning and self-authorship: constructive-developmental pedagogy. Nashville, TN: Vanderbilt Press.

Maslow, A. (1964). Religions, values, and peak-experiences. New York: Penguin. Mâsse, L. C., Moser, R. P., Stokols, D., Taylor, B. K., Marcus, S. E., Morgan, G. D., Hall, K.L.,

Croyle, R.T., Trochim, W. (2008). Measuring Collaboration and Transdisciplinary Integration in Team Science. American Journal of Preventive Medicine, 35(2, Supplement 1), S151-S160.

Matsumoto, D., & Yoo, S. H. (2006). Toward a new generation of cross-cultural research. Perspectives on Psychological Science, 1(3), 234-250.

Mikroyannidis, A. (2007). Toward a social semantic web. Computer, November 2007, 113-115. Los Alamitos, CA: IEEE Computer Society.

Miles, M.B. & Huberman, A.M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage. Neisser, U. (1976). Cognition and reality. San Francisco: W.H. Freeman. Neisser, U. (Ed.) (1996). The perceived self: ecological and interpersonal sources of self

knowledge. Cambridge, UK: Cambridge University Press. Neisser, U., & Jopling, D.A. (Eds.) (1997). The conceptual self in context. New York: Cambridge

University Press. Paavola, S. & Hakkarainen, K. (2005). The knowledge creation metaphor – An emergent

epistemological approach to learning. Science and Education, 14(6), 535-557. Pellegrino, J., Chudowsky, N., & Glaser, R. (Eds.) (2001). Knowing what students know: The

science and design of educational assessment. Washington, DC: National Academy Press. Perry, R. & McEnery, K. (2009). Army reconnaissance course: Defining the aim point for

reconnaissance leader training. Armor, July-August, 14-20. Peterson, C., & Seligman, E. P. (2004). Character, strength, and virtues. Oxford, UK: Oxford

University Press. Plato (1977a). Phaedo. In: R.M. Hutchins et al. (Eds.). Plato. Chicago, IL: Encyclopedia

Britannica. (Reprinted from The Dialogues of Plato, by B. Jowett, Trans., 1892a, Oxford, UK: Oxford University.) (Original work written ca. 360 BCE).

Rasmussen, J. (1997). Risk management in a dynamic society: a modeling problem. Safety Science, 27, 183-213.

Riccio, G. (1993a). Information in movement variability about the qualitative dynamics of posture and orientation. In: K. Newell (Ed.). Variability and Motor Control (pp. 317-357), Champaign, IL: Human Kinetics.

Riccio, G. (1993b). Multimodal perception and multicriterion control of nested systems: Self motion in real and virtual environments. (UIUC-BI—HPP-93-02). University of Illinois at Urbana-Champaign: Beckman Institute for Advanced Science & Technology (also in NASA Technical Paper series 3703).

Riccio, G., & McDonald, P. (1998). Characteristics of EVA mass handling skill. Society of Automotive Engineers Paper No. 981625 (also NASA Technical Paper 3684).

Riccio, G., & Stoffregen, T. (1991). An ecological theory of motion sickness and postural instability. Ecological Psychology, 3, 195-240.



Riccio, G., Sullivan, R., Klein, G., Salter, M., & Kinnison, H. (2004). Warrior ethos: Analysis of the concept and initial development of applications. ARI Research Report 1827. Arlington, VA: US Army Research Institute for the Behavioral and Social Sciences.

Rick, S. and Loewenstein, G. (2008). Intangibility in intertemporal choice. Philosophical Transactions of the Royal Society B, 363, 3813–3824.

Ryan, R. & Deci. E. (2008). From ego depletion to vitality: Theory and findings concerning the facilitation of energy available to the self. Social and Personality Psychology Compass 2(2), 702–717.

Savory, J.R. & Duffy, T.M. (1995). Problem based learning: An instructional model and its constructivist framework Educational Technology, 35, 31-38.

Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences (pp. 97-118). New York: Cambridge University Press.

Shaw, R. & J. Bransford, J. (eds.) (1977). Perceiving, acting and knowing: toward an ecological psychology. Hillsdale, NJ: Erlbaum.

Shaw, R., & Kinsella-Shaw, J. (2007). The survival value of informed awareness. Journal of Consciousness Studies, 14(1), 137-154.

Sidman, J., Riccio, G., Semmens, R., Geyer, A., Dean, C., & Frederick Diedrich, F. (2009). Reshaping Army institutional training: Current training. Final Report to the Army Research Institute for the Behavior and Social Sciences, contract number W74V8H-04-D-0047 DO 0010.

Solomon, R. (Ed.) (2004). Existentialism. Oxford, UK: Oxford University Press. Spady, W. (1994). Outcome based education: Critical issues and answers. Arlington, VA:

American Association of School Administrators. Spady, W. (2009). Outcome based education. Florence, KY: Cengage Learning. Spady, W., & Marshall, K. (1991). Beyond traditional outcome-based education. Educational

Leadership 49 (2):67–72. Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and

quantitative approaches. Thousand Oaks, CA: Sage. Tobias, S. & Duffy, T. (eds.) (2009). Constructivist instruction: Success or failure. New York:

Routledge. Trochim, W. (2004). Concept mapping. In Matheson, S. (Ed.) Encyclopedia of Evaluation.

Thousand Oaks, CA: Sage. Trochim. W., Marcus, S.E., Mâsse, L.C., Moser, R.P., Weld, P. (2008). The evaluation of large

research initiatives: A participatory integrative mixed-methods approach, American Journal of Evaluation, 29, 1, 8-28.

Tversky, A. (1977). Features of similarity. Psychological Review, 84(4), 327-352. United States Army Training and Doctrine Command (2009). Army Leader Development Strategy

(Draft, May 21, 2009), Ft. Monroe, VA: Training and Doctrine Command. Vandergriff, D. (2006). Adaptive leaders course, Part 1: Old dogs teaching new tricks. Army,

November 2006, 59-66. Vandergriff, D. (2007). From swift to Swiss: Tactical decision games and their place in military

education and performance improvement. Performance Improvement, 45(2), 30-39. Van Merrienboer, J., & Kirschner, P. (2007). Ten steps to complex learning: A systematic

approach to four-component instructional design. New York: Routledge. Vaughan, D. (1997). The Challenger Launch Decision: Risky Technology, Culture, and Deviance

at NASA. Chicago, IL: University Of Chicago Press. Webster, D. S. (2001). In defense of the 'Natural Attitude': charting a return from the madness of

interpretation. Assemblage 6, Available at: http://www.shef.ac.uk/~assem/issue6/webster_web.html

85 Riccio & Darwin


Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York : Cambridge University.

Winograd, E., Fivush, R., & Hirst, W. (1999). Ecological approaches to cognition: Essays in honor of Ulric Neisser. Mahwah, NJ: Lawrence Erlbaum.

Wise, G. (1980). American historical explanations: A strategy for grounded inquiry. Minneapolis, MN: University of Minnesota.

Wolfram, S. (2002). A new kind of science. Champaign, IL: Wolfram Media.

ix Riccio, Diedrich, & Cortes (Eds.)


TABLE OF CONTENTS .

page Prologue: A Programmatic View of the Inquiry into Outcomes-Based Training & Education.......1

Historicity of our Research on OBTE..........................................................................................1The Approach and Lessons Learned from the Research..............................................................3Documentation of the Research ...................................................................................................4

Section I. Development of Stakeholder Requirements for OBTE..............................................6 Chapter 1. Preparation for Full Spectrum Operations ......................................................................7

1.1 Requirements of Full Spectrum Operations ...........................................................................81.2 Outcomes-Based Training and Education (OBTE)..............................................................10

1.2.1 Exemplar of OBTE: Combat Applications Training Course........................................111.2.2 OBTE as a Multifaceted Instructional System .............................................................12

1.3 An Appraisal of Instruction with Respect to OBTE ............................................................131.3.1 A Systems Engineering Framework for Integration and Development of OBTE........131.3.2 Preparation for Validation and Verification .................................................................14

1.4 References ............................................................................................................................17 Chapter 2. Formative Measures for Instructors ..............................................................................20

2.1 Development of Formative Measures ..................................................................................202.1.1 The COMPASS Methodology ......................................................................................202.1.2 Development of Measures for OBTE ...........................................................................21

2.2 Description of Formative Measures .....................................................................................212.2.1 Results of the COMPASS Process................................................................................212.2.2 Elaboration on the Description of Measures.................................................................23

2.3 OBTE Performance Measures: Planning for Training.........................................................232.3.1 Define Outcomes ..........................................................................................................232.3.2 Create a Positive Learning Environment ......................................................................252.3.3 Create the Parameters of Learning................................................................................27

2.4 OBTE Performance Indicators: Training Execution............................................................282.4.1 Communicate the Parameters of Learning....................................................................282.4.2 Training Emphasizes Broad Combat or Mission Success ............................................292.4.3 Customize Instruction When Possible Based on Constraints/Conditions ....................312.4.4 Facilitates Learning of Concepts ..................................................................................322.4.5 Creates a positive learning environment.......................................................................342.4.6 Instructors Utilize Measures of Effectiveness & Self-Evaluation ................................362.4.7 Uses scenarios to facilitate learning..............................................................................382.4.8 Instructors exhibit intangible attributes in own actions ................................................402.4.9 Hotwashes and Mini-AAR............................................................................................42

2.5 Uses of the Measures ...........................................................................................................432.5.1 Formative Measures for Instructors ..............................................................................442.5.2 Quality Assurance and Instructor Education ................................................................442.5.3 Continuous Improvement of Assessments....................................................................452.5.4 Program Evaluation and Organizational Change..........................................................46

2.6 References ............................................................................................................................46

Table of Contents x


Chapter 3. Principles and Practices of Outcomes Based Training & Education............................503.1 Multifaceted Inquiry.............................................................................................................50

3.1.1 Interaction with Progenitors of OBTE..........................................................................513.1.2 AWG Documents on OBTE .........................................................................................523.1.3 Collaborative Reflection on Participant Observation in CATC ...................................523.1.4 Interaction with Stakeholders .......................................................................................53

3.2 Essential Characteristics of OBTE.......................................................................................533.2.1 The Meaning of Developmental is a Critical Difference..............................................533.2.2 The Definition of Outcomes is a Critical Difference....................................................563.2.3 The Emphasis on Values and Causally Potent Intangibles is a Critical Difference .....583.2.4 The Meaning of Experience is a Critical Difference ....................................................613.2.5 The Emphasis on Instructor-Student Interactions is a Critical Difference ...................623.2.6 The Emphasis on Learning to Learn is a Critical Difference .......................................633.2.7 The Emphasis on Collaborative Design and Development is a Critical Difference.....65

3.3 Toward a Grounded Theory for OBTE................................................................................663.3.1 Need for an Integrated Interdisciplinary Framework ...................................................663.3.2 Formative Measures of Instructor Behavior as Evolving Best Practices of OBTE......67

3.4 Emerging Best Practices in OBTE for a Community-Centered Environment.....................683.4.1 Leadership and Enculturation of Soldiers.....................................................................683.4.2 Robust and Adaptable Plan...........................................................................................703.4.3 Instructors as Role Models ...........................................................................................703.4.4 Collaborative Identification of Outcomes and Measures .............................................71

3.5 Emerging Best Practices in OBTE for a Knowledge-Centered Environment .....................713.5.1 Integrated Understanding of Basic Soldier Skills in Full Spectrum Operations ..........723.5.2 Task Relevance of Planned Instructional Events..........................................................723.5.3 Reveal Operational Relevance of Training...................................................................733.5.4 Incorporate Stress into Instructional Events .................................................................733.5.5 Identify General Lessons Learned and Extrapolate to New Situations ........................74

3.6 Emerging Best Practices in OBTE for an Assessment-Centered Environment ...................743.6.1 Collaborative Reflection and Problem Solving ............................................................753.6.2 Communication.............................................................................................................753.6.3 Nature and Extent of Guidance.....................................................................................763.6.4 Establish a Pervasive Mindset of Collaborative Reflection..........................................76

3.7 Emerging Best Practices in OBTE for a Learner-Centered Environment ...........................773.7.1 Soldier Motivation and Development of Intangibles....................................................773.7.2 Plan for Development of the Individual .......................................................................783.7.3 Get Students to Take Ownership ..................................................................................783.7.4 Collaborative Reflection as a Means to Develop Self Efficacy....................................79

3.8 References ............................................................................................................................79 Chapter 4. Grounded Theory for Values-Based Training & Education .........................................86

4.1 Exploration of Holistic and Functionalistic Underpinnings for OBTE ...............................864.1.1 Fundamental Units of Analysis.....................................................................................874.1.2 Nested Time Scales and Adaptability ...........................................................................884.1.3 Adaptability and Ambiguity .........................................................................................904.1.4 Mechanistic Analogies and Predominant Experimental Paradigms .............................92

4.2 Three Pillars for the Scientific Foundation of OBTE ..........................................................934.2.1 Ecological Psychology..................................................................................................934.2.2 Self-Efficacy Theory.....................................................................................................974.2.3 Positive psychology ......................................................................................................98

4.3 A More Integrated Scientific Infrastructure .......................................................................101

xi Riccio, Diedrich, & Cortes (Eds.)


4.3.1 Self Determination Theory .........................................................................................1014.3.2 Situated Learning Theory ...........................................................................................1034.3.3 Existential Psychology................................................................................................105

4.4 Building on the Scientific Infrastructure for OBTE...........................................................1094.4.1 Triadic Frameworks ....................................................................................................1094.4.2 Further Development ..................................................................................................112

4.5 References ..........................................................................................................................112 Chapter 5. Passion and Reason in Values-Based Learning & Development ...............................118

5.1 The Nested Self ..................................................................................................................1185.1.1 An Alternative to Individual versus Collective ..........................................................1185.1.2 Cognition and Reality .................................................................................................119

5.2 Conscious Experience and the Dynamics of Thinking ......................................................1225.3 Emotion, Information, and Engagement ............................................................................125

5.3.1 Ecological Perspective on Emotion ............................................................................1255.3.2 Emotion as Engagement .............................................................................................1265.3.3 Implications for Training and Education ....................................................................129

5.4 Emotion, Decision-Making, and Inter-Temporal Choice...................................................1295.4.1 Toward a More Integrated Theory..............................................................................1295.4.2 Emotion and Decision-Making...................................................................................1305.4.3 Emotion and Nested Time Scales ...............................................................................1315.4.4 Neuroeconomics and Inter-Temporal Reasoning .......................................................1325.5.5 Inter-Temporal Reasoning and Adaptive Dynamical Systems...................................133

5.5 Beyond Science ..................................................................................................................1345.5.1 Existentialism..............................................................................................................1345.5.2 The Soldier-Scholar as an Emergent Property of a Collective Pursuit.......................135

5.6 References ..........................................................................................................................137 Section II. Verification and Validation of OBTE as a Service System ..................................142 Chapter 6. Initial Impressions of Participation in CATC .............................................................143

6.1 Methods..............................................................................................................................1436.1.1 Participants..................................................................................................................1436.1.2 Procedure ....................................................................................................................1436.1.3 Analyses......................................................................................................................144

6.2 Results ................................................................................................................................1446.3 Implications for Service System Development: Peer Review ...........................................1466.4 References ..........................................................................................................................147

Chapter 7. Local Development of Measures of Effectiveness .....................................................149

7.1 What do Instructors Believe Soldiers Should Learn in Initial Entry Training? .................1497.2 Measure Development Process ..........................................................................................1507.3 What do OBTE-Trained DS Believe is Important to Assess in BRM/ARM? ...................1517.4 Implications........................................................................................................................1567.5 Conclusions ........................................................................................................................1587.6 References ..........................................................................................................................159

Table of Contents xii


Chapter 8. Observations of Behavior and Communication in Rifle Marksmanship Training .....1608.1 Methods..............................................................................................................................160

8.1.1 Participants..................................................................................................................1608.1.2 Procedure ....................................................................................................................1608.1.3 Analyses......................................................................................................................161

8.2 Results ................................................................................................................................1638.2.1 Behavior of DS ...........................................................................................................1638.2.2 Behavior and Performance of Privates .......................................................................1658.2.3 Patterns of Communication ........................................................................................1688.2.4 Potential Influence of Instructor Behavior on Performance of Privates .....................170

8.3 Implications for Service System Development..................................................................1718.3.1 Verification of OBTE .................................................................................................1718.3.2 Validation of OBTE....................................................................................................172

8.4 References ..........................................................................................................................173 Chapter 9. Impact on Rifle Marksmanship Training....................................................................174

9.1 Behavioral Data Collection During Basic Rifle Marksmanship ........................................1749.1.1 Method ........................................................................................................................1749.1.2 Assessment..................................................................................................................1759.1.3 Results – An Overview ...............................................................................................1779.1.4 Evidence for Influence of OBTE ................................................................................1789.1.5 Behavior of Drill Sergeants after Exposure to OBTE ................................................1809.1.6 Behavior of Privates....................................................................................................1829.1.7 Patterns of Communication ........................................................................................1869.1.8 Summary.....................................................................................................................186

9.2 Attitudes Toward an OBTE in Basic Training...................................................................1879.2.1 Method ........................................................................................................................1879.2.2 Results.........................................................................................................................187

9.4 References ..........................................................................................................................191 Chapter 10. Influence of CATC in an Operational Setting ..........................................................192

10.1 Methods............................................................................................................................19210.1.1 Participants................................................................................................................19210.1.2 Procedure ..................................................................................................................19210.1.3 Analyses....................................................................................................................193

10.2 Results ..............................................................................................................................19310.2.1 Downstream Impact on Marksmanship ....................................................................19310.2.2 Downstream Impact on Training in the Units ..........................................................19410.2.3 Downstream Impact on Self Efficacy .......................................................................195

10.3 Implications for Service System Development: Validation.............................................19610.4 References ........................................................................................................................197

Chapter 11. Implications for Service System Development.........................................................198

11.1 Lessons Learned about Transfer of OBTE.......................................................................19811.2 Implications for Service System Development................................................................199

11.2.1 Further Development and Analysis of Stakeholder Requirements for OBTE..........19911.2.2 Further Development of OBTE as a Service System ...............................................19911.2.3 Further Verification and Validation of OBTE..........................................................201

11.3 References ........................................................................................................................203

xiii Riccio, Diedrich, & Cortes (Eds.)


Section III. Further Development of OBTE as a Service System ..........................................206 Chapter 12. Development of General Measures for Students ......................................................207

12.1 Intent ................................................................................................................................20712.2 Performance Measure Development Process...................................................................207

12.2.1 Phase One: Define Performance Indicators (PI).......................................................20712.2.2 Phase Two: Translate PI into performance measures ...............................................20812.2.3 Phase Three: Measure refinement.............................................................................20812.2.4 Phase Four: Retranslation of Measures ....................................................................208

12.3 Product of Measure Development....................................................................................20912.3.1 Learner Perception of the Instructor and Course ......................................................20912.3.2 Learner Engagement .................................................................................................21112.3.3 Student Relationship with Teacher ...........................................................................21212.3.4 Student Results .........................................................................................................21412.3.5 Self-Report Measures ...............................................................................................216

12.4 Conclusion........................................................................................................................21712.5 References ........................................................................................................................217

Chapter 13. Adapting OBTE in a Classroom Environment .........................................................219

13.1 Intent ................................................................................................................................21913.2 Observing OBTE in the Classroom Environment............................................................219

13.2.1. Participants...............................................................................................................21913.2.2. Procedure .................................................................................................................22013.2.3. Measures ..................................................................................................................220

13.3 Utility of OBTE Measures in a Classroom Environment ................................................22013.3.1 Generality of Measures .............................................................................................22013.3.2. Implications for Improvement of Measures.............................................................22113.3.3 Implications for improvement of course design .......................................................222

13.4 Use of 360° Reviews for Collaborative Reflection..........................................................22313.4.1 The Role of a 360° Review in OBTE .......................................................................22313.4.2 Narrative of a Participant Observer ..........................................................................225

13.5 Learning, cognitive load and motivation..........................................................................22813.5.1 The NASA Task Load Index as a subjective measure of student workload.............22813.5.2 Results.......................................................................................................................22913.5.3 Implications ..............................................................................................................230

13.6 Conclusions ......................................................................................................................23013.7 References ........................................................................................................................231

Chapter 14. Organizational Climate and Creation of Durable Change ........................................233

14.1 The Need ..........................................................................................................................23314.2 Initial Indications of Possible Resistance to Change .......................................................23414.3 Models and Considerations for Sustainable Change........................................................235

14.3.1 The Change Transition Period ..................................................................................23514.3.2 Organizational Culture..............................................................................................23714.3.3 Clarity of Mission and Shared Understanding..........................................................23714.3.4 Relevant Observations During the Current Investigation.........................................23814.3.5 Organizational Support and Incentives.....................................................................238

14.4 Conclusions ......................................................................................................................23914.5 References ........................................................................................................................239

Table of Contents xiv


Chapter 15. Five ways OBTE can enable the Army Leader Development Strategy....................24215.1 Background ......................................................................................................................24215.2 An Emerging Consensus ..................................................................................................244

15.2.1 What Part to Balance?...............................................................................................24415.2.2 Improving Training, by Design ................................................................................24515.2.3 Increased Use of dL and Dependence on Self-Development ...................................24615.2.4 Future Orientation, Unknown Requirements............................................................24715.2.5 The Quality Instructor Challenge .............................................................................24715.2.6 Purpose and Design are Key.....................................................................................24815.2.7 A Natural Advantage ................................................................................................24915.2.8 Task Specialization or Generalized Competency .....................................................249

15.3 Conclusion........................................................................................................................25115.4 References ........................................................................................................................252

Epilogue. Integration of Leader Development, Education, Training, and Self-Development .....254

Toward Values-Based Standards for Army Doctrinal Requirements ......................................254Nested Standards and Quality Assurance.................................................................................256Needs and Opportunities for Staff & Faculty Development ....................................................259

A Role for Science and Measurement .................................................................................259Toward Best Practices in Instructor Education....................................................................260

Critical Considerations for Further Scientific Investigation ....................................................263The Necessity of Long-Term Studies ..................................................................................263False Dichotomy of Objective-Subjective ...........................................................................264Clarity About What Is Evaluated.........................................................................................265Next Steps ............................................................................................................................266

References ................................................................................................................................268 Section IV. Appendices...............................................................................................................270 Appendix A. OBTE Principles & Practices: Instructor Measures................................................271

A.1 Genesis of Formative Measures for Instructors ................................................................271A.2 Principles of Outcomes-Based Training & Education ......................................................272A.3 Guide to Using Measures of Instructor Behavior..............................................................276A.4 Complete Menu of Instructor Measures............................................................................279

Appendix B. OBTE Principles & Practices: Student Measures ...................................................318

B.1 Guide to Using Measures of Student Behavior .................................................................318B.2 Complete Menu of Student Measures ...............................................................................319

Appendix C: A Commander’s View of Outcomes-Based Training and Education .....................340

Summary ..................................................................................................................................340Definition .............................................................................................................................340Description...........................................................................................................................340

Elements of OBTE. ..................................................................................................................341Developing the Outcomes....................................................................................................341Developing the Training Plan ..............................................................................................341Conducting Training ............................................................................................................342How Training is Assessed....................................................................................................344

Conclusion................................................................................................................................344

xv Riccio, Diedrich, & Cortes (Eds.)


Appendix D: Warrior Ethos..........................................................................................................345Analysis of the Concept and Initial Development of Applications..........................................345

Current Understanding of Warrior Ethos.............................................................................345Purpose.................................................................................................................................348Approach..............................................................................................................................348Expansion of the Definition of Warrior Ethos.....................................................................348The Tenets of Warrior Ethos ...............................................................................................349Clarifying the Definition of Warrior Ethos..........................................................................351Warrior Attributes Derived from the Tenets of Warrior Ethos ...........................................353

References ................................................................................................................................355Supplementary Work Product from Warrior Ethos Project .....................................................355

Appendix E: Indicators of Warrior Ethos.....................................................................................356

Methods....................................................................................................................................356Participants...........................................................................................................................356Instruments and Facilities ....................................................................................................356Procedure .............................................................................................................................356

Results ......................................................................................................................................358Qualitative Findings.............................................................................................................358Quantitative Findings...........................................................................................................358

Discussion ................................................................................................................................359

Evolution of the investigation as reflected in the chapters of this monograph.