G E N E V I È V E B E N I N G E R

8 M a r c h 2 0 1 6

Design-Based ResearchA method for achieving Impact in the real world

An Overview: What did you call it?

A Rose by any other name…

Design-Based Research

Design Experiments (Brown)

Educational Design Research (Reeves)

Development Research

Design Research

What is Design-Based Research?

Wang and Hannafin (2005) define DBR as:

“A systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in real-world settings and leading to contextually-sensitive design principles and theories” (pp. 6-7).

The Foundations of DBR

Anne Brown (1992):

"Classroom life is synergistic: Aspects of it that are often treated independently, such as teacher training, curriculum selection, testing, and so forth actually form part of a systemic whole. Just as it is impossible to change one aspect of the system without creating perturbations in others, so too it is difficult to study any one aspect independently from the whole operating system." (pp. 141-143)

The Foundations of DBR

Anne Brown (1992):

“Theoretical advances can emerge from both the laboratory and classroom settings. They are just that, different settings whose features must be included in the description of the data they produce." (p. 154)

The Foundations of DBR

Barab & Squire (2004):

"design-based research and other methods should be viewed as complementary and supportive—allowing researchers to understand more completely their claims. For example, laboratory-based researchers should ask themselves how their laboratory-based claims would benefit from further testing in naturalistic contexts and design-based researchers should be asking how their claims

would benefit from more

rigorous testing within

laboratory-based contexts."

(Note, p. 4)

An Overview: How does it work?

McKenney and Reeves’ (2012) generic model for design research (GMDR) describes three phases to the design-based research process:

1. Analysis and exploration;

2. Design and construction;

3. Evaluation and reflection.

An Overview: How does it work?

Generic model for design research in education (McKenney & Reeves, 2012)

Isn’t it “Just a Project”?

Key principles that differentiate design-based research from other forms of participatory research (and projects):

http://www.cea-ace.ca/education-canada/article/design-based-research

requirement for a well-defined problem with a research-informed design solution;

testing of theory in real-world contexts;

contribution to theory and practice in addition to local impact.

How does it compare to other approaches?

Why not existing research methods?

(Collins, Joseph, & Bielaczyc, 2004, pp. 20-21).

laboratory and training settings do not account for multiple variables, multiple participants’ expertise, and “the messy situations that characterize real life learning”;

ethnographic research describes in detail what and why relationships and events occur, but it does not try to change practice;

large-scale studies “do not provide the kind of detailed picture needed to guide the refinement of a design”

How does it compare to other approaches?

Barab & Squire (2004):

"The goal is not to “sterilize”

naturalistic contexts from all

confounding variables so the

generated theory is more valid and reliable.

Instead, the challenge is to develop flexibly adaptive theories that remain useful even when applied to new local contexts." (p. 11)

A Question of Alignment

“Design-based research suggests a pragmatic philosophical underpinning, one in which the value of a theory lies in its ability to produce changes in the world." (Barab & Squire, 2004, p. 6)

It asks “What can be done?” and “How can we effect change?” Seeks practical solutions

Ontology: Reality is complex - the effects of ideas

Epistemology: both objective and subjective points of view, knower seeks to uncover and apply the known

Methodology: Mixed Methods, combinations of qualitative and quantitative, whatever leads to a practical solution!

“‘to understand what people mean and intend by what they say and do and to locate those understandings within the historical, cultural, institutional, and immediate situational contexts that shape them’ (Moss et al., 2009, p. 501).

It asks “How is this understood?” and“What is the meaning behind this?”

Ontology: Relativist – there are multiple realities

Epistemology: subjective - knower and respondent co-create understandings

Methodology: Qualitative (ethnographies, interviews, case studies…)

PRAGMATIC? INTERPRETIVIST?

Example: COAT Project

MarylandOnline’s Certificate for Online Adjunct Teaching (COAT) course

"The research project explored whether the training course had any impact on the participants’ later teaching practice. The major outcome of this research study is the identification of design principles that can be used by other researchers and practitioners designing online instructor training." (Shattuck & Anderson, 2013, p. 1)

Using a Design-Based Research Study to Identify Principles for Training Instructors to Teach Online http://www.irrodl.org/index.php/irrodl/article/view/1626/2710

Example: COAT Project Cont’d

"This study collected data using online, asynchronous, threaded discussion groups as focus groups…”

Challenges

“Difficulties arising from the complexity of real-world situations and their resistance to experimental control.

Large amounts of data arising from a need to combine ethnographic and quantitative analysis.

Comparing across designs."

(Collins, Joseph, & Bielaczyc, 2004, p. 16)

Challenges Cont’d…

the role of the researcher

the time it takes to enact multiple cycles of a design

knowing when to stop the cycles of iteration (Hogue, 2013).

Challenges…?

“There is a fundamental challenge in developing a design science of education in that the enacted design is often quite different from what the designers intended. Brown and Campione (1996) referred to this problem in terms of “lethal mutations,” where the goals and principles underlying the design are undermined by the way the design is enacted.” (Collins, Joseph, & Bielaczyc, 2004, p. 17)

Q: Is this a “challenge” or should we embrace it as reality?

Meeting the Challenge(s)!

Design-Based Research Collective…

“to refine a definition of design experimentation that is broad enough to encompass a diversity of research perspectives, yet rigorous enough to sustain theoretical and methodological attacks on its robustness and cumulativity”

Closing thoughts…

Combination of qualitative and quantitative “Mixed Methods” is appropriate to the complexity of education, learning, and context

Exciting approach – starting to mature but needs refinement!

Addresses a strong need to bridge the gap between research and real-world application

Research must not only make sense, but must be valuableto educators, learners, and future research!

“Where’s the BEEF ?”

References

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of Learning Sciences, 2(2), 141–178.

Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The journal of the learning sciences, 13(1), 1-14.

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15-42. doi.10.1207/s15327809jls1301_2

Hogue, R. J. (2013). Epistemological Foundations of Educational Design Research. In E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (Vol. 2013, No. 1, pp. 1915-1922).

McKenney, S. E., & Reeves, T. C. (2012). Conducting educational design research. New York, NY: Routledge

Shattuck, J., & Anderson, T. (2013). Using a design-based research study to identify principles for training instructors to teach online. The International Review Of Research In Open And Distributed Learning, 14(5). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1626/2710