Design Experiment Technologies

7/27/2019 Design Experiment Technologies

1/7

Design Experiment Technologies

Technology is a prominent component of design experiments. This is partially an opportunistic situation.Computer technology has only recently made design experiments feasible in terms of allowing for the easycollection of rich data from complex settings, facilitating the analysis of that data, and making innovativelearning technologies possible. Technology enables design experiments mainly through providing an adaptiveand powerful medium. Technology affords new instructional goals and practices, support for different forms oflearning, mechanisms for distributing curricular innovation, computational models of cognition, and more.

Of course, technology must never be considered a panacea for educational woes. We do believe, however, thattechnology can act as a catalyst for educational reform. In terms of reform-focused outcomes, design experimenttechnology can take the form of: (a) an increased theoretical understanding of educational issues, or (b)educational products (i.e. curriculum, tools, or resources). If the focus is the former outcome, technology servesas a mechanism for better understanding issues relating to educational reform. Given a focus on the latteroutcome of educational products, design experiment technology can serve as a new medium for personalexpression or as a conduit for instructional reform. With regard to these reform-minded goals, technology doesnot stand alone. The use of design experiment technology is often wedded to other efforts in curriculumdevelopment, teacher development, research methodologies, school resource management, and studentassessment.

In terms of conducting design experiments, we divide design experiment technology into four classes:

1. technology for educational innovation,2. technology for teacher development,3. technology as a research tool, and4. technology to support dissemination.

In order to properly harness technology in these various arenas, it is necessary to understand the purposes of thetechnology in each arena, the forms technology can take in each arena (or genres), as well as the associatedissues which arise.

Technology for Educational Innovation

Educational technologies developed as part of design experiments run the gamut of possible forms frommicroworlds to collaborative spaces to scripted courseware. They are often developed with different goals inmind.

Goals for Developing Educational Innovations Using Technology

A common goal for developing and using educational technology in general is to improve students' learning insome manner. Design experiments also frequently share this goal, but push the technological innovation one stepfurther. Many design experimenters are interested in developing design principles for educational technology

(diSessa, 1991; Linn, 1995).

The goal is that if design principles could be derived which map technological design to student outcomes, theseprinciples could be used to inform the design of future innovative technologies and curriculum. Surely, this is acomplex endeavor given the range of connections between the technology and the other components of thedesign experiments described in this document. Additionally, other goals for educational innovation can bepursued as well which are more pragmatic in the short-run.

Common Genres of Educational Technology Used in Design Experiments

Educational innovations which make use of technology come in a broad variety of forms and functionalities.

6/27/0

e and Design Pages: Design Experiment Technologies http://www.soe.berkeley.edu/sandhtdoc


2/7

While computer technology is an amazingly plastic medium in many regards, the range of existing technologiescan be relatively cleanly separated into distinct genres based on the functionality of the systems. These genres ofeducational technology are, of course, somewhat fuzzy, but they will serve the purpose here of painting thetechnological landscape and bringing up salient issues with regard to design experiments.

The following are genres of educational technology which are often used as part of design experiments. Anormative definition and pointers to exemplars are provided for each genre.

Microworlds & SimulationsDEFINITION: exploratory environments which systematically model relationships between abstractcomponentsEXAMPLES: Logo, ThinkerTools, Model-It, Boxer, GenScope

Direct InstructionDEFINITION: multimedia courseware, databases, or computer tutoring about a specific domainEXAMPLES: Academic Systems, Geometry Tutor, Kane (Spiro)

Data Collection & Visualization Tools

DEFINITION: systems which allow for the pooling and representation of data, often in scientificallyauthentic waysEXAMPLES: BlueSkies / KGS, CoVis Climate S/W, (TERC s/w)

Collaboration Tools

DEFINITION: software environments which allow groups of individuals to interact with each other or withcommunal resourcesEXAMPLES: SpeakEasy, Belvedere, Collaboratory Notebook

Learning PartnersDEFINITION: a hybrid of computational tools and instructional environments which scaffold studentinquiryEXAMPLES: KIE, CLP E-LabBook, Convince Me!, BGuILE

Computational MediaDEFINITION: sophisticated and extensible software environments tailorable be tailored for particularpurposesEXAMPLES: Boxer, Lotus Notes

Dimensions which can be used to characterize the educational technology genres include: (a) the degree towhich they are domain-general vs. domain-specific systems, (b) the level of scaffolding available in theenvironment, and (c) whether the system is individual or collaborative. It is left as an exercise for the reader tomap the aforementioned genres to the dimensions listed.

Issues

A range of issues present themselves as we consider the range of technological innovations being used in designexperiments. These issues are presented along with brief discussions.

Driving Forces During Development

There are a variety of different forces at work when designing educational technologies for design experiments,including: (a) technological availability, capabilities, and constraints, (b) prior instructional goals and practicesof the participants (teachers, researchers, and students), (c) current instructional goals, (d) theoretical influencesfrom conceptual and instructional frameworks, and (e) impacts deriving from research interests.

6/27/0



3/7

Flexibility and Adaptation

An important set of design issues for these technologies involves the degree to which they are adaptable toindividual students or to particular classroom settings. Some software environments keep track of studentprogress and respond accordingly during the instructional experience (e.g. Belvedere, Geometry Tutor). Thiskind of instructional feedback is often a focus of research for these systems since many aspects of the feedbacktechnology involve open research questions. It can come in the form on domain-general or domain-specific

feedback.

Other software systems are customizable by the teacher in that the instructional experience for the students canbe modified to fit the particular curricular needs of their classroom (e.g. KIE). This has some implications foradoption of the innovation, especially for reform-minded teachers who wish to become closely involved in thecurriculum development process. This will be discussed more in the next section on teacher developmenttechnologies.

Another aspect of system flexibility involves the degree to which the system allows for cross-subject instructionor integration with other pieces of classroom curriculum. Much of the direct instruction and microworld /simulation genres are not easily extensible in this way because of their tight connections to the domain ofinstruction. On the other hand, genres like computational literacies and collaboration tools are often moreextensible.

Hybrid Educational Technology Environments

An added amount of complexity comes up when more than one genre of educational technology are usedconcurrently during instruction. Synergistic aspects of these genres may complicate design experiments andmake them quite different from instances of using tools from the different genres in isolation.

For example, imagine a learning environment which coordinates a computer simulation environment along witha data collection and visualization system. This type of system could ask students to not only construct adynamic model for some phenomena, but could also ask them to coordinate it with real-world data. It is likely

that the interaction between these technologies-from a cognitive perspective on the students' learning-may resultin quite a different outcome than considering the outcomes of these two technologies in isolation. Students aremore likely to engage in a cyclical exploration of the model and the data.

Role of Technological Intuitions & Literacies

In a way that is not altogether understood, people approach new technologies attempting to make use of theirprior knowledge of other technologies (Norman-mental models, ???; others???). Some educational technologygenres (and their underlying components) are likely to more familiar to people than others. Much of directinstruction (videos, pictures, and text) are familiar to students. On the other hand, computer modeling as used byscientists is likely to be not nearly as familiar. People will have to become literate with new technologies to

different degrees depending on how different the new technology is to what they have experienced previously.

This has implications for the designers and disseminators of educational technologies since people's intuitionsare likely to impact the adoption and correct use of the technology. If possible, it seems like a good strategy tobuild on the existing technological literacies and intuitions of students whenever possible as long as it does notcompromise the design. When it is necessary to introduce a new technological component, it is best to design thecomponent as clearly as possible and to introduce it to students in a sensible manner.

Cognition in the Wild Classroom

For a significant portion of the cognitive science history, the research spotlight has shone primarily in laboratory

6/27/0



4/7

settings. Design experiments differ by attempting to shed light on issues as they arise in actual classrooms andother real-world settings. The data derived from such a setting are more complex while also being moreauthentic in terms of how it describes the educational innovation process. In shifting away from the controlled"testing" of single hypotheses, design experiments call on more sophisticated theory-building processes.

Design experiments are challenging to pull off not only for theoretical reasons, but also for very pragmatic ones.Innovative technology (especially that created without the benefits of a sizable quality assurance department) islikely to require more technical support than off-the-shelf software. This should be expected and planned for as

much as possible. It is important to test, debug, and refine technology quickly within the confines of a classroomsetting so students have the opportunity to move "beyond" the technology and engage in the content of theinstruction.

Innovative design experiment technology also usually requires more conceptual support for students andteachers as well. That is to say, that the genres listed above require specific types of scaffolding to make themapproachable to students. For example, the visualization genre may involve engaging students in a hands-onintroduction to the different representations before they will understand how they are used in the software.

Technology for Teacher Development

Teacher development is often an important component of design experiments. The development of neweducational innovations usually requires the teacher to approach instruction differently or at least learn about theparticulars of the innovation.


The goals of using technology for teacher development can include: (a) introducing teachers to the materials tobe used in a design experiment, (b) learn about conceptual frameworks for instruction and learning, (c) engagein reflection about their own practice, or (d) develop materials which can be useful to other teachers wanting touse a particular innovation or reform their teaching method.

Common Genres of Teacher Development Technology

The following are genres of teacher development technology which have been used as part of designexperiments. A normative definition and pointers to exemplars are provided for each genre.

Classroom Case Studies

DEFINITION: collections of materials which describe teaching and classroom practices associated withdesign experimentsEXAMPLES: Frederiksen Video Project, (CSCL Article)

Collaboration Tools

DEFINITION: software environments which allow groups of teachers to interact with other people and

communal resourcesEXAMPLES:

Curriculum PlannersDEFINITION: software tools which allow teachers to organize classroom activities based on a range ofcriteria (e.g., national standards)EXAMPLES: (Michael Jay's curriculum tool)

Knowledge ToolsDEFINITION: software environments which can be used to represent knowledge spacesEXAMPLES: SemNet, Gains' concept mapper, KIE SenseMaker

6/27/0



5/7

Direct Instruction & Tutorials

DEFINITION: multimedia presentations about a particular topicEXAMPLES: Annenberg Constructivist Teaching Videos

Classroom Research ToolsDEFINITION: technology which allows teachers to better understand their classroom environmentEXAMPLES: NDEC Video Guide

Issues

With regard to using technology to support teacher development during design experiments, a range of issuespresent themselves.

Sensitivity to Prior Teaching Experience

Most teachers have diverse and broad experience about teaching from their classrooms. When technology isused to foster teacher development, the materials should be sensitive to this prior experience. Teachers should beencouraged to bring their teaching intuitions and experience into the development process. For example, the roleof case studies can be to help teachers reflect upon their experience based on approaches taken by other teachers

It Takes More Than Technology To Make A Community

The most powerful use of technology for teacher development may be to allow teachers to communicate withothers in their profession who face similar issues. Collaboration tools like electronic mail and newsgroups canconnect teachers with similar interests. At the same time, it should be noted that it is very difficult to develop aproductive community using only technological communication. A face-to-face meeting-on even an infrequentbasis-can provide the social glue necessary to form a community.

Teacher Development is a Difficult Process

Reflecting upon and revising one's teaching is a difficult endeavor to say the least. The endeavor includescultural and individual constraints. Culturally, teachers are embedded in a community with various group normsand mechanisms which influence what they do in the classroom. Beyond these constraints, changing one'sinstructional practice can be a difficult instance of conceptual change at an individual level. We would expectthat changing one's view of classroom instruction (e.g., becoming a guide on the side rather than a sage on astage) to be similar to children changing their scientific ideas about the physical world.

Technology as Research Tool

Technology is also an important tool for the researcher of a design experiment. Recent technologies likevideotaping have made it possible to capture the complex classroom setting with greater fidelity than previously

possible. Similarly, technologies like databases have afforded more complex analyses of rich classroom data.The role of technology in research is also strongly influenced by the information processing perspective oncognition. Beyond the particular approach of modeling cognition using computers, this perspective has shedlight on the possibilities of using computers to model cognition in general.


The goals of using technology as a research tool include: (a) facilitating the collection of classroom researchdata, (b) aiding in the analysis of that data, and (c) providing a means of developing a deeper theoreticalunderstanding of design experiment issues.

6/27/0



6/7

Common Genres of Research Technology

The following are genres of technology used as a research tool during design experiments. A normativedefinition and pointers to exemplars are provided for each genre.

Classroom Data Collection Tools

DEFINITION: tools which capture aspects of the classroom context during design experimentsEXAMPLES: video cameras, tape recorders, computer log files

Statistical PackagesDEFINITION: software environments which allow for the statistical analyses of quantitative dataEXAMPLES: StatView, Systat, Stata

Database Packages

DEFINITION: software environments which allow for a structured collection of data to be constructed andanalyzedEXAMPLES: NUD*IST, FileMaker

Video Analysis Packages

DEFINITION: tools which support the interpretation and analysis of video dataEXAMPLES: C-Video

Computational ModelsDEFINITION: simulations or models of theoretical aspects of design experimentsEXAMPLES: ACT*R, Swarm

Issues

With regard to using technology to do research associated with design experiments, a range of issues presentthemselves.

Plasticitiy of Technology Hinders Research Collaboration

Technology is a plastic medium in many respects. Researchers have adapted this technology to suit their ownparticular purposes. In particular, technology affords new methodologies. This represents a double-edged swordin that although it allows for a tailored representation and analysis of research data, it also hinders consensuswithin the research community. That is, specific implementations and adaptations of technology can be difficultto convey to other researchers which makes forward progress as a community difficult as each researcher isacting independently in terms of methodology. At the same time, of course, the search for new powerfulresearch representations is a central feature of design experiments. It remains an open question whether or notanalysis standards will be established and broadly adopted. The ability of technological components to be easilyshared makes this prospect very feasible.

Computational Modeling May Be Counterproductive

Invariably, technology simultaneously affords and constrains what we do with it. This is made extremely clearwhen one considers the use of technology to model theoretical aspects of design experiments (e.g., individuallearning). Successfully engaging in such an undertaking is certainly limited by our current theoreticalunderstanding of educational innovations in classrooms. In addition, current computer technology may also beconstraining the development of productive models. Many early attempts at modeling human cognition resultedin highly symbolic systems which were brittle in their performance. More successful models of cognitiveprocesses (of groups and individuals) will likely require the development of new computational forms.

6/27/0



7/7

Technology to Support Dissemination

Dissemination of educational innovations is becoming an increasingly important component of designexperiments. At the same time, design experiments are often wedded to educational innovations which includeinstructional approaches and principles which are difficult to convey. Although we will not discuss it in detailpresently, technology can play a facilitating role in the dissemination of educational innovations. For example,network technology can be used to convey instructional materials or provide support for teachers in theirclassrooms. Video technology can convey an instructional approach better than previous written forms. The

issues which result include how well educational principles can be conveyed through technology and thespecific types of support needed in design experiments.

Dissemination also involves equity issues regarding who adopts innovations based on technological constraints(incidental filtering). The following is a list of potential filters for the appropriation of "reformed" instruction:

Having the necessary technology.Fit with classroom curriculum needs.Compatibility with written and spoken languages.Pedagogical approach. (For example, a curriculum that "doesn't give the right answer" may seeminefficient and stupid to some.)

Availability of classroom support.Perceived usefulness of software.

[ ]

1996 UC [email protected]


Documents

Design Experiment Technologies