A Teaching with Technology White Paper - Classroom Response Systems

8/14/2019 A Teaching with Technology White Paper - Classroom Response Systems

1/14

A Teaching with Technology White Paper

http://www.cmu.edu/teaching

ClassroomResponseSystems

Ashley Deal | [email protected] | 11.30.2007

Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License http://creativecommons.org/licenses/by-nc-nd/3.0/us/

How to oster meaning ul engagement among students is a long-

standing question in large lecture halls. In e ort to address this

issue, electronic classroom response systems (CRS) have beentested and used in higher education classrooms since the 1960s.

The studies summarized in this paper show that CRS can acili-

tate the process o drawing out students prior knowledge,

maintaining student attention, and creating opportunities or meaning ul engagement. They can also assist instructors in

assessing student comprehension and developing classroom

activities that allow or the application o key concepts to

practical problems.


2/14

Classroom Response Systems 2

Teaching with Technology November 2007

There arethree generalcategorieso activitiesand equipment

involved inusing aclassroomresponsesystem:

Instruction andquestioning

Response and

display Datamanagementand analysis W

o r k i n g

D e f n

i t i o n A classroom response system (CRS) is anysystem used in a ace-to- ace setting to

poll students and gather immediate eed-back in response to questions posed byinstructors. A non-technical example o a CRS is an instructor asking students toraise their hands to agree or disagree with

a given question. A slightly more sophisti-cated practice involves the use o coloredashcards, with each color correspondingto a possible response in a multiple-choicequestion.

Over the past 30 years, technologistshave developed and refned electronicresponse systems that allow students tokey in responses using transmitters (alsocalled remotes or clickers). Themain advantages o electronic responsesystems over non-technical methods orgathering eedback are the anonymity o

responses, and the ability to immediatelyproject response graphs overhead or theclass to see. Electronic response systemscan also store response data or utureanalysis and assessment.

There are three categories o activitiesand equipment involved in using a class-room response system: presentation andquestioning, student response and display,and data management and analysis.

Instruction and questioning So tware or most classroom responsesystems has been designed to integratewith common presentation so tware, likeMicroso t PowerPoint. Some additionale ort is required to develop questionslides, but since many instructors alreadyuse presentation so tware (particularlyinstructors in large lecture courses, wherethe use o CRS is most appealing), theextra e ort is minimized.

The kinds o questions posed by theinstructor can range rom simple actualrecall to questions designed specifcally toreveal and challenge common misconcep-tions in a given topic. Development o e ective questions is crucial to the successo teaching with CRS, and is discussed indetail in a later section.

In class, the instructor presents con-cepts and materials, interspersed withslides asking or eedback rom students.Questions are typically in true or alse ormultiple choice ormat. Question slidescan be placed in line with regular lecture

presentations so instructors can gathereedback on the y, without switchingapplications during the presentation.Students are typically given a short periodo time to key in responses.

Response and display

Students key in responses using smallremote transmitters. These transmitterssend signals to a receiver that is con-nected to the instructors laptop or lecternPC. So tware on the instructors machineinstantly tabulates and graphs studentresponses, and these simple graphs can bedisplayed on the ollowing presentationslide. One o the more compelling aspectso using CRS is that students can com-pare their own responses to the responseso other students in the class, which canencourage a level o metacognition thatmight not otherwise occur.

Once students see the distributiono responses, many instructors take theopportunity to encourage discussion, ask-ing students to reconsider the question ingroups and to reach an agreement aboutthe best response. Instructors o ten ol-low the discussion with a second cycle o questioning, response, and display be orewrapping up the presentation o a givenconcept. This approach is o ten re erredto as peer instruction.

Data management and analysisMost classroom response systems allowinstructors to export and save responsedata or uture analysis and assessment.Some systems also integrate with coursemanagement systems, like Blackboard.This integration allows instructors tosave and track student responses over thecourse o the semester, and simplifes theassessment process.

Instructors can projectresponse graphsoverhead or the classto see, so students

can compare their ownresponses to those o their classmates.


3/14



Instruction & Questioning

Response & Display

Data Management & Analysis

Instructor presents concepts and

materials, interspersed with slidesasking for feedback from students.Questions are typically in true or falseor multiple choice format.

Students key in their responses usingsmall remote transmitters. Thesetransmitters send signals to a receiver that is connected to the instructorslaptop or lectern PC.

Many instructors then ask students todiscuss the responses in groups, andto reach an agreement about the bestresponse. This discussion can befollowed with a second cycle of questioning, response, and display.

Software on the instructors machineinstantly tabulates and graphsstudent responses, and these simplegraphs can be displayed on thefollowing presentation slide.

Most classroom response systemsallow instructors to export and saveresponse data for future analysis andevaluation. Some systems alsointegrate with course managementsystems, like Blackboard, to simplifythe assessment process.

Classroom Response Systems: Technical Components and Interactions


4/14



the relevant literatureincreasing improve-ment in terms o learning outcomes.

At the most basic level o implemen-tation, classroom response systems serveas means or the instructor to monitorthe classroom. The instructor uses CRSto take attendance, to ensure some level

o participation, and to increase the stu-dents level o attention during the lecture.The instructor might also ask very basicquestions about reading assignments asa means to veri y whether students com-pleted the reading. In this scenario, theinstructor uses CRS as a way to encour-age attendance and some basic level o attention and participation, but makesvery ew intentional changes to thesequence, delivery, or duration o lectur-ing on a given concept.

At the second level o implementa-

tion, the instructor uses CRS to gatherreal-time in ormation about student com-prehension o a given concept. From theresponses, the instructor is able to deter-mine whether she should spend moretime explaining an idea, or i the majorityo the class understands the idea, allow-ing her to move on to the next topic. Thestudents help set the pace o instructionwith clear indication o their comprehen-sion or con usion.

The third approach to teaching withclassroom response o ten involves a trans-ormation in the instructors teachingphilosophy and strategies. This approachinvolves interspersing the presentationo concepts with question and responsecycles, ollowed by periods o discussionwhere students de end their responses andtry to persuade classmates with their rea-soning. Discussions are typically wrappedup with another question and responsecycle where students can indicate theirnew response to the same question.

Monitoring the ClassroomThe motivation to use classroom responsesystems most commonly derives rom adesire to stimulate student engagement.Instructors in large lecture halls o tenstruggle against what Rand Guthrie andAnna Carlin call the sea o slouchingbodies and expressionless aces (2004).E orts to engage students with questionsyield ew volunteers, and instructors o ten

High-enrollment courses present chal-lenges to many basic principles andestablished best practices in teachingand learning. Instructors in large lecturecourses o ten ace di fculty drawingout prior knowledge or misconceptions,motivating students and maintaining

their attention, creating opportunitiesor meaning ul engagement, assessingstudent comprehension, and developingclassroom activities that allow or theapplication o key concepts to practicalproblems. Yet high-enrollment lecturesremain the norm or introductory coursesin many disciplines, and instructors havelong sought tools and teaching strategiesto help overcome these challenges.

For over 40 years, electronic classroomresponse systems have been investigatedas a potential bridge or the communica-

tion gap between lecturers and students.Early systems were hard-wired intoclassrooms, with an input device at eachseat in a lecture hall. These systems werecostly and di fcult to use, given the lacko graphical user inter ace and the com-plexity o so tware or data manipulationand display.

Improvements to hardware and so t-ware solutions and vast changes to thetechnology landscape have created aresurgent interest in CRS in the pastdecade. New systems are much morea ordable, o ten portable, and takeadvantage other technologies already inuse in most large lecture halls (i.e., pre-sentation so tware, lectern hardware, andprojection systems).

There are essentially three levels o implementation o classroom response sys-tems, each with progressively more changein pedagogical approach, andas best wecan determine through an examination o

Most strategiesor using CRSall into one o three generalcategories o

implementation:

Monitoring the classroomAttendance,attention,completiono assignedreadings

Audience-pacedinstructionReal-timeevaluationo studentcomprehension

Peer instructionQuestion/response cyclecombined withdiscussion anddebate amongstudents C

a s e

S t u d

i e s &

A s s e s s m e n t s

Lecture courses canpresent challengesto basic principlesand established bestpractices in teachingand learning.


5/14



cannot determine where their lecturessucceed or ail until examination time.

Even at the most basic level o imple-mentation, where the instructor makesvery ew intentional changes to thelecture strategy, it seems that the use o classroom response systems can contrib-

ute to an instructors participation andattendance goals.Certainly, attendance policies can be

much harder to en orce in high-enrollmentcourses than in smaller courses. In anintroductory Earth Science course at PennState University, the instructor used CRSresponses over the course o the semesteras 15% o the fnal course grade (Greerand Heaney, 2004). Whether responseswere correct or not was not a actor, onlywhether students participated. Mean atten-dance ranged rom 81% to 84% over the

our CRS semesters measured, comparedto an estimated 50% by the midpoint o semesters without CRS. Head counts werealso conducted to account or the pos-sibility that absent students had handedtheir remote control units to riendswho entered responses on their behal .Discrepancies between CRS attendancenumbers and the results o head countswere typically +/- 2% (p. 348).

Although the comparison numbersare estimates, not directly measured in acontrolled setting, many other CRS stud-ies report similar fndings. Judson andSawada conducted an extensive literaturereview on response systems, and reportthat research rom the 1960s through thelate 1990s ound that the use o electronicresponse systems made students morelikely to attend class (2002, p. 177).

Classroom response systems have beenevaluated in several studies conducted inthe context o continuing medical educa-tion (CME) lectures and seminars. Surveyresults rom these and other studies indi-cate that student satis action increasesin lectures using CRS (or ARS, audienceresponse systems). Further, studentsreport believing that the use o CRS has apositive impact on their per ormance.

Miller et. al. (2003) conducted arandomized controlled trial o audienceresponse systems at 42 clinical round tableprograms across the country, and surveyednearly 300 participants about their experi-ence in ARS and non-ARS lectures.

Results showed that participantswho attended ARS lectures rated thequality o the talk, the speaker, and theirlevel o attention signifcantly higher thanthe non-ARS group (p. 112). Whileanalysis did show that the di erencesbetween the two groups were statistically

valid, the mean di erences did not divergeto the degree that might be expected.Presentation quality was rated at a meano 3.9 on a scale o 5 or non-ARS, and4.0 or non-ARS. Speaker quality wasrated 3.9 or non-ARS and 4.1 or ARS,while ability o presentation to maintainattention and interest was rated 4.2 ornon-ARS, and 4.4 or ARS (p. 113).

The survey instrument in Millersstudy also included a hand ul o questionsassessing participants understanding o the material presented, and results showedno signifcant di erence in learning out-comes in ARS and non-ARS classes. Theresearchers theorize that the types o ol-low-up questions posed might not addressdi erences in long-term retention andapplication o knowledge (p. 113).

In a separate study o CME lecturescovering multiple years o classes o eredwith and without audience response sys-tems, Copeland et. al. ound that lecturesin which the ARS was used were signif-cantly better rated than those in which theARS was not used (1998, p. 231). Overthree years, mean ratings or lectures withARS were about 3.47 on a scale o 1 to4, compared to non-ARS lectures rated as3.32. Even more telling, however, was acomparison o ratings rom multiple lec-tures rom a single speaker, two deliveredwith audience response, and one without.In the lectures delivered without audience

C a s e

S t u

d i e s &


c o n

t i n u e

d

As we gaze out at the

sea o slouching bodiesand expressionlessaces, it is hard to resistwondering i studentswant less education andmore entertainment.Rand W. Guthrie and Anna Carlin


6/14



response, the speaker received an averagerating o 3.09 on a scale o 1 to 4. In a lec-ture delivered with audience response, thesame speaker was rated at 3.74 (p. 232).

Another orm o classroom monitor-ing with CRS is to present short quizzesat the beginning or end o a lecture period.

Quizzes might cover homework or readingassignments, or basic concepts rom thematerial covered in the previous or currentlecture. In the Fall o 2004, Richard Halland others at the University o Missouri,Rolla, conducted a pilot evaluation o classroom response systems in a GeneralChemistry course (2005). They openedeach lecture with a brie quiz about theassigned readings, and ound that thequizzes served as a power ul motivatornot just or attendance, but class prepara-tion as well (p. 5). Students reported that

the quizzes helped them learn what thepro essor was wanting us to get out o thereading, and that you can see the areasyou need to go back and look at whenyou get questions wrong (p. 5).

In the same study, student responses toopen-ended survey questions verifed thatinterspersing lectures with CRS question/ response cycles acilitates some level o increased engagement and metacognition.Students indicated that CRS helped youpay attention in class because you knewyou had a question coming, and that

its a good way to know i you under-stand the material (p. 5).

Evidence rom these and other studiesindicates that the use o CRS in the class-room, even at a basic level, can increaseattendance rates, bring problem areas tothe sur ace, and increase student satis ac-tion with lectures.

Audience-Paced Instruction

Once instructors can see plainly whatstudents do and do not understand, theintuitive next step is to adjust the paceo presentation and explanation strate-gies accordingly. The second level o CRSimplementation is a very natural exten-sion o the frst.

The previously discussed CRS pilot atthe University o Missouri was in manyways a combination o all three levelso using CRS. In addition to quizzes onreading assignments and question cycles

throughout lecture (classroom monitor-ing), the lectures were modifed basedon student understanding as representedby the accuracy o their responses (Hallet. al., 2005, p.3). This modifcation isthe essence o audience-paced instruction.Students were also o ten allowed to dis-

cuss the questions with classmates be oreresponding during the lecture. It is di fcultto characterize this study as distinctly onetype o CRS implementation. However,because the main ocus o the study is onusing CRS to increase engagement andassess comprehension in order to custom-ize lecture presentation, we present anddiscuss their results primarily as class-room monitoring and audience-pacedinstruction.

Upon completion o the CRS pilotcourse in Halls study, students were sur-veyed regarding their perceptions o theuse ulness o CRS. Most students agreedthat CRS made class lectures more engag-ing (87%), and enhanced their learning inclass lectures (73%). A smaller majorityalso agreed that CRS made the lecturesmore motivational (63%). Studentswere divided as to whether CRS madeclass more challenging, and most dis-agreed (63%) that CRS helped thembetter understand how course materialrelated to real world problems (p. 4).

Halls research group compared thegrade distribution or the semester withclassroom response to a previous semesterwithout CRS. While they acknowledgea lack o specifc control measures toassure consistent grading standards andto account or student ability acrosssemesters, they report that grades were

C a s e

S t u

d i e s &


c o n

t i n u e

d

Once instructorssee plainly whatstudents do and donot understand, thenext step is to adjustthe explanations andpace o presentationaccordingly.


7/14



substantially better in semesters withCRS (p. 5). The percentage o studentsearning As increased rom 23% to 40%,and the percentage o students receivingCs or Ds in the course decreased rom34% to 21% (p. 4).

The Penn State University study

discussed in the classroom monitoringsection also employed audience-pacedeedback methods to encourage activestudent participation and allow bothstudents and instructors to gauge studentcomprehension instantaneously (p.345).

Multiple instructors involved in thisstudy, teaching various sections o anintroductory science course. Althoughinstructors shared the same course syl-labus, lecture outline, course structure,and grading scheme throughout eachsemester, they also developed their own

questions and maintained their ownclassroom style during the semester(p.347). From the description o their CRSimplementation, it seems again that theirapproach can not be described exclusivelyas classroom monitoring, audience-pacedeedback, or peer instruction. However, ata minimum instructors used CRS or real-time assessment o student understanding,and to directly address misconceptionsthat were revealed through questioning.

Students were asked at the midpointo the semester to participate in a brie anonymous survey to share their impres-sions on the e ectiveness o CRS. Amajority o students (65-77%) agreedthat the use o CRS helped them gaugetheir understanding o course material,and 71-85% agreed that it rein orcedimportant concepts presented in the lec-ture. Between 65% and 81% o studentssurveyed believed that the use o CRS inlecture helped them learn (p. 348349).

Qualitative eedback o ered in courseevaluations emphasize that students appre-ciate the anonymity o CRS (it gave shykids the chance to participate, I donteel put on the spot); that CRS encour-ages attendance (gave you an incentive togo to class, orces me to come to class);

and that CRS acilitates more engagementthroughout the lecture (helped everyoneget involved in such a large class, letsme see i I am understanding the lectureor not and it truly does give a nice breakrom straight lecturing). Negativecomments included rustration with atten-dance monitoring (extremely expensiveway to take attendance, dislike the waythe teacher uses the system); cost (oneo the most expensive classes I have evertaken, overrated and expensive); andspeculation that question and response

cycles are not the most e ective use o class time (takes up class time when sheshould be lecturing) (p. 349).

Although students can be resistantto any additional expenses associatedwith classroom response systems, manyinstructors equate the cost o the trans-mitter to the purchase o a textbook.Transmitters range in price rom about$20 or basic in rared systems to $125 orthe higher-end radio requency systems.Transmitters can also be sold at the endo a semester to help o set the cost. Formore in ormation on di erences in avail-able systems, see the Appendix.

Researchers at Eindhoven Universitycompared survey and per ormance dataor 2,500 students in course sectionsdelivered with audience-paced instructionto equivalent data rom 2,800 studentsin sections delivered in the traditionallecture ormat (Poulis et. al., 1998). Thisstudy showed that the mean pass rate oraudience-paced instruction lectures issignifcantly higher than where traditionalmethods have been employed (p. 441).The mean pass rate or traditional sec-tions was less than 60%, while or CRSsections it was over 80%. The research-ers also note that the standard deviationwas substantially lower in the CRS group,indicative o a more consistent level o comprehension throughout any givenclass (p. 441).

Certainly, this type o comparative per-ormance data, gathered in a controlled

C a s e

S t u

d i e s &


c o n

t i n u e

d

The mean pass rateor traditional lecturesections was lessthan 60%, whileor audience-pacedinstruction sections,it was over 80%.


8/14



experimental environment, is a moreconvincing measure than students sel -reported behavior and perceptions o thee ectiveness o CRS or audience-pacedinstruction. What students believe andreport to believe about their learning andbehavior can di er signifcantly rom what

they actually do and learn. Taken together,the fndings rom the studies cited shedsome light on the potential impact o usingCRS to gauge student comprehension inorder to tailor the lecture delivery.

Peer Instruction

In this approach to teaching with CRS,the lecture process shi ts rom the bal-listic model o knowledge trans er (planand launch a lecture at the students, checklater to see i you hit the target) to a more

constructivist model, with the studentactively building knowledge as a resulto meaning ul classroom interactions andactivities.

Peer instruction (PI) was pioneeredand has been evaluated extensively byEric Mazur and others in the Departmento Physics at Harvard University. Mazurand his colleague, Catherine Crouch,defne peer instruction as the modifca-tion o the traditional lecture ormatto include questions designed to engagestudents and uncover di fculties with thematerial (2001, p. 970). They continue:

A class taught with PI is divided into aseries o short presentations, each ocusedon a central point and ollowed by arelated conceptual question.... Studentsare given one or two minutes to ormu-late individual answers and report theiranswers to the instructor. Students thendiscuss their answers with others sit-ting around them; the instructor urgesstudents to try to convince each othero the correctness o their own answerby explaining the underlying reasoning.During the discussion, which typicallylasts two to our minutes, the instructormoves around the room listening. Finally,the instructor calls an end to the discus-sion, polls students or their answersagain (which may have changed based onthe discussion), explains the answer, andmoves on to the next topic (p. 970).

These discussion periods help studentsunderstand the key concepts behind their

answers, and acilitate a deeper, morepractical comprehension than what mightresult rom a traditional lecture. Whileelectronic response systems are not essen-tial to peer instruction, they certainlyacilitate the process more e fciently andcapture data more e ectively than other

methods o gathering eedback (pollingby use o colored ash cards or show o hands).

There are many articles documentingthe e ectiveness o peer instruction (alsodescribed as interactive engagement) invarious settings. In one such assessment,Mazur and Crouch report on ten years o fndings rom physics courses at Harvard(2001). They analyze learning outcomesin terms o conceptual mastery, using testresults rom the Force Concept Inventory(FCI), and quantitative problem solving,

using data rom the Mechanics BaselineTest (MBT). (The FCI and MBT are stan-dard assessments o student per ormancein physics. They are administered nation-ally as pre- and post-tests to evaluate andcompare student learning in physics.)

The average normalized gain rom pre-and post-test FCI doubled in the frst yearo implementing peer instruction in a cal-culus-based physics course. Furthermore,as the instructors refned their use o peerinstruction and the choice o discussionquestions (called ConcepTests), FCIresults continued to improve (p. 971).

In 1990, the last year o traditionalinstruction, the FCI normalized gain wasapproximately 0.23. In 1991, the gainwas 0.49, and that number increasedevery year until 1997, when it hadreached 0.74. In 1998, they switched toalgebra-based introductory physics, butFCI gains remained high at 0.65. In 1999,the course reverted to the traditional lec-

C a s e

S t u

d i e s &


c o n

t i n u e

d

The average normalizedgain doubled in the frstyear o implementingpeer instruction, andcontinued to groweach year as teachingmethods were refned.


9/14



ture ormat, and gains dropped again to0.40. The ollowing year, peer instructionwas implemented again and the gain roseto 0.63 (p. 971).

In the peer instruction courses, tradi-

tional problem solving is de-emphasizedand students are required to learn andpractice these skills primarily throughdiscussion sections and homework assign-ments (p. 971). Nonetheless, students inPI classes ared better than traditionallytaught cohorts on the MBT, a widelyaccepted assessment o physics problem-solving skills. The average score on theMBT increased rom 66% in 1990 withtraditional instruction to 72% in 1991with the introduction o PI (p. 971).As with the FCI, student per ormancecontinued to rise as instructors refnedthe approach, reaching 79% in 1997. Inanother comparative problem-solvingexamination, the mean score increasedrom 63% with traditional instructionto 69% with peer instruction, and therewere also ewer extremely low scoresin the peer instruction group (p. 971).

In an analysis o all ConcepTestresponses over the course o a semester,Crouch and Mazur ound that, on average,40% o students answered correctly bothbe ore and a ter peer discussion. Some32% o students answered incorrectlyat frst but correctly a ter the discussion,while 22% o students answered incor-rectly twice. Only 6% answered correctlyfrst then changed to the incorrect answera ter discussion. They also ound that

no student gave the correct answer tothe ConcepTests prior to discussion morethan 80% o the time, indicating that

even the strongest students are challengedby the ConcepTests (p. 973).

The outcomes in the Department o Physics at Harvard are certainly convincingin terms o improved learning outcomeswith peer instruction, and several otherlarge-scale studies serve to corroborate

these fndings. In a variant approach topeer instruction, pro essors at Iowa StateUniversity attempt to achieve virtuallycontinuous instructor-student interactionthrough a ully interactive physics lecture(Meltzer and Manivannan, 2002, p. 639).Again, the use o CRS is not critical to theteaching approach, but serves to supportand acilitate the methods employed.

Comparing results rom studentsin the ully interactive ISU courses tonational results on the Conceptual Surveyin Electricity and Magnetism shows even

greater leaps in learning than those oundat Harvard. The normalized pre- andpost-test gains or interactive lectures was

triple those ound in the national survey,which presumably consists o a sample o students primarily in traditional lectureclasses. Normalized gains or ISU interac-tive courses were 0.68, compared to 0.22in the national sample (p. 648).

And fnally, Richard Hake conductedan analysis o 6,000 students resultson two national physics assessments,the Force Concept Inventory and theMechanics Baseline Test (1998). By gath-ering and analyzing additional data onthe type o instruction o ered in thesestudents courses, Hake ound that the14 traditionally taught courses had anaverage normalized gain o 0.230.04.By contrast, the 48 courses that usedinteractive engagement methods likethose outlined in the previous two stud-ies showed an average normalized gain o 0.480.14 (p. 71).

These studies confrm that teachingmethods that encourage active and mean-ing ul participation and engagement onthe part o the student can radically trans-orm the nature and scope o learning thattakes place. While CRS was not a centralcomponent o any o these evaluations,many instructors have ound that theuse o CRS can simpli y and streamlinein ormation gathering and display in theinteractive lecture hall.

C a s e

S t u

d i e s &


c o n

t i n u e

d

No student gave the

correct answer to theConcepTests prior to discussion morethan 80% o the time,indicating that eventhe strongest studentsare challenged...


10/14



Classroomresponsesystems canbridge thecommunication

gap betweeninstructor andstudents, andacilitate high-engagementlectures. C o

n c l u s

i o n This report is intended to serve as anoverview o classroom response systems,

and a summary o fndings rom ormalevaluations o various implementationscenarios.

These studies show that classroomresponse systems can acilitate the pro-cess o drawing out prior knowledge,maintaining student attention, andcreating opportunities or meaning ulengagement. They can also assist instruc-tors in assessing student comprehensionand developing classroom activities thatallow or the application o key conceptsto practical problems. As with other edu-cational technologies, the most success ul

Re erencesCopeland HL, Stoller JK, Hewson MG, Longworth DL (1998) Making the continuing medicaleducation lecture e ective. Journal of Continuing Medical Education in the Health Professions ,Vol. 18, 227-234.

Crouch CH, Mazur E (2001) Peer Instruction: Ten years o experience and results. American Journal of Physics , Vol. 69 No. 9, 970977. DOI= http://dx.doi.org/10.1119/1.1374249 [last access: 11.23.2007]

Greer L, Heaney PJ (2004) Real-Time Analysis o Student Comprehension: An Assessment o Electronic Student Response Technology in an Introductory Earth Science Course. Journal of Geoscience Education , Vol. 52 No. 4, 345351.http://www.nagt.org/fles/nagt/jge/abstracts/Greer_v52n4.pd [last access: 11.23.2007]

Guthrie R, Carlin A (2004) Waking the Dead: Using interactive technology to engage passivelisteners in the classroom. Proceedings of the Tenth Americas Conference on Information Systems ,New York NY, 18. http://www.mhhe.com/cps/docs/CPSWP_WakindDead082003.pd [last access: 11.23.2007]

Hall RH, Collier HL, Thomas ML, Hilgers MG (2005) A Student Response System or IncreasingEngagement, Motivation, and Learning in High Enrollment Lectures. Proceedings of the AmericasConference on Information Systems , 621626.http://lite.umr.edu/documents/hall_et_al_srs_amcis_proceedings.pd [last access: 11.23.2007]

Hake R (1998) Interactive-engagement versus traditional methods: A six-thousand-student surveyo mechanics test data or introductory physics courses. American Journal of Physics , Vol. 66 No. 1,6474. DOI= http://dx.doi.org/10.1119/1.18809 [last access: 11.23.2007]

Audience-paced andpeer instruction canhave a greater impact onlearning outcomes, butalso require a greater degree o deviationrom the traditionallecture approach.

implementations occur when instructorsset clear educational goals, and acilitatethe achievement o those goals throughthought ul, engaging learning activities.

Each o the three levels o implemen-tation discussedclassroom monitoring,audience-paced instruction, and peer

instructioncan have a positive impacton the learning experience and edu-cational outcomes when thought ullydeployed. Audience-paced and peerinstruction show the most potential orimpact in terms o learning outcomes, butalso require a greater degree o deviationrom the traditional lecture approach.

Support

I you are an instructor at CarnegieMellon and are interested in discussing

the use o classroom response systems inyour class, please contact the:

O fce o Technology or [email protected]

Our consultants will be happy to assist youwith any phase o planning, designing,implementing, unding, and evaluatingthe use o technology tools and strategiesor teaching.


11/14



Re erences (continued) Judson E, Sawada D (2002) Learning rom past and present: Electronic response systems in collegelecture halls. Journal of Computers in Mathematics and Science Teaching , Vol. 21 No. 2, 167181.

Meltzer DE, Manivannan K (2002) Trans orming the lecture-hall environment: The ully interactivephysics lecture. American Journal of Physics , Vol. 70 No. 6, 639654.DOI= http://dx.doi.org/10.1119/1.1463739 [last access: 11.23.2007]

Miller RG, Ashar BH, Getz KJ (2003) Evaluation o an audience response system or the continuingeducation o health pro essionals. Journal of Continuing Education in the Health Professions ,Vol. 23, 109115. http://www.cbil.vcu.edu/Resources/ARSMiller.pd [last access: 11.23.2007]

Poulis J, Massen C, Robens E, Gilbert M (1997) Physics lecturing with audience paced eedback.American Journal of Physics , Vol. 66 No. 5, 439441. DOI= http://dx.doi.org/10.1119/1.18883 [last access: 11.23.2007]

C o n c l u s i o n

c o n

t i n u e

d

A b o u t t h e

S e r i e s The purpose o the Teaching With Technology White Paper series is to provide Carnegie Mellon

aculty and sta access to high-quality, research-based in ormation with regard to a given classroomtechnology. These papers o er a general overview o the technology topic, summarize fndings romavailable assessments and evaluations, and give direction toward urther reading and online resources.

This series does not introduce original research fndings rom technology assessments or evaluations

conducted at the O fce o Technology or Education and/or Carnegie Mellon University. The papersserve as literature reviews, intended to provide scholarly integration and synthesis o the most soundand comprehensive studies documented at the time o publication.

This work is licensed under:Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.To view a copy o this license, visit: http://creativecommons.org/licenses/by-nc-nd/3.0/us/write to: Creative Commons, 171 Second Street, Suite 300, San Francisco, Cali ornia, 94105, USA.


12/14



A v a i

l a b l e P r o d u c t s

TurningPoint , rom Turning Technologies, is one o the most versatilesolutions on the market. Their so tware integrates completelyinto Microso t PowerPoint, allowing pro essors to author, deliver,assess, and report rom within PowerPoint. TurningPointo ers in rared and radio requency versions o their student

transmitters, and simple USB-based receivers. Virtual Keypad(or vPad) is available or wireless classrooms as a so twarealternative to transmitters and receivers.

InterWrite also o ers both in rared and radiorequency transmitters. Unlike most transmitters onthe market, both the IR and RF transmitters o er highand low confdence level keys, allowing students toindicate how confdent they are in their answers. The RF

transmitter screen can display 16 charactersmore texton screen at once than most other systems. This allows ora sel -paced testing unctionality, where students can scrollthrough a series o questions and answerat their own pace.

A p p e n

d i x O n e

The most predominant sales model or classroom response systems is or a singlecompany to provide all necessary hardware and so tware or the system. (The mostcommon exception is when the product is a so tware-only solution, designed or useon existing wi-f or wired networks. Such solutions are discussed in more detail inthe next section.)

In many cases, so tware and receivers are provided at little or no cost, with mostproft resulting rom the sales o student transmitters. Many universities choose topass along the ull or partial cost o transmitters to the students, especially wheresystems are widely adopted and students can expect to use a single transmitter inmultiple classes.

eInstruction o ers a basic, inexpensive classroom response system.Although the so tware was designed to be used easily with presentationso tware like PowerPoint, there is currently no direct integrationbetween these applications. Similarly, while most systems integrateairly directly with Blackboard, fle manipulation is required to useeInstruction PRS data in Blackboards gradebooks. The transmittersare airly inexpensive to purchase, but require the purchase o a newactivation code each semester. Activation currently costs $12 to $18.


13/14



H-ITT boasts the lowest system cost on the market. It o ersonly in rared transmitters, but its receivers have a wider pick-uprange than most other in rared systems (a 180-degree cone,

compared to 90 degrees on other systems). H-ITT comes withtwo separate so tware packages: one is used in class to collectand view responses, and the other is used to manage and graderesponses over the course o the semester. Each remote costsabout $25, and there are no activation ees. The receiver costs about$200, and one receiver must be purchased or every 50 remotes. A

v a i l a b l e

P r o

d u c t s

A p p e n

d i x O n e ,

c o n

t i n u e

d

Qwizdom places heavy emphasis on the physical design o theirtransmitters. The large text display on both instructor andstudent remotes allows or one-to-one communication, a

eature not supported on any other system reviewed here.Receivers and so tware are ree, but the remotes costabout $100 apiece. The price is high compared to othertransmitters, but they are intended to be purchased onceand used or the duration o a students college career.

TI-Navigator , rom Texas Instruments, uses radiorequency hubs toconnect students calculators to the teachers PC. The systemseems to ocus most heavily on mathematics applicationsor obvious reasons, but it is unclear as to whether thecalculators can be used or more general questions andpolling. The system is airly costly, pricing at around $4000(not including calculators) or a 32-student classroom. Itsadvantages over other classroom response systems are specifcto math applications.

Many classroom response systems are being developed solely as so tware applicationsthat are designed to run over Wi-Fi on existing portable devices, likelaptops and Pocket PCs. Numina II is a browser-based applicationdeveloped at the University o North Carolina, Wilmington.Class in Hand is so tware or the Pocket PC that was developed atWake Forest University. Both o these applications are currentlyree to use. For those who are more com ortable with theaccountability and support o a retail product,ETS Discourse is the commercial equivalent.


14/14



S y s t e m

C o m p a r i s o n

A p p e n

d i x T w o

System Advantages DisadvantagesIn rared (IR) systems basically use thesame line-o -sight technology that isused in household television remotes.

They have the lowest overallequipment cost.

There are no inter erence issues romclassroom to classroom, as signals donot go beyond the walls o the room.

Because the clickers must be aimeddirectly at the receivers in order to

work (and thus have high visibility inthe classroom), they also reduce thelikelihood that students will bringin each others transmitters whenresponses are used or at tendanceor participation grades.

In radio requency (RF) systems, thereceiver does not have to be placedin line-o -sight o students, allowingor increased portability in hardware

solutions.

Signal reception is more reliable and hasa longer range.

RF systems also allow or two-waycommunication, so clickers canconfrm when students responsehas been received.

Wi-Fi systems use a web-based inter aceor student interaction. These systemsallow or text entry and open-endedresponses.

Students can use a wide variety o Wi-Fi devices.

Uses the existing campus wirelessin rastructure.

Most IR systems o ten o er only one-waycommunication, which does not allow or confrmation when students responsehas been received.

They also require the placement o receivers in line-o -sight o students,which o ten means permanent or semi-permanent installation.

Each receiver can only support between40 and 80 transmitters (depending onmanu acturer), so multiple receivers arenecessary or larger classes.

In very large classes, signal reception canbe unreliable and have a shorter range.

Clicker administration and managementcan also be expensive.

Low visibility might make it easier or students to cheat the system by bringingin each others transmitters whenresponses are used or at tendance or participation grades.

RF clickers are more expensive than IR.

There is a higher likelihood o inter erenceissues as RF clickers can operate on thesame requencies as Wi-Fi and other RF devices.

Clicker administration and managementcan be expensive.

Requires students to have a Wi-Ficomputing device.

Fewer choices currently available inthe marketplace.

In rared

Radio Frequency

Wi-Fi

In rared, radio requency, and Wi-Fi systems each have advantages and disadvantages,and deciding which system is best depends heavily upon the needs and prioritiesor a given context.

Adapted rom the University o Minnesota O fce o Classroom Managements Student Response Systems Overviewhttp://www.classroom.umn.edu/notes/support_srs.asp

Documents

A Teaching with Technology White Paper - Classroom Response Systems