Aapt peer wise_bates

Creatively engaged online: student-generated content in anon-majors introductory course

Emily Altiere, Simon Bates, Firas Moosvi Department of Physics and AstronomyUniversity of British Columbia

AAPT Summer Meeting, July 2013, Portland OR

Monday, 15 July, 13

Outline

• Design and Methodology

• Engagement

• Question/Explanation Quality

Monday, 15 July, 13

a"web&based"MCQ"repository"created"by"students"Monday, 15 July, 13

Ins$tu$ons((signing(up(per(year:(!

2009:! ! !22(2010:! ! !66(2011:! ! !204(2012:! ! !266(2013!(Jan,Jun):! !214(

Growing(content(repository:(!Courses:! !2,500(Logins/month:! !75,000(Ques<ons:! !600,000(Answers:! !12,000,000(

Monday, 15 July, 13

Student'ownership'over'learning'resource'

Student'familiarity'with'social'so7ware'

Leveraging'student'energy'and'crea9vity'

Monday, 15 July, 13

Design and Methodology

Minimum participation requirements for each of two assessment exercises (PW1, PW2)

Write 1 Answer 5 Rate / comment 3

5% course credit

Physics 101, Energy & WavesWinter Semester: 3 sections, 791 students

Monday, 15 July, 13

Design and MethodologyTOCCLT'.' -

Qest\I\sct

How 1o...

q/t4osTer

Submit ond onswer questionson topics in lhe torget region,just obove lhe physicsyou hove olreody mostered.

Dislroclers

) l*"*"ò

rNrrltngIh s region corrk:ins llte

plrysics knowledgo ond colrco¡lsyou connol leorn yel

becouse the louncalionscrrc nol in plocc

PHYSICS TOPICS IN YOUR TARG T REGION

buo'5ot't1 'Den¡¡lu

B¿:o\v\vì 1 JF:rcq - YOUR CHOSEN TOPIC

à..\auC'q^ x@bôo'osro

pn ro, d logroo, \o 9tS

COMMON MISCONCEPÎIONS AND ERRORS(Sec hHÞ:/jÞhy¡.udofo¡.cdu/CJP/trêconcoÞllo¡t.pdt

tor o llil ol common mhconcepllons)

¿K N(,qht %cce Açxs no\ exi¡þ, c.rrrçr\J On oblec'-t \5 õÞYJrnr cìü?ÀI in rtuiò. ü

lrh'brot4ont eprç,e .I¡1enÀs crqden-s,\{^ c>Ç c.,þ¡ec\ , cìo+ .\*¿n:r

oÇ Çtu,8,fr- btrrqon! $crCe c¿c\s Àoc¡:'cr

fr¿lt{e- r *hcrn ú?

Trr¿e. $ììfifflsz hrrr: ¿lÕoolr-g lnnS

(**= tcpo þ/rnl {otr-* qrÇ bq.\\:. 0" Ooz

Physics knowledgeond conccpluol underslonding

you lrove olrcodyconslrucleC ¡n your heod

¡,<'t.lerJ rt)\^,¿z! Þ.rrp \r- \rr't\¡ ,l?

Chcck lhol yovr on3Íêr ¡t rêo!ànoblê ðnd potrlble

\'f<,,cc\

Photo by Seth Casteelhttp://www.littlefriendsphoto.comPermission to use agreed

Introduced in tutorials

Extensive scaffolding exercises

Revisited in subsequent tutorials

Tutorials delivered by 24 TAs

Monday, 15 July, 13

Engagement with PeerWise

Monday, 15 July, 13

Engagement with PeerWise

PW1 PW2

Contributed and met minimum requirementsContributed but did not meet mininum requirementsDid not participate

0.84 0.80

Total: 791

Monday, 15 July, 13

Engagement with PeerWise

Number Multiplier Number MultiplierQuestions 1105 [1.7] (1.4) 998 [1.6] (1.3)

Answers 11393 [17.2] (14.4) 11807 [18.7] (15.0)

Comments

4901 [7.4] (6.2) 5509 [8.7] (5.0)

PW 1PW 1 PW 2PW 2

Monday, 15 July, 13

Question/Explanation Quality

Monday, 15 July, 13

Question/Explanation Quality

Bloom’s Taxonomy of levels in the cognitive domain

Score Level Description

1 Remember Factual knowledge, trivial plugging in of numbers

2 Understand Basic understanding of content

3 Apply Implement, calculate / determine. Typically one-stage problem

4 Analyze Typical multi-step problem; requires identification of strategy

Evaluate Compare &assess various option possibilities; often conceptual

Synthesize Ideas and topics from disparate course sections combined. Significantly challenging problem.

Monday, 15 July, 13

1 2 3 4 1 2 3 40

20

40

60

Num

ber o

f que

stio

ns

Assessment 1 Assessment 2

Bloom's Taxonomy: Question Quality

Textp>0.05, NS

Monday, 15 July, 13

Question/Explanation Quality

Description of explanation quality

Score Level Description

0 Missing No explanation provided or explanation incoherent/irrelevant

1 Inadequate Wrong reasoning and/or answer; trivial or flippant

2 MinimalCorrect answer but with insufficient explanation/justification/ Some aspects may be unclear/incorrect/confused.

3 Good Clear and detailed exposition of correct method & answer.

4 ExcellentThorough description of relevant physics and solution strategy. Plausibility of all answers considered. Beyond normal expectation for a correct solution

Monday, 15 July, 13

0 1 2 3 4 0 1 2 3 40

20

40

60

Num

ber o

f que

stio

ns

Assessment 1 Assessment 2

Explanation Quality

p=0.02

Monday, 15 July, 13

Conclusions

Participation is generally well beyond the minimum requirements for course credit.

Unlike similar studies with Physics majors*, non-majors in this study produced questions of lower overall quality.

Evidence to suggest that question quality and detail of explanation improves with practice.

*Bates, et al ‘Assessing the quality of a student-generated question repository’ PRST-PER submitted

Monday, 15 July, 13

References

Student-facing system http://peerwise.cs.auckland.ac.nz/

All the research studies referenced and scaffolding materials referred to are accessible through the PeerWise community

site http://www.peerwise-community.org/

Monday, 15 July, 13