Learning from student experiences for online assessment tasks ?· Learning from student experiences…

  • Published on
    12-Sep-2018

  • View
    212

  • Download
    0

Embed Size (px)

Transcript

  • Learningfromstudentexperiencesforonlineassessmenttasks

    M.AsimQayyumandDavidSmithCharlesSturtUniversity,WaggaWagga,Australia

    Abstract

    Introduction.UseoftheInternetforopenWebsearchesiscommonamonguniversitystudentsinacademiclearningtasks.Thetoolsusedbystudentstofindrelevantinformationforonlineassessmenttaskswereinvestigatedandtheirinformationseekingbehaviourwasdocumentedtoexploretheimpactonassessmentdesign.Method.Amixedmethodsapproachwasfollowed,usingrecordedobservationsofonlinesessionsandretrospectiveinterviews.Tenstudentswhorecentlystarteduniversityundertooktwoonlineassessmenttaskswhereallonlineinteractionsweredigitallyrecorded,followedbyretrospectiveinterviewstogaininsightintotheirbehaviour.Analysis.Theobservationswereanalysedtogetherwiththeinterviewresponsestodevelopcategoriesandthemesofinformationtoolsusage,withaparticularfocusonassessmentdesigninhighereducation.Results.Mostparticipantspreferredtoseekinformationusingbasictoolsandapproaches,anddidnotmaximisethecapacityofinformationtoolstoimprovequalityandsynthesisofdiscoveredinformation.Thesenovicereadersshowedaneedforinstructioninspeedreadingandinformationtoolusagetoimprovetheirlearning.Conclusions.Findingssuggestthatimprovementinassessmentdesignandsubsequentstudentlearningmaybeachievedbyprovidingguidancetouniversitystudentsininformationtoolsusageduringthedesignoflearningtasks.

    CHANGEFONT

    Introduction

    SearchingandfindinginformationisahumantraitthathasbeentransformedwiththeadventofWebbasedtechnologies.TheuseofcommonsearchenginessuchasYahoo,GoogleandBinghasincreasedexponentiallytheamountofdatathatindividualscanhaveaccessto(Jansen,Liu,Weaver,CampbellandGregg,2011)andthesharingofinformationthroughplatformssuchasWikipedia,aswellassocialmedianetworks,effectivelycreatesknowledgesharingcommunities.However,withsomuchdataavailableitraisesquestionsonhowtoeffectivelysearchforinformationandthen,justasimportantly,howmuchofthatinformationiseffectivelyabsorbedandretainedbythisnewgenerationofstudentsenteringthehighereducationsystemtoday.Toaddressthesequestions,thisstudywasconductedtoexaminetheavailabilityandselectionofinformationtoolsandimpactoftheirusageonthedesignofonlineassessmentsintheacademicenvironmentsofstudents.Amixedmethodsapproachwasadoptedtoexploretheinformationandlearningbehaviourofstudentswhentheyengaged

    http://www.informationr.net/ir/20-2/paper674.html#authorhttp://www.informationr.net/ir/20-2/paper674.html#jan11

  • withonlinecourserelatedreadingmaterial.

    Theemphasisofthisstudyisoninvestigatingthetoolusagepatternsandinformationbehaviourofstudentsinthecontextoftheirlearningneedsinanonlineacademicenvironment.Thespecificresearchgoalwasthentoinvestigatedistinctpatternsofinformationbehaviourinonlineassessmenttasksbyinvestigatingtheuseandmanagementofinformationtools,aswellasthesubsequentmanipulationofinformationbystudents.TheonlineassessmenttasksusedinthisstudywereoftheWebbasedtasktypes,whicharenormallyembeddedwithindistancelearningeducationmodulestoquicklyteststudentlearningonaspecifictopic.

    Forthepurposeofthisproject,theterminformationtoolshasbeendefinedasanycomputerbasedapplicationthatstudentsmayusetofindinformationontheWeb,forexamplesearchengines,videodisseminationserviceslikeYouTube,librarycatalogues,socialmedia(includingblogs,wikis),andevenorganizationalWebportalsthatpackagerelevantinformationaccordingtotheneedsoftheirviewers.Emphasisisoncomputerapplicationsthatprovideuserswithseveralvariedsourcesoflinkedinformationtochoosefrom.Findingsfromthisstudyareexpectedtoenhanceourunderstandingofhowthestructureandscaffoldingofanonlineassessmentcanbeimprovedbyobservingselectionanduseofinformationtoolsbystudents,andtherelatedinformationseekingandbrowsingactivitiesthathappenwhensuchtoolsareused.

    Thisstudyissignificantbecauseabetterunderstandingofonlinebehaviourofnewuniversitystudentswillhelpeducatorsacrossarangeofdisciplinesplanandcreateimprovedonlinelearningassessmentsandactivities.Creatingonlinelearningexercisesfortheappropriateacademicstageofdegreestudyisimportantasitprovidesascaffoldforstudentswhosetertiarystudysettingsandrequirementsvary.Insuchcases,learningfromthelearner'sonlinebehaviourusingvideomonitoring,interviews,andeyetrackingdevicesisagreatwaytooptimiseanddesignmaterialsuitedtoastudent'sneeds.

    Thevalueandneedofthestudyrelatestopreferencesforonlineinformationformatsandcontentbystudents,andthustothequalityandeffectivenessofuser/informationinteractions.Giventhetrendsidentifiedintheliterature,onlineinformationtools,learningmaterialsandotherservicesareheretostay,makingthisinvestigationimportant.Priorresearchinthefieldsofeducationandinformationsciencecometogethertomakeacontributioninthisregardasindicatedinthedesignofthisstudy.

    Reviewoftheliterature

    Customisingtechnologyusetoenhancelearnerexperienceprovidesuserswithanaffordance(McLoughlinandLee,2007)tocapitaliseandbenefitfromtechnology.Currentapplicationsincloudcomputing,augmentedrealityand3Dprintingensurethatmultimodalinformationisavailableatanytime,experimentscanbeconductedthroughsimulationsanddesignscanbeproducedquicklyandcheaplythesearejustsomeexampleswheretechnologyismeetinglearningneedsbydesign.However,thewellpracticedandfamiliarabilitytouseInternetsearchenginesandsocialmediaplatformsarenotattheforefrontoftechnologydevelopmentwhenitcomestolearningneedsofstudents.Universitystudentsoftenstrugglewithvariousaspectsofinformationliteracyeventhoughtheiraccesstotechnologyhasimprovedandtheyappeartobeverycomfortableinitsusage(Rowlandsetal.,2008).Thusfindingtherightinformation,usingtherighttool,fromtheright(credible)sourceisstillproblematicformost

    http://www.informationr.net/ir/20-2/paper674.html#mcl07http://www.informationr.net/ir/20-2/paper674.html#row08

  • students.Moreover,radicaluseoftechnologiesinasophisticatedmannertomakeuseoftheincreasingsetofinformationplatformsisstillconsideredamythforeventhedigitallyknowledgeablestudents(Margaryan,LittlejohnandVojt,2011)becausestudentswilltrytoconformtoaconventionalpedagogywithminimaluseofonlythetechnologiesthatdelivercontent,suchas,forexample,GoogleorWikipedia.Onereasonstatedisthattheteachingstaff,whichdoesexperimentwithnewertechnologies,usuallyrevertsbacktoestablishedtoolsandmethodsandisreluctantintheuseofemergentsocialtechnologies.Thestudentsthensimplyfollowthetrendssetbytheirinstructors.

    Theneedforinformationispivotalandfoundationaltothelearningexperienceandthestudentsmustbeabletodevelopeffectivesearchstrategiestofind,refine,andabsorbthatinformation.Thesetrendsindicateanopportunitytolearnfromandcustomisetheonlinepracticesofstudentsthroughimprovementsinlearningmaterialstoincreasetheefficiencyofinformationretrieval.Garrison,AndersonandArcher(2000)suggestthatthereisalackofguidancegiventostudentswhenlecturersdesignonlineinformationseekingactivities,however,ifattentionwasgiventorefiningtechnology'scapacitytogenerateinformationthenitislikelythatstudentswouldbebetterabletoconstructandconveytheirunderstanding(Oliver,Harper,Wills,AgostinhoandHedberg,2007).

    ThereareafewstudiesthatexploretheuseofInternetresourcestechnologiesbystudentsingeneral(forexample(Kennedy,Judd,Churchward,GrayandKrause,2008Li,2012Rowlandsetal.,2008Thompson,2013)butnonethatspecificallyfocusontheinformativecomponentsofthesetechnologiesforstudentlearning.Moreover,theuseofthesetechnologiesandtoolsinonlineassessmenttaskshasnotbeenexploredpreviously.Forexample,Head(2007)lookedonlyattheuseofresourcesavailabletothirteenundergraduatestudentsenrolledinhumanitiesandsocialsciencesclassesforafourtosixpageresearchpaper,andconcludedthatmoststudentsusedlibraryorinstructorprovidedresourcesratherthanrelyingonInternetbasedtoolslikeGoogleandWikipedia.Li(2012)reportsfromastudy,includingelevenundergraduatestudentsoverthreesemestersinHongKong,thatasearchengineisoftenusedasastartingpointbeforeswitchingtouniversityprovidedresources.

    Someofthetechnologicaltoolsidentifiedfromtheliteraturethatarecommonlyusedbystudentsintheirlearningare:contentmadeavailablebyinstructors(inLearningManagementSystems)Google,GoogleScholarandGoogleBooksmobilephonesandmessaging(includingtextmessaging)WikipediaandotherWebsites(Head,2007Kennedyetal.,2008Li,2012Margaryanetal.,2011).Interestingly,Kennedy'sstudyindicatesmoststudentsstillwantedsometrainingintechnologyuseforuniversitylearningdespitethefactthattheywerereportedlycomfortableinitsusage.Thisfacthighlightstheimportanceofaninvestigationintotheuseofthesetoolsforinformationalandlearningdesignpurposes,whichhasnotbeenexploredmuchintheliteratureagapthiscurrentresearchprojectaimstofill.

    Thereadinghabitsofstudentsareexperiencingashiftingeneral,andtheonscreenreadingtrendsindicateashifttowardsnonlinearreadingandskimmingcontentratherthanfollowingthecontentveryclosely(Liu,2005).Thedebateonwhichformat(onlineversuspaper)isbetterwillundoubtedlycontinueforawhile,butothers(Coiro,2011Konnikova,2014)arguethatonlinedistractionsaffectareader'sfocusmorethantheenvironmentitself.Onlinereaderstendtodomoreonlinebrowsingandspendlesstimeonsustainedindepthreading(Liu,2005)andhencetakelongertimetocomprehendcontentgiventheperceivedtime

    http://www.informationr.net/ir/20-2/paper674.html#liy12http://www.informationr.net/ir/20-2/paper674.html#mar11http://www.informationr.net/ir/20-2/paper674.html#ken08http://www.informationr.net/ir/20-2/paper674.html#gar00http://www.informationr.net/ir/20-2/paper674.html#liy12http://www.informationr.net/ir/20-2/paper674.html#oli07http://www.informationr.net/ir/20-2/paper674.html#row08http://www.informationr.net/ir/20-2/paper674.html#liu05http://www.informationr.net/ir/20-2/paper674.html#mar11http://www.informationr.net/ir/20-2/paper674.html#hea07http://www.informationr.net/ir/20-2/paper674.html#liy12http://www.informationr.net/ir/20-2/paper674.html#liu05http://www.informationr.net/ir/20-2/paper674.html#kon14http://www.informationr.net/ir/20-2/paper674.html#ken08http://www.informationr.net/ir/20-2/paper674.html#coi11http://www.informationr.net/ir/20-2/paper674.html#tho13http://www.informationr.net/ir/20-2/paper674.html#hea07

  • pressures.TakeawaytheInternet,hencethedistractions,andreadersarelikelytoreadonscreenaswellastheydoonpaper,especiallyiftheyalsotakenotesalongtheway(Subrahmanyametal.,2013).Thedifferencesmaybepronouncedhoweveriflongertextsaretobereadonlineas,duetothevarieddistractions,readerscanspendlongertimesinfrontofthecomputerscreenswhilereadingarticles.AndtheimportanceoftimedevotiontoreadinghasbeenpointedoutbyCull(2011):

    Thepowerofreading,whetherofprintoronlinetext,continuestolieinthispoweroftimetimetodigestwords,timetoreadbetweenthelines,timetoreflectonideas,andtimetothinkbeyondone'sself,one'splace,andone'stimeinthepursuitofknowledge.

    Cullrecommendsthatitthenbecomestheresponsibilityofuniversityeducatorstomotivatethestudentstosetasidethetimerequiredforindepthreading.Theseskillstaketimetodevelopandcancontinuetodevelopinstudentsevenaftergraduationbutthemaincentralthemeisthatonlinereadinghabitsofstudentsneedtoberefinedearlieroninhighereducationinstitutes.Forexample,Head(2007)reportsthatmoststudentsinherstudyfeltalittlechallengedbyresearchassessmentsand11ofher13participantsfeltthattheinstructorsshouldhavegiventhemmoreinformation.Li(2012)reportssimilarfindingsandrecommendsthatlecturersneedtoaddresstheseshortcomingsduringclasstimes.Alsointhiscontext,Coiro(2011)presentsaplantodevelopthemindsetreadersneedtoadoptwhentheyapproachonlinetextstoimprovetheirnavigatingandnegotiatingskills,developpathways,andlearntorespondtoorreflectonthecontent.

    AchallengeinlearningfromInternetbasedmaterialisthesheeramountofinformationavailableonlinethatcanpotentiallyhinderdeepreading,evenifthereaderisnotdistractedbymiscellaneouselementsthatareeverpresentincyberspace.InacomprehensivestudyoftheWebbrowserlogsof25participants,Weinreich,Obendorf,HerderandMayer(2008),analysednearly60,000firstpagevisitstoconcludethat17%ofnewpageswerevisitedforlessthan4seconds,whilenearly50%ofthefirstpagevisitslastedlessthan12seconds.LongerWebpagestaydurationmeasurementsinthatstudycarriedanerrorfactorasthecountincludedopenedsitesthatpeopleforgottocloseastheymovedontoothertasks.Orinotherwords,theusersgenerallyglimpseorscanovertheinformationtopickupkeywordsratherthandoinganyactualreading.MostuserstopsonGooglesearchresultswereevenshorter(rangingfrom212seconds),andunderstandablytherewerenolengthierstays.Thompson(2013)reportssimilarsearchingtrendsfromasurveyof388firstyearstudents,whereitissuggestedthatstudentsusethesearchenginesto'getin,gettheanswer,getout'.Giventheseshortstays,itisessentialthatreadersbetrainedinspeedreadingtechniques,orasThompsonrecommends,studentsbegivenexplicitinstructioninformingsearchtermsandevaluatingthediscoveredinformation.

    Astudy(Rowlandsetal.,2008)exploringstudentsonlineinteractionswithebooksandejournalsmadeavailablethroughanacademiclibraryshowslittleimprovementinuserretentionandreadingtimes.Theresearchersinthiscaselookedatpreviousliteratureandcombineditwithloganalysisofuniversitystudentstoindicatethatuserstypicallyspendonly4minutesonebooksitesand8minutesonejournalsites.Indeed,peoplegenerallyviewonlyapageortwofromanacademicsitebeforeexiting,andthentheypowerbrowsehorizontallythroughseveralarticleshopingforquickwinsandintheprocess,avoidreadinginthetraditionalsense.

    Withnodistractionsandworkingwithalimitedsetoffocuseddocuments,Internetlearnersseemtoperformequallywellonscreenandonpaper,inlearningandcomprehension,ashas

    http://www.informationr.net/ir/20-2/paper674.html#hea07http://www.informationr.net/ir/20-2/paper674.html#tho13http://www.informationr.net/ir/20-2/paper674.html#row08http://www.informationr.net/ir/20-2/paper674.html#liy12http://www.informationr.net/ir/20-2/paper674.html#coi11http://www.informationr.net/ir/20-2/paper674.html#wei08http://www.informationr.net/ir/20-2/paper674.html#sub13http://www.informationr.net/ir/20-2/paper674.html#cul11

  • beenshownbysomeresearchworkse.g.(EdenandEshetAlkalai,2013Moyer,2011).However,paperformatsstillcomeoutawinnerwhenitcomestolengthieranddensercontents(Stoop,KreutzerandKircz,2013).Inhisoverviewofdigitalreading,Liu(2012)concludesthatonlinereadingbehaviourofyoungadultsisevolvingastheydevelopnewstrategiestodealwithnewtechnologies.However,activereading(includingannotationsandhighlights)usuallyaccompaniesindepthreadinganditisunlikelythatthepreferenceforpaperasamediumforactiveindepthreadingwilldisappearsoon.Thoughonlinereadingtrendscontinuetogrowandstudentswillcontinuetousetheonlinemediaforactivereadingasmuchasthetechnologyallows.Itisuptotheeducatorstoguidetheminitsusage.

    WatchingvideosontheInternetisanotherinterestingaspectthatneedsfurtherexplorationinconventionaleducationandlearning.Timerequirementsandthesequentialnatureofcontentaccessaresomeofthevisiblebottlenecksthatneedtobeovercome,andonesuggestedmannerismakeavailablethetext,oratleastasummaryofthevideo,tohelpthereadersdecideiftheywishtospendthetime(Schade,2014).Thatway,theusershavemorecontroloverwhatvideo,and/orpartofit,theywishtoaccess.Otherwise,asMargaryanetal.(2011)discovered,morethan50%ofyearthreeuniversitystudentsneverusevideosduringformallearning,whilealittleover10%ofstudentsusevideosregularlyintheirlearningwork.Hardlyanyofthemcreatedvideosforlearning...

Recommended

View more >