1

2

3

NANCYSHOWOFHANDS:

1. Hasanyoneusedarubrictoevaluateorgradeindividualstudentwork(e.g.,aspartofacourseoraco-curricularprogram)?

2.Hasanyonedevelopedarubric?(Ormodifiedone?)

3.Hasanyonesharedarubricwithstudentsaspartofanassignment?Orengagedstudentsinapeerreviewprocessusingarubric?

4.Hasanyonebeenpartofateamassessingstudentworkattheprogramlevel usingarubric?(Definedas:youwoulddrawyourartifactsfromtheworkstudentsproducedaspartofthatprogram.)

Couldprobe:Where– inGE,major,cocurriculum?Graduateprogram?Couldprobe:Hasitbeensnapshot(e.g.,sophomoreyear)?Trends(e.g.,first

yearsandseniors)?Longitudinal?

5.Hasanyonebeenpartofateamassessingstudentlearningattheinstitutionallevelusingarubric?(Definedas:youwoulddrawyourartifactsfromtheworkstudentsproducedacrossmultipleprograms– GE+major,couldincludeco-curriculum.)

4

Notasapplicableforgraduateprogramsunlessperhapsinterdisciplinaryordualdegree/hybrid).

6.Hasanyoneled ateamassessingstudentlearningattheprogramorinstitutionallevelusingarubric?

POPCORN:Whatareyouhopingtogetoutofthissession?

4

5

NOTES: Samplewasallwegot(didn’tgetpapersfromallsections,butdidgetfrommost)

Embedded– questionofdefinition– ”students’bestwork”embeddedassessmenttypicallymeansexaminingstudentworkthathasbeenproducedinthecontextofcourses/curricula,vs.anexternalstandardizedtest,e.g.

“Embedded”canalsorefertotherubricsassessmentprocessitself.Inthiscase,therubricsassessmentwasexternaltothecourseentirely,andinstructorsofthosecourseswerenotinvolved.IntheWRITcasethatiscomingup,itwasablendofinstructorsandexternalreviewers.Athirdoptionwouldbetohavefacultyinstructorsonlyalsodoingrubricsassessment(inadditiontograding).Importanttoclarifywhatyou(andothers)meanby“embedded.”

6

Disciplinary diversityintermsofwhatconstitutesevidenceandperformancelevel.

GECmeetingstructure:Q:Howarewegoingtowoofacultyintoaroomfor2days?A:Takeawaysomemeetings!!

7

Could beimprovementstotheprogramcontent,totheprogramorganization,towaystudentscomeintotheprogram(likeDSP),orchangestotheassessmentprocess(SLOs,rubric,assessmentsessions,etc.)itself.

8

9

1. Handscoringwasapain!2. Themovefrombiweeklytomonthlyandbankingtime3. Cangetcaughtupinthingsearlyon,soimportanttokeepmoving!E.g.,know

whatyou’reassessing– writing,notcontentknowledge,curiosityabouttheassignment(that’sgood,butmaybenotatthatmoment),remindinggroupthatthey’renotgrading,cultureshockofgraduatefacultyreadingsophomorepapers(!)

4. Two-tierconsensusprocesshelpedkeepthingsmovingalsobutgoteveryone’svoice/perspectiveincluded

5. Normingprocessiscrucial– considerations:hasthegroupworkedtogetheralreadyoraretheybrandnewtooneanother?Beexplicitabouttheaimofachievingconsensus.

10

11

Programredesignin2014:WeSequencedthecurriculum:first-yearcompositiontoupper-divisionGErequirementforrisingjuniors

FacultycreatedSLOsforprogram,andforeachcourse:whatdowewantstudentstodemonstrateateachlevel,andbythetimetheygraduate?

Bestpracticesinwritingstudiesindicatesportfolioisbestmethodofassessinggrowth;FacultycollaboratedtodevelopSignatureAssignmentsforeachcourse:theyalsocreatedrubricstoassesstheportfoliosineachcourse.Thatway,studentshaveaconsistentexperienceacrossmultiplesectionsofthecourse.

Eachsemester,theinstructorcreatesassignmentsbasedontheSignatureAssignmentparameters,respondstothoseassignmentsthroughmultipledrafts,andthengiveseachstudent’sportfolioascoreattheendofthesemester.Thisisamajorcomponentofthecoursegrade.

Then,allfacultybringtheirportfolios(or,foronlineclassestheirgoogledocs files)toportfolioscoringday;weallgettogethertoassessallthestudents’portfolios(around1800inanygivensemester!).

Atportfolioscoringday,wespendsometimereviewingthecourserubric,sharingSignatureassignments,andcalibratingourscoresusingsampleportfoliosfromeach

12

level.

FacultyscoreportfoliosNOTfromtheirownclassesusingthecourserubric—soeachportfoliohastwoscores,theinstructorgavethefirstscore,andananonymoustrainedreadergavethesecondscore.SEEHANDOUTFORPORTFOLIOSUBMISSIONCOVERSHEET.

Instructorandotherscorerwritescoresonbottomofsheet.Scoresareholistic:Outstanding,Pass,Fail.Whenthescoresarethesame,that’sthescore;whentheyaredifferent,athirdtrainedreaderscoresit.We’rethereallday,usually:it’salotofwork,butalotoffun!

Thereallygreatthingisthatweaccomplishprogramassessmentatthesametime–weuseadevelopmentalprogramrubricforthis.

SEEHANDOUTfordevelopmentalRUBRIC.

Developmental meansthattherubriccanassessastudent’swritingskillatanylevelinthesequenceofwritingcourses.Weexpectthatstudentsinfirst-yearcompositionwillscorea1or2inmostcategories;andweexpectthatstudentsinjuniorcompositionwillscorea3or4.

Program meansthatweusethisrubrictoevaluatehowwellwearedoingasaprogram.Asfacultyscoreeachportfolioforthecoursegrade,theyalsoassessonthisdevelopmentalrubricandrecordascoreonaseparatepagejustforprogramuse.

12

Weknowportfolioassessmentiseffective, butitdoestaketimetosetupanddowell:soit’sgreatthatwehavebeenabletointegratecourseandprogramassessmentinthisway.IttookasemesterandasummertosetuptheSLOsandsignatureassignmentsandrubrics.Oncethosewereinplace,planningandpullingoffthebigday—portfolioscoringday– theadministrativetimetoplanit,andthefacultytimethere,ismostlywhatittakestosustain.Awholedaytoscore,anotherworkdayforthedirectortocollateprogramassessmentresultsandcreatereport.Wedothiseverysemester,tomonitoreffectivenessofprogramreform.

Wehavealsomanagedtodothisverycheaply.Sofar,ourportfoliosarepaperforf2fclasses.Foronlinesectionswe’veusedgoogledocs,aswedon’thaveacampusplatformthatwouldworkwell.Aneportfolio platformwouldbeideal—butthosecanbeexpensive– doesanyoneherehaveacampusePortfolioplatform?

Themainresourcewe’vereliedontomakethissystemworkhasbeenincrediblegoodwillandcommitmentofouradjunctfacultywhoteachintheprogram.Theyarenotpaidtoattendscoringday,buteverysemesterwe’vehad100%attendance.WealsoinviteGECouncilmembers.TheDeanhasprovidedjustenoughfundingtocoverlunch,andmyhusbandbakesbrownies!

13

Fall14fyc andjuniorwritingbothhadlowerscoresinwritingprocesscategoryofdevelopmentalrubric:usedthisevidencetorampupfacultydevelopmentforportfoliopreparation,forexample,totalkaboutvalueofreflectionaspartofwritingandlearningprocess– manyfacultynewtoportfolioassessment-- andaboutwaystodevelopstrong,usefulreflectionassignments.Fall15showedhigherscoresinthatarea.

WealsorevisedourDirectedSelf-Placementmaterials:first-yearcompositionstudentscominginsawanexplicitfocusonself-reflectionaspartofthewritingprocess,andpracticedself-evaluationoftheirownhabitsandskillsinthatarea.

Thefollowingyear,proofreadingstrategiesweretheareaneedingmostimprovement,sowefocusedfac deveffortsonthatcategory.

Wealso respondedtofacultyfeedbackbyadding“Outstanding”asapossiblescoreforsecondreadertogiveonPF;betterincentiveforstudentstodowellbeyondpass/failThebottomlineonthisistorememberthatlikewriting,assessmentisaprocess– allthedocumentswecreateareworkingdrafts,readyforrevisionasthecontextsuggests.

14

Adownsideisthatweareaskingfacultytocomeintospendawholedayonprogramassessment.Theupsidethoughisthatfacultycomeintospendawholedayonprogramassessment!

Byfartheenormousadvantageofportfolioassessmentasagroupistheperfectfacultydevelopmentmomentitprovides:whilefacultyarescoringportfoliosofstudentsfromothersectionstheygettoseetheircolleagues’assignments;inthestudents’reflectiondocumentstheygettohearaboutthekindsofactivitiestheircolleaguesareteaching,aboutreadingsandhomeworkassignments.Wecertainlyencouragedthisintheprogramasweaskedfacultytoworkcloselytogethertodevelopthenewcurriculum,andtheyhavecontinuedthisapproach.

Facultyreallyappreciatehavingthesupportandconfidenceofoneanotherastheygradetheirstudents’work;andatthescoring,theyappreciatehavingtherecommendationofthefirstscorer—whoisthestudent’sinstructorandknowsthestudentbest.

They’vealsotoldmethattheyappreciatenothavingtheusualmassivepaperload attheendofthesemester:portfolioreviewisnotaslaboriousasgradingthatgiantstackofresearchpapers,astheyhaverespondedwithformativecommentsonallthedraftsthroughoutthesemester,sopfscoringissimplythat,areviewofthefinaldocumentsusingtherubric.

15

Onepotentialissueisfiguringouthowtomakestudent’sgradeworkwiththeportfolioscore:westartedoutbysayingthatthePFhadtoearnaPASSinorderforthestudenttoearnapassinggradeintheclass;facultywantedclearerguidelines,sowedevelopedthemthesecondyear—nowtheportfolioscoreisatleast75%ofthecoursegrade.Eachprogramhastofindawaytomakethatworkfortheirowncontext.

15

16