Embed Size (px)
Citation preview
Making Assessment More Meaningful and Sustainable at Scale with the Multi-State Collaborative
Jeff Reid, TaskstreamPaula Haines, University of Massachusetts - Lowell
assessment ∙ accreditation ∙ e-portfoliosEDUCAUSE Learning Initiative Annual Meeting, February 2016
Outcomes
• Understand the goals and scope of the Multi-State Collaborative (MSC) to Advance Learning Outcomes Assessment
• Understand the benefits and experiences for participants
• Appreciate how simple technology can reduce barriers to doing direct assessment of student learning outcomes (SLO’s)
• Consider how the technology could be used on your campuses to engage faculty and start or improve assessment initiatives
• Founded in 2000
• 500+ client institutions
• Proven, reliable technology
• Experienced, caring team
• Dedicated to client success
Taskstream advances effective assessment to improve student learning and institutional quality.
Let’s start with two questions that have been keeping us up at night…
Why is it so difficult for so many institutions to engage in meaningful, sustainable
assessment of student learning?
How easy can we make this?
The Multi-State Collaborative
(MSC) to Advance Learning
Outcomes Assessment
VALUEGates-funded project to assess essential learning outcomes with authentic student work scored by faculty using VALUE rubrics
The MSC Challenge: Scaling Direct Assessment
Institutional Perspective
• Mid-size, public institution; 17,500 students• Doctoral University: Higher Research Activity• 6 Colleges & Professional Schools• 200+ Degrees and Certificates• Accredited by NEASC• Strong faculty union
Institutional Perspective
MSC Benefits• Establishing a baseline measure of
student learning in key outcomes• Reinforcing practice of assessment
using authentic student work• Developing reliable, cost-effective,
sustainable scoring protocols • Providing faculty development for
improved teaching and learning• Encouraging faculty engagement
with assessment
MSC Challenges• Dispelling suspicion • Recruiting faculty• Collecting representative artifacts• Trusting the process• Trusting the results
How we approached the work at UMass Lowell
• AMCOA: statewide support network (2013-14 state pilot)• Multi-institutional IRB • Office of Institutional Research: demographic data
• Faculty recruiting• Collection of artifacts• Redacting, coding, & uploading artifacts
• National, state, and local scoring• Reporting and feedback (still in progress)• Closing the loop with local projects
The MSC Challenge: Scaling Direct Assessment
• All with day jobs and other responsibilities
• Each role had distinct goals, tasks
• Limited opportunity to train
• Limited time to complete their work
• Focused on one activity at a time
PROJECTLEADS
DATALEADS
FACULTYSCORERS
Technical Challenge: Engaging Users
How easy can we make this?
Visual tracking reports to show progress of collection and scoring
PROJECTLEADS
DATALEADS
FACULTYSCORERS
Engaging the MSC users: How we did it
use space for screenshots or other large images
Track progress of collection across the project and by statePROJECTLEADS
use space for screenshots or other large images
Track progress of scoring overall and by outcomePROJECTLEADS
Visual tracking reports to show progress of collection and scoring
PROJECTLEADS
DATALEADS
FACULTYSCORERS
Engaging the MSC users: How we did it
Simple tools for tagging assignments and bulk uploading artifacts
Create and tag assignments with key metadataDATALEADS
Bulk upload student artifacts (papers)DATALEADS
Visual tracking reports to show progress of collection and scoring
PROJECTLEADS
DATALEADS
FACULTYSCORERS
Engaging the MSC users: How we did it
Basic tools for tagging assignments and bulk uploading artifacts
Simple, intuitive and tablet-friendly evaluation experience to make scoring a breeze
A simple queue of workFACULTYSCORERS
Easy interaction with the contentFACULTYSCORERS
Quick access to the rubricFACULTYSCORERS
Clear way to review scoresFACULTYSCORERS
Little User Training Needed
• Demonstrated the “VALUE Rubric Scoring System” to MSC participants
• Training was a 30-minute demo and 15 minutes of questions
How easy did we make it?
“Taskstream took any concerns about technology out of the
scoring experience!”
“Taskstream was a useful system that allowed me to assess the student artifacts quickly.”
“I was pleased with the ease of using and dependability of Taskstream.
“Taskstream was great. It was an easy system to use, understand, and navigate.”
“Taskstream is very user-friendly.”
MSC pilot study participant feedback
Campus reflections on the Pilot Year
• The starting point of assessment work- Understand what we had and where we were- Develop an incremental plan
•Relationships first - Cultivate trust & engagement around teaching- Meet individual faculty where they are, be flexible- Build capacity for collection and scoring through multiple efforts- Start conversations for the long term
• Technology to magnify limited resources- Ability for one person to contain and manage a large number of documents- Ease of use for constrained time- Reliable technical assistance- Tools to help close the loop
The questionwe barely dared to ask ourselves:How much bettercould this get?
Reflections and Evolution
• Improve user experience for uploading student work
• Present assessment results with visually engaging reports
• Offer additional ways to bring work into the system
• Support multiple artifacts per student, assessment of multiple outcomes for each assignment and even “live” evidence.
• Provide the means for institutions to engage in similar initiatives locally
Beyond the MSC: Local Assessment
A breakthrough response to the complexities ofoutcomes assessment that provides a simpler pathto meaningful assessment and lets you focus moreclearly on improving student learning.
A simple path for direct assessment of learning
Critical Thinking
Written Communication
Intercultural Knowledge
What?Which outcome(s)
will you assess?
Where?Where will you get
the evidence?
course assignments
Who?Who will score
student work?
SimpleSetup
How?How will you collect
the evidence?
A simple path for direct assessment of learning
What?Outcome(s)
Where?Assignments
Who?Scorers
SimpleSetup
Data!Streamlined
Scoring Experience
Aqua is the latest member of our product suite to support you on your outcomes assessment journey…
AMS
Assessment Planning & Documentation
LAT
ePortfolio-Based Assessment
Aqua
Streamlined Direct Assessment
