1ExamSoft Assessment Conference | #ASSESS2015

ExamSoft Assessment Conference

#ASSESS2015

2ExamSoft Assessment Conference | #ASSESS2015

Coming Full Circle: Tips & Tricks from Test Item Creation to Assessment Data Retrieval

3ExamSoft Assessment Conference | #ASSESS2015

Introduction

Western University of Health Sciences – Pomona, CA

4ExamSoft Assessment Conference | #ASSESS2015

Learning Objectives

• Discuss methods of engaging faculty members in appropriately tagging questions to Institutional Learning Outcomes (ILOs), Program Learning Outcomes (PLOs), & relevant topics.

• Understand the process in pulling informative longitudinal data using the tagged categories to derive meaningful data relevant to student performance/progress.

• Identify strengths and weaknesses of the curriculum.• Identify strengths and weaknesses of student

performance.

5ExamSoft Assessment Conference | #ASSESS2015

6ExamSoft Assessment Conference | #ASSESS2015

Our Previous Process

• Word document/RTF import• Problems with category tagging• Technical issues with import• Ineffective reporting

7ExamSoft Assessment Conference | #ASSESS2015

OUR CURRENT PROCESS

8ExamSoft Assessment Conference | #ASSESS2015

Landmark #1

9ExamSoft Assessment Conference | #ASSESS2015

Test Item Collection

• Question Export using Item Analysis feature– Point Bi-Serial– Question ID/Rev– Upper/Lower– Disc Index

• Creation of Question Bank

10ExamSoft Assessment Conference | #ASSESS2015

Test Item Collection

• Benefits– User Friendly– Quality Assurance– Collaboration

Platform– Organize Data

11ExamSoft Assessment Conference | #ASSESS2015

Test Item Collection

• Question Submission Form– Purpose: To collect questions from multiple faculty and

house in one central location– Structure:

• Question type• Instructor• Question text and indicate correct answer• Attachment (any file type accepted,

allows for hi-res images)• Lecture title• Location of content in lecture• Randomize answers & group questions

(Optional)

12ExamSoft Assessment Conference | #ASSESS2015

Test Item Collection

• Category TaggingTRICK! Clean up your categories by regularly organizing them in ExamSoft at the end of each term.

13ExamSoft Assessment Conference | #ASSESS2015

Test Item Collection

• Form interacts with Question Bank

• Increases efficiency for faculty & staff

• Simplified method of tagging categories that engages faculty members

• Ease of use promotes faculty buy-in

• Deadline CalendarTIP! Hold

development

workshops to

train faculty on

how to tag

consistently.

14ExamSoft Assessment Conference | #ASSESS2015

Landmark #2

15ExamSoft Assessment Conference | #ASSESS2015

ExamSoft Assessment Creation

• Data from the Question Submission Form is extracted as an ExamSoft template Excel file

• Staff converts spreadsheet to .txt file for import into ExamSoft

TRICK! Eliminate special character errors by using the“Find & Replace” feature in Microsoft Excel.

16ExamSoft Assessment Conference | #ASSESS2015

ExamSoft Assessment Creation

• Staff create assessment in ExamSoft portal• Staff export an exam draft to SharePoint

– Faculty members collaborate on reviewing/making necessary edits and corrections prior to final approval

17ExamSoft Assessment Conference | #ASSESS2015

Landmark #3

18ExamSoft Assessment Conference | #ASSESS2015

Generating Reports

• Pull reports & post to SharePoint for review:– Exam Taker Results– Item Analysis– Summary Report– Category Reports

TIP! Maintain a location that

houses question bank, exam

draft, and exam reports for faculty to access.

19ExamSoft Assessment Conference | #ASSESS2015

Generating Reports

• Faculty recommend key changes– Review item analysis/question metrics– Improve test items in question bank

• Score reports are released via ExamSoft• Scores uploaded to

students’ web-based academic profile.

20ExamSoft Assessment Conference | #ASSESS2015

Landmark #4

21ExamSoft Assessment Conference | #ASSESS2015

Reporting Longitudinal Data

• Mid-Term & End-of-Term• Pulled from ExamSoft

22ExamSoft Assessment Conference | #ASSESS2015

Reporting Longitudinal

• Uses of Data (Faculty, Staff, & Administration):– Academic Dean: University Strategic and

Effectiveness Report– Course Directors/Managers: Evaluate student

performance in courses– Curriculum Design Committee: Consider changes to

curriculum– Advisors: Pinpoint areas of weakness for student advisees

23ExamSoft Assessment Conference | #ASSESS2015

Conclusion

Benefits

• Streamline• Efficiency• Accountability• Technical• Reporting• Organization

Improvements to Workflow

• Staff time– Previous Process: 4 hours– Current Process: 30 minutes

• More manageable Question Bank

• Quicker turnaround time for assessment creation

24ExamSoft Assessment Conference | #ASSESS2015

Conclusion

• Coming Full Circle• Updating questions • Curriculum changes• Students track

progress from term-to-term and compare to class average. Beneficial for targeted learning.

• Lessons Learned• Questions?

25ExamSoft Assessment Conference | #ASSESS2015

Contact Information

Jayzona Alberto, MS

Academic Manager

(1st & 3rd Year Curriculum)

jalberto@westernu.edu

Jorge Godinez, BS

Academic Manager

(2nd & 4th Year Curriculum)

jgodinez@westernu.edu