View
31
Download
0
Category
Preview:
Citation preview
www.caveon.com1
The What, Why and How of On-Site Monitoring
March 1, 2017 12:00 p.m. ET
Caveon Webinar Series
Today’s Presenters
www.caveon.com2
Steve AddicottVice President , Sales
Marc WeinsteinVice President, Caveon
Investigative Services
Today’s Questions
•What is on-site monitoring?•Why implement on-site monitoring?•How do we get started with on-site monitoring?•What do we do with the information from monitoring?
www.caveon.com3
What Is On-site Monitoring?
Trained monitors visit schools and testing rooms during assessment administrations.Monitors will:Review and document school-level test security compliance practices and procedures.Observe test administrators, proctors and students.Review all parts of the testing day.Capture observations using pre-defined checklists for later review.
www.caveon.com5
Why Have A Monitoring Program?• Support the validity of assessment results.• Provide quality assurance for your testing program.– Gather information about implementation and efficacy of training programs.– Measure compliance with test administration processes and protocols.– Identify areas for improvement of process/policies.
• Comply with USED Peer Review Requirements.
www.caveon.com6
Should We Announce Monitoring?•Announced– Pro: Administrators are more likely to ensure compliance with policies and procedures.– Con: School personnel on “best behavior,” so not a true indicator or what happens during assessments that are not monitored.
www.caveon.com8
Should We Announce Monitoring?•Unannounced– Pro: Provides a true indicator of what happens during testing and encourages increased compliance at all times– Con: Added stress for test administrators
www.caveon.com9
During On-Site Monitoring
Look at all parts of the testing day–Training logs and proctor agreements.–School-level secure testing materials management.–Proctor and student login procedures.–Actual testing sessions in testing rooms.–Final sign-off/ secure testing materials management.
www.caveon.com10
During On-Site Monitoring
Focus on process and people– Test administration process.– School staff/proctor roles and behaviors.– Student behaviors.– Testing environment (classroom, computer lab).
www.caveon.com11
During On-Site Monitoring•Use objective checklist or form– Yes/no (binary) checklist items to eliminate discretion of monitors and allow for data analytics across schools.– Require explicit narrative descriptions of any observed potential irregularity or deviation from policies.– Clear procedures for data capture and management.– Archiving and evidence preservation.
www.caveon.com12
During On-Site Monitoring•Consider electronic tools for incident management– Real-time data capture.– Data analytics across schools.– Workflow tools for incident management.– Archiving and evidence preservation.
www.caveon.com13
Use of Monitoring Data and Results•Do irregularities or potential violations observed during monitoring require follow up investigation?•What level of monitoring data and results will be shared with schools and staff?•Should public reports of monitoring be released?
www.caveon.com15
Use of Monitoring Data and Results• Evaluate with other test security data sources.–Incident or irregularity reports.–Data Forensics or other test results analysis.•Compare monitoring data points across schools.• Evaluate monitoring data within overall test security context.
www.caveon.com16
Take Action• Provide feedback to staff at observed sites.•Conduct follow up investigations if warranted.•Select sites for future monitoring.• Improve test materials, instructions, or training.
www.caveon.com17
Three questions to ask•What are the goals of monitoring?– Quality assurance.– Ensuring compliance.– Evaluating training effectiveness.•What resources are available for monitoring?•What will we do with the data from the monitoring?
www.caveon.com19
Six Steps
1. Set purpose and goals.2. Determine budget/funding source.3. Select sites.4. Evaluate staffing options.5. Develop audit forms.6. Set periodic program reviews.
www.caveon.com21
Recommended