Embed Size (px)
Citation preview
KENTUCKY: POLICIES & PRACTICE
Preventing, Detecting, and Investigating Test Security Irregularities: A Comprehensive Guidebook On Test Security For States
NCSA Conference 2013: June 20, 2013 1
NCSA Conference 2013: June 20, 2013 2
How Many People Cheat On State Tests?
5% ?10% ?20% ?
NCSA Conference 2013: June 20, 2013 3
·
Kentucky Annual Summative and EOC Testing
Total Number Percent Cheating
Possible Number Cheating
Students Tested 380,000 1/10 of 1% 380
Teachers and Administrators 45,000 1/10 of 1% 45
NCSA Conference 2013: June 20, 2013 4
• Prevention• Detection• Investigation
1. Security Plan2. Roles and Responsibilities3. Budget and Funding4. Legal Precautions and Funding5. Test and Item Design
NCSA Conference 2013: June 20, 2013 5
6. Test/Item Development and Maintenance7. Test Publication8. Test Administration9. Test Scores and Results10.Physical Security
NCSA Conference 2013: June 20, 2013 6
11.Information Security12.Web and Media Monitoring (checking the
internet for test content)13.Security Awareness and Training
NCSA Conference 2013: June 20, 2013 7
1. Security Plano Findings: The Department of Education does not maintain a
single, comprehensive security plan/manualo Action Tasks:• Developing a comprehensive security plan/manual• Providing security plan/manual to all key staff and portions
to outside contributors as appropriate
NCSA Conference 2013: June 20, 2013 8
2. Roles and Responsibilitieso Findings: All Department assessment staff are involved in test
security with responsibilities for different test security processes; however there is no formal security management structure
o Action Tasks:• Appointment of a “Security Coordinator”• Design a plan instilling a strong security culture within the KDE• Build security roles and responsibilities into job descriptions and
performance appraisals
NCSA Conference 2013: June 20, 2013 9
3. Budget and Fundingo Findings: Strong commitment to test security from KDE
management, there is no separate budget for test security activities
o Action Tasks:• Develop an overall security budget• Plan for forensic analysis and increased allegations of
cheating
NCSA Conference 2013: June 20, 2013 10
8. Test Administrationo Action Tasks:
• Test security issues are covered in Test Administration Manuals; however continue to review and improve.
• Implement the requirement that two persons are present whenever test books are counted or accessed in preparation for testing, during testing and return stages.
• Modify Student Response Booklets to better engage students in maintaining test security.
• Capture additional data for future analysis of data.
NCSA Conference 2013: June 20, 2013 11
NCSA Conference 2013: June 20, 2013 12
12.Web and Media Monitor o Action Tasks:• Monitor the internet looking for test items and passages
that appear on KDE assessments.• Monitor discussion boards, especially around testing
periods.• Formalize plans for handling significant test breaches.
NCSA Conference 2013: June 20, 2013 13
Data Forensics Analysiso Action Tasks: Adding additional data forensics analysis• Erasure Analysis• Similarity Analysis• Score Growth and Losses
NCSA Conference 2013: June 20, 2013 14
Where Student is TestedEach Student Response Booklet has the location within the school where student is tested.
NCSA Conference 2013: June 20, 2013 15
Kentucky InvestigationsConducting Security InvestigationsoFindings: Kentucky has procedures for investigating possible security incidents and staff respond swiftly as needed.oAction Tasks:
• Increase investigative resources to handle the significant in allegations to avoid extended time from reporting to resolution.
• Change the current de facto policy on initiating a security investigation if the allegation has not come from the field.
NCSA Conference 2013: June 20, 2013 16
Test SecurityReporting of Allegations
Get the facts straight.•Record the date and time of the incident.•List all involved (i.e., staff, volunteers, students).•Identify content area and test parts impacted.•Identify specific content question(s), if applicable.•Record what you learn.•Report the incident.
NCSA Conference 2013: June 20, 2013 17
1. Commitment From Senior Leadership2. Professional Audit
NCSA Conference 2013: June 20, 2013 18
• Crosses Many Divisions in State Education Agency• Can Involve Other State Organizations• Requires District and School Leadership Support• Paper and Pencil• On-line Testing
NCSA Conference 2013: June 20, 2013 19
• Reasons for Independent Evaluationo Legislatureo Broader Experience
• Low Hanging Fruit• Medium Effort Solutions• High Cost Solutions
NCSA Conference 2013: June 20, 2013 20
Roger W. ErvinSystems Administrator
Kentucky Department of EducationOffice of Assessment and AccountabilityCapital Plaza Tower, 17th Floor500 Mero St.Frankfort, KY 40601
NCSA Conference 2013: June 20, 2013 21
