Report on the Arapahoe High School Shooting...Report on the Arapahoe High School Shooting ii our schools are now, as never before, in a unique position to identify and secure help

REPORT ON THE

ARAPAHOE HIGH SCHOOL

SHOOTING:

Lessons Learned on Information Sharing,

Threat Assessment, and Systems Integrity

Sarah Goodrum, Ph.D. William Woodward, M.P.A.

University of Northern Colorado University of Colorado Boulder

January 18, 2016

Report on the Arapahoe High School Shooting:

Lessons Learned on Information Sharing, Threat Assessment,

and Systems Integrity1

Presented to

and

Colorado SB 15-214: Committee on School Safety and Youth in Crisis

In compliance with JAG No. 2015-0665A, In re the arbitration of:

Michael and Desiree Davis, Claimants and Littleton Public School District,

Respondent

William Woodward, M.P.A.

Center for the Study and Prevention of Violence

University of Colorado Boulder

Boulder, CO 80302

Sarah Goodrum, Ph.D.

Department of Criminal Justice

University of Northern Colorado

Greeley, CO 80639

January 18, 2016

1TheprojectwasfundedbyagranttheArapahoeHighSchoolCommunityFundHonoringClaireDavis,adonor-advised

fundofTheDenverFoundation.ReportdesignbyRachelKennedy,CenterfortheStudy&PreventionofViolence.

DEDICATION

This report is dedicated to Claire Davis,

and her parents, Michael and Desiree Davis.

A MESSAGE FROM MICHAEL & DESIREE DAVIS

Report on the Arapahoe High School Shooting i

As we write this dedication page, almost two years after our daughter died

from the gunshot wounds she suffered at a Colorado high school on

December 13, 2013, we are reminded of the God-given gift of life that we all

share. One year ago, on the first anniversary of the tragedy that took this gift

away from our daughter, we were contemplating what we would say to the

students and families that gathered at the school to remember our daughter.

The school wanted to promote healing for the community; we shared that

goal, but also felt compelled to remember the students that have lost their

lives to school violence in this State. With the help of a number of friends,

students and Governor John Hickenlooper, we launched 14 large illuminated

balloons into the star-lit Colorado sky that evening, in loving memory of Rachel

Scott, Daniel Rohrbough, Kyle Velasquez, Steven Curnow, Cassie Bernall, Isaiah

Shoels, Matthew Kechter, Lauren Townsend, John Tomlin, Kelly Fleming, Daniel

Mauser, Corey DePooter, Emily Keyes and our daughter, Claire Davis. Almost

everyone in the crowd held a candle, and the Governor’s flame was passed

around until all the candles were lit. The balloons were to remember the kids

we have lost, and the candles were to encourage all of us that remain to work

together to light the way for a more peaceful and loving future.

On the campus of Arapahoe High School in Centennial, in a field known as

Clarity Commons, stands a large granite pillar with the inscription: “All that you

are is a result of what you have thought.” When applied to the entire

community, and even the entire State, one is left to ponder the implications of

our collective thoughts and points of view on society at large. Perhaps if we

individually turn our attention and thoughts, and then our actions, to being

more compassionate, tolerant and willing to help others, then collectively our

communities will become less harsh and less violent. The angry young man

that murdered our daughter was a student in crisis who desperately needed

guidance in a different direction from the one he pursued. The lesson to learn

is not that our schools should be less tolerant and more punitive, rather that

Report on the Arapahoe High School Shooting ii

our schools are now, as never before, in a unique position to identify and

secure help for troubled students. The current state of our society demands

that it’s time to change our thinking about the role schools should play in the

lives of students in crisis. Schools are the first place in most children’s lives

where they learn to socialize and it should be one of the first places where

children learn to practice respect for themselves as well as others. In many, if

not most cases, helping troubled youths with unmet emotional needs costs

nothing more than some time given by a caring administrator or teacher to

lend a helping hand, share words of hope and encouragement, and open the

door to other available resources. The goal of this report and the entire

arbitration process was to encourage this change in thinking about our public

schools – to challenge parents, administrators, teachers and legislators to

embrace a caring, tolerant and compassionate culture that empowers our

schools to intervene and help kids in crisis.

Going through the arbitration process was our gift to the State of Colorado. It

is now up to the parents of public school students, school administrators and

our State legislators to take the recommendations in this report and

implement them – to put into practice the things we have learned from this

report so that all the children are safe from harm in our public schools.

We are extraordinarily thankful to the University of Colorado’s Center for the

Study and Prevention of Violence for writing this report and making the school

safety recommendations included here. In particular, we are grateful to Bill

Woodward of the University of Colorado and Sarah Goodrum of the University

of Northern Colorado for their labor of love in attending all of the depositions,

collecting the data, and writing the report. We also want to express our

sincerest thanks to: our friend and attorney, Michael Roche, of Lathrop & Gage,

who handled the depositions with a graceful and compassionate expertise that

is rare in the legal profession; Arapahoe County Sheriff Dave Walcher, who

Report on the Arapahoe High School Shooting iii

originally conceived of the idea to engage the Center for the Study and

Prevention of Violence to extract the lessons to be learned from our

daughter’s tragic death; the administrators of Littleton Public Schools and

Arapahoe High School, and the teachers of Arapahoe High School, for the

open and honest information provided in the arbitration proceedings with the

shared hope of preventing future shootings in Colorado schools; to the

Republican and Democratic leadership of the Senate and House during the

2015 legislative session, who had the courage to introduce legislation that

turns Colorado’s attention to safer schools; Desiree’s mother, Lois, who has

never wavered in her love and support; and our dear friend and confidante,

Carol, who stood by us in love and understanding throughout the entire

arbitration process.

Choose to love.

Michael & Desiree Davis

December 2015

ACKNOWLEDGEMENTS

Report on the Arapahoe High School Shooting iv

The authors would like to thank Michael and Desiree Davis for their unwavering

commitment to learn from the tragedy that stole their beautiful daughter

Claire. Despite their tremendous grief, they have managed to both honor the

memory of their daughter and feel sympathy for the troubled teen that killed

her and then himself. We also want to thank Michael Roche, Robert Lembke,

Carol Lembke, Arapahoe County Sheriff David Walcher, and Arapahoe County

Sheriff’s Office Investigator Kristin McCauley, who have been dedicated to the

fact-finding effort and the Davis family. We are grateful to Dr. Delbert Elliott

(Distinguished Professor Emeritus, Director of the Program on Problem

Behavior, University of Colorado Boulder) and Dr. Beverly Kingston (Director

of the Center for the Study and Prevention of Violence, University of Colorado

Boulder) for their decades of research and work on school safety. Their vision

and optimism are inspirational. We also want to acknowledge others for their

advice and help along the way:

• Brad Bogue, CEO, Justice System Assessment and Training

• Dawn Bunnga, Jefferson County Public Schools

• Cynthia Coffman, Colorado Attorney General

• Dewey Cornell, Professor, Curry School of Education, University of Virginia

• Tony Dyl, Assistant Solicitor General, Colorado Attorney General’s Office

• Katharine Evans, Juvenile Assessment Center, Jefferson County

• Monica Fitzgerald, Center for the Study and Prevention of Violence

• Pat Hamilton, Director of Safe and Secure Environment, Adams 12 Five Star School District

• Christine Harms, Director, Colorado School Safety Resource Center

• Eric Jacobson, Risk Management Director, Colorado School Districts Self Insurance Pool

• John Michael Keyes, Executive Director, “I Love You Guys Foundation”

• Susanne Maher, Professional Research Assistant, Center for the Study and Prevention of

Violence

• John McDonald, Executive Director, Security and Emergency Management, Jefferson

County Public Schools

• Sarah Park, The Denver Foundation

• Susan Payne, Executive Director, Safe2Tell

• Maryann Peratt, Judicial Specialist, Jefferson County Public Schools

• Kelly Purdy, The Denver Foundation

• Andrew Thompson, Graduate Student, University of Northern Colorado

• Denitta Ward, Director of Contract and Grants, University of Colorado Boulder

We also thank our spouses and children, who understood the importance of

this work and our heartfelt commitment to it. Thank you to Kathleen

Woodward and Paul, Sam, and Henry Goodrum.

CONTENTS

FORWARD 1

Preface 1

Mission & Scope 4

Disclaimer 5

Acronyms Glossary 6

EXECUTIVE SUMMARY 7

Information Sharing 8

Threat Assessment 9

Systems Thinking 12

Major Recommendations 15

PART 1: BACKGROUND 20

Shooting at Arapahoe High School 20

Prior Research 22

Project Data 33

PART 2: FINDINGS 37

MAJOR FINDING 1: INFORMATION SHARING 37

Finding 1a: Infinite Campus 44

Finding 1b: Safe2Tell 50

Finding 1c: Interagency Information Sharing Agreement 55

MAJOR FINDING 2: THREAT ASSESSMENT 58

Finding 2a: LPS’s Threat Assessment Guidelines 59

Finding 2b: LPS’s Threat Assessment Training Procedures 78

Finding 2c: Use of Untested Risk Assessment & Threat Assessment Tools 79

Finding 2d: Threat Assessment Safety and Support Action Plan 83

MAJOR FINDING 3: SYSTEMS THINKING 85

Finding 3a: Using a Continuous Improvement Model (Short Term) 85

Finding 3b: Understanding a Complex Problem 91

Finding 3c: Solving a Complex Problem (Long Term) 100

PART 3: RECOMMENDATIONS FOR LEGISLATION AND POLICY REFORMS 105

Recommendations on Major Finding 1: Information Sharing 106

Recommendations on Major Finding 2: Threat and Risk Assessment 107

Recommendations on Major Finding 3: Systems Thinking 108

REFERENCES 109

PART 4: APPENDICES 115

Appendix 1: Chronological List of KP’s Concerning Behaviors 115

Appendix 2: Differences between Tools for Risk Assessment and Threat Assessment 118

Appendix 3: Child in Crisis Assessment Recommendation 119

Appendix 4: Interagency Agreement and Social Support

Team Self-Assessment Check List 122

Appendix 5: Timeline of KP’s Concerning Behaviors 123

Appendix 6: Skills Training with Guided Practice 124

Appendix 7: Threat Assessment of KP 125

Appendix 8: Briefing on Virginia Student Threat Assessment Guidelines 129

Appendix 9: CSPV’s Safe Communities Safe Schools Model 132

FORWARD

Report on the Arapahoe High School Shooting

1

Failure does not strike like a bolt from the blue; it develops gradually

according to its own logic. As we watch individuals attempt to solve

problems, we will see that complicated situations seem to elicit habits of

thought that set failure in motion from the beginning.

Dietrich Dörner, The Logic of Failure (1996, p. 8)

We all carry some guilt.

Darrell Meredith, AHS Assistant Principal (2015, Deposition, p. 174)

Preface

As Dietrich Dörner, suggests in the above quote, the errors leading up to the

December 13, 2013 shooting at Arapahoe High School (AHS) did “not strike like

a bolt from the blue.” The errors developed gradually over several months and

years, and as AHS Assistant Principal Darrell Meredith eloquently stated, “We

all carry some guilt” for the tragic incident that left two students dead. The

truth is that we all do carry some guilt, because as researchers, legislators,

policy makers, educators, parents, and community members, we have yet to

adequately solve the problem of school violence in the U.S. Malcom Gladwell

(2015) has recently argued that school shootings have, sadly, become a

problem that we live with, not a problem we work to solve (see also Blair, et

al., 2014; Doyle, 2010). Persistent and threatening problems, like school

violence, demand a comprehensive strategy, in which multiple solutions get

implemented simultaneously within social institutions (e.g., schools, districts,

legislature, mental health care) and the culture (e.g., values, beliefs, and

attitudes) (Costa, 2012).

The findings presented here will prove difficult to read. The evidence indicates

that several individual staff at AHS and within Littleton Public Schools (LPS)

made many small errors. More significant is the fact that the system in place

within the school and the district failed catastrophically – in both the


2

prevention of school violence and the promotion of school safety. As is

common with tragedies, there will be a tendency to blame one or two frontline

actors and the shooter’s parents as the cause of the problem in this case (see

Costa, 2012; Doyle, 2010). The personalization of blame, however, does not

actually solve the problem of school violence; it mistakenly focuses the

problem on one or two individuals, not the latent system flaws and

organizational culture that created the problem (see Costa, 2012; Doyle, 2010).

Healthy organizations build systems that view human error as inevitable,

design systems that can prevent and absorb human error, and create climates

that encourage workers to take appropriate action when they become aware

of mistakes.

The recommendations presented in this report may also prove difficult to

accept, because they outline the comprehensive reforms needed to promote a

culture of safety within schools. The complexity of the problem, the financial

implications of the solutions, and the fear of an innovative comprehensive

approach must not paralyze us. The report represents a call-to-action for

schools, districts, legislatures, and larger society to create positive school

climates, assess and support students in crisis, and (continuously) reflect upon

school safety efforts.

In The Other “F” Word: How Smart Leaders, Teams and Entrepreneurs Put

Failure to Work, John Danner and Mark Coopersmith (2015) argue that the first

step to improvement within organizations is admitting failure. Many leaders,

however, fail to acknowledge mistakes, and the evidence indicates that the

leaders within AHS and LPS were no different. “Accepting failure without

learning from and leveraging it is a recipe for mediocrity” (Danner and

Coopersmith, 2015, p. 26). Of course, admitting failure, particularly following

the tragic death of two students on school grounds, proves incredibly difficult.

But admitting failure can also be restorative and transformative (Doyle, 2010).


3

Relying on evidence from the medical field where offering an honest apology

reduced malpractice settlements (Doyle, 2010) 2 , AHS and LPS’s early

admittance of failure in this case might have accelerated the lessons learned.

Thus, the first step will be for AHS and LPS leadership to own the

organizational errors that created the conditions that led to the December 13,

2013 shooting. The next step will be to implement the recommendations on

information sharing, threat assessment, and system reform. The findings are

specific to AHS and LPS, but the recommendations may be relevant to all

schools and districts.

While not the focus of this report, preliminary information suggests that AHS

and LPS have started to institute new policies and practices that may make a

difference for school safety in the future. LPS’s new Superintendent, Brian

Ewert, and AHS’s Principal Natalie Pramenko appear willing to acknowledge

the failures and promote the reforms necessary to make AHS and LPS a safer

place for children. In fact, Pramenko has already begun to rectify some of the

problems identified in this report, which provides a positive step toward the

full integration of the recommendations presented here. In her deposition

testimony, Pramenko stated (p. 201):

I think for me as the principal, as much as I have to delegate and rely on others, [I need] to be much more diligent and follow-up and follow through with my assistant principals with regard to any discipline issues, particularly resulting in suspension and/or mental health concerns, [for example, with] kids that are suicidal, kids that have made threats, kids that are bullied or being bullied. [I need to be] continuing the conversation, continuing to ask them: “What have you done? When have you last met with them?” That has definitely been a change. And holding them accountable, as well for having those conversations with counselors or teachers as appropriate, [and] communicating more with parents. I think overall [we should continue to improve] communication across the board at our school, you know, from teachers to teachers, from teachers to counselors, teachers to administrators, and every direction in between.

2 Similar results were produced when a Denver medical malpractice insurance company began a new programrecommendingtheuseofapologiesandquicksettlements.TheHartfordCourant(2006,p.1)reportedthat“paymentstoaggrievedpatientswereunder$6,000,comparedwithabout$284,000fordoctorsnotintheprogram."


4

This perspective will prove critical to improving the culture of safety at AHS

and within LPS.

Mission and Scope

To understand how similar school shootings might be prevented, the

Arapahoe High School Community Fund Honoring Claire Davis, a donor-

advised fund of The Denver Foundation, approached the Center for the Study

and Prevention of Violence (CSPV) at the University of Colorado Boulder to

assist with the collection, analysis, and interpretation of data obtained through

an investigative arbitration agreement reached between Michael and Desiree

Davis, representatives of the Estate of Claire Davis, and LPS. The investigation

sought to discover the facts and circumstances leading up to the December 13,

2013 shooting at AHS and LPS’s response. The purpose was to understand the

school’s risk and threat assessment procedures and responses, the school’s

approach to safety and climate, and the lessons that may be learned from this

incident that could improve youth violence prevention in school settings in the

future. More specifically, the goals of the arbitration were to provide

information to experts who could assist in: (1) developing policy

recommendations for identifying students in crisis, (2) outlining steps to

reduce the likelihood of and the severity of harm caused by students in crisis,

and (3) suggesting response protocols for best practices in response to a

student in crisis. In short, the ultimate objective is to discover ways to make

schools safer and to help prevent future tragedies like the one that occurred at

AHS.

The project scope did not include a review of the physical aspects of campus

security or emergency responders’ use of tactical responses at the time of the

shooting. The project also did not include a biographical or psychological

reconstruction of the shooter, as his mental health records were never made


5

available and his family members, private counselor, and friends were not

deposed in the proceedings. Individuals who participated in a deposition or

the Arapahoe County Sheriff’s Office Report are named in the report. Since all

depositions are being made public, the authors quote from those depositions.

Disclaimer

The report reflects the opinions of the authors and not the official position of

the University of Colorado Boulder, University of Northern Colorado, Denver

Foundation, Michael and Desiree Davis, LPS, or AHS. The data for the report

came from deposition testimony, interrogatory responses, deposition exhibits,

and the Arapahoe County Sheriff’s Office Report. The authors did not recruit

study participants or interview law enforcement officials, LPS staff, AHS staff,

AHS students, or AHS parents. Thus, the group of individuals deposed for the

arbitration proceedings may not necessarily represent the larger population of

people with knowledge of the case. Finally, the sequestration order in the

arbitration agreement did not allow for a peer review of the findings prior to

public release. This report represents a call for action to discuss, question, and

reflect upon school safety measures in Colorado. This is not the end of the

work on school safety; it is just the beginning.


6

Acronyms Glossary

Acronyms used in this report

ACSO Arapahoe County Sheriff’s Office

AHS Arapahoe High School

CSPV Center for the Study and Prevention of Violence

CSSRC Colorado School Safety Resource Center

FERPA Family Education Rights and Privacy Act

IEP Individual Educational Plan

ISST Interagency Social Support Team

LPS Littleton Public Schools

MTAT Multijurisdictional Threat Assessment Team

NREPP National Registry of Evidence-Based Programs and Practices

RRCU Risk and Resiliency Check Up

SAVRYTM

Structured Assessment of Violence Risk in Youth

SIT Student Intervention Team

SRO School Resource Officer

V-STAG Virginia Student Threat Assessment Guidelines

EXECUTIVE SUMMARY


7

While proximal error leading to an accident is, in fact, usually a ‘human

error,’ the causes of that error are often well beyond the individual’s control.

All humans err frequently. Systems that rely on error-free performance are

doomed to fail.

Lucian Leape (1994, p. 1852)

To better understand how the December 13, 2013 shooting at Arapahoe High

School, in which senior Karl Pierson (hereafter, referred to as KP3) shot and

killed Claire Davis and then himself, might be prevented, the Arapahoe High

School Community Fund Honoring Claire Davis, a donor-advised fund of The

Denver Foundation, approached the Center for the Study and Prevention of

Violence (CSPV) at the University of Colorado Boulder to assist with the

collection, analysis, and interpretation of data obtained from an arbitration

proceeding in the case. The purpose was to understand the school’s threat

and risk assessment procedures and responses, and the lessons that might be

learned from this incident that could improve youth violence prevention in

school settings in Colorado and the U.S. The data for the report came from

the Arapahoe County Sheriff’s Office’s (ACSO) investigation materials,

Littleton Public School’s (LPS) interrogatory responses, deposition exhibits,

and deposition testimony. The principal investigators attended most of the

depositions and reviewed all of the documents produced by ACSO and LPS.

The findings revealed three major failures within AHS and LPS in the months

and years leading up to the shooting: (1) a failure of information sharing, (2) a

failure of threat assessment, and (3) a failure of systems thinking. While not

the focus of this report, preliminary evidence indicates that AHS staff and LPS

administrators have made several changes in their approach to school safety

since 2013, and those changes represent important steps in the right direction

3 Inordertodrawmoreattentiontoschoolviolenceprevention,drawlessattentiontotheindividualshooter,andavoid

contributing to the “cultural script” on school shootings, this report uses the shooter’s initials and not his name (see

Gladwell,2015;Newman,etal.,2004).


8

and are noted wherever possible. However, a great deal of progress still needs

to be made. The findings and recommendations reveal the steps needed to

strengthen school safety at AHS and within LPS, but they should also be

reviewed and considered by other schools in Colorado. This Executive

Summary highlights the three major failures and 14 of the 32

recommendations.

Information Sharing

There were many missed opportunities to share information about and

intervene with KP prior to the December 13, 2013 shooting at Arapahoe High

School (see Appendix 1: Chronological List of KP’s Concerning Behaviors and

Appendix 5: Timeline of KP’s Concerning Behaviors). The three major failures

in information sharing included: (1) a failure to use the student information

system (e.g., Infinite Campus) to document behavioral and safety concerns

(e.g., threat, risk, academic, discipline response), (2) a failure to train students

and staff in an anonymous reporting system (e.g., Safe2Tell), and (3) a failure

to implement an Interagency Information Sharing Agreement (encouraged by

SB 00-133) to exchange vital information about students of concern with law

enforcement and other community agencies.

First, information about KP was not consistently maintained in hard-copy files

or AHS’s Infinite Campus student information database. Not one AHS teacher,

administrator or staff person had a complete record of KP’s history of

concerning behaviors over his more than three years at AHS, making it

challenging to adequately assess the threat he presented. If AHS staff had

consistently documented his behaviors, a pattern of “boundary testing” would

have been more apparent. Consistently using a student information system to

document student concerns makes it easier to identify the early warning signs

of violence, escalation in anger management issues, and decline in academic

performance. In addition, evidence indicates that FERPA was misinterpreted,


9

leading the school staff to believe that they would be more liable if they had

shared information about KP’s concerning behaviors, than if they had not.

Second, the Sheriff’s Report clearly states that at least ten AHS students had

substantive concerns about KP’s anger problems and gun ownership prior to

the shooting, but only one student reported their concern to a counselor and

no students reported their concerns to Safe2Tell (see ACSO Report, pp. 10-11).

If just one student or teacher, had called Safe2Tell, this tragedy might have

been averted. At the time of the shooting and as of July 2015, LPS and AHS

administrators did not have a policy regarding Safe2Tell training and did not

require that students or staff receive training on the Safe2Tell system. In fact,

the information shared about Safe2Tell at AHS was limited to a sticker on the

back of student identification cards, posters displayed in the school hallways,

and a PowerPoint slide displayed in the cafeteria.

Third, AHS and LPS failed to implement an Interagency Information Sharing

Agreement to facilitate the sharing of vital information about an individual’s

safety concerns with law enforcement, juvenile justice, and social services

agencies, which is recommended by Colorado statute (SB 00-133), the Center

for the Study and Prevention of Violence (CSPV), and the Colorado School

Safety Resource Center (see the CSSRC’s Essentials of School Threat

Assessment: Preventing Targeted School Violence, LPS 03421–03443).

Threat Assessment

There were two major failures with threat assessment in AHS and LPS: (1)

AHS’s failure to adequately implement LPS’s threat assessment policies and

(2) LPS’s failure to validate its threat assessment tool and process. First, AHS

administrators and counselors failed to implement LPS’s prescribed threat

assessment policy, including (a) thorough completion of the threat assessment


10

instrument, (b) staff-wide training on the threat assessment instrument, and

(c) adequate follow-up and safety planning.

Completion of the Threat Assessment Process. There was a minimal attempt

to proactively obtain information about all of the risk factors during the threat

assessment process. As a result, KP was assessed as a “low level” of concern

and the district did not review his threat assessment (at the time of the

shooting, the district only reviewed threat assessments with “medium” and

“high” levels of concern). In addition, the U.S. Secret Service’s six principles

and 11 questions – which were included in LPS’s Threat Assessment Training

PowerPoint (see Exhibit 4) – were inadequately investigated, and a “skeptical,

inquisitive mindset” was not used to evaluate the information in the case.

Training on Threat Assessment. In addition, there was a failure to train the

AHS principal, most assistant principals, and all teachers in LPS’s threat

assessment procedures. In fact, from 2011-12 to 2013-14, only seven AHS staff

received threat assessment training (see LPS, p. 00858). According to LPS’s

records, the principal was never trained and the assistant principal who

conducted the threat assessment of KP was never trained. Moreover, LPS’s

two-hour threat assessment training had no role-playing, one-on-one

coaching, and participants did not actually complete a mock threat

assessment. Research finds that didactic, reading, and audiovisual

presentation methods used by LPS in their threat assessment training typically

only yield 20% retention among participants (see Appendix 6: Skills Training

with Guided Practice).

Threat Assessment Follow-up and Safety Planning. AHS’s threat assessment

process did not include adequate follow-up, support, and safety planning for

KP. AHS did not create a physical location for the information vortex in the

student information system or establish an information vortex coordinator


11

within the threat assessment team, as recommended by CSSRC (Exhibit 5, LPS

03426) and implied in LPS’s Threat Assessment Training PowerPoint (Exhibit

4, LPS 0494). The safety plan was never updated after the threat assessment

follow-up meeting on September 26, 2013, in spite of the fact that some AHS

staff knew new risk and threat factors in October, November, and December.

The threat assessment performed at AHS and the follow-up safety plan

performed on KP, on September 9, 2013 did not follow LPS’s Threat

Assessment Training or the Secret Service’s basic principles of threat

assessment (see Fein, et al., 2002). For example, out of 24 possible risk

factors on KP’s threat assessment (Exhibit 35), only five were checked, and

this investigation revealed that seven to nine additional risk factors could have

been checked. If the threat assessment and follow-up plan had been properly

executed, KP’s violent plans might have been interrupted. A properly

executed threat assessment would have revealed a higher level of concern,

and a higher level of concern should have prompted more serious disciplinary

action and more thorough monitoring and support planning. If the threat had

been taken more seriously and an Interagency Social Support Team (ISST) had

been assembled, they could have crafted a support plan for KP. In this case

and as is common practice, AHS’s threat assessment team (e.g.,

Multijurisdictional Threat Assessment Team or MTAT) acted as both the threat

assessment team and the ISST. In general, the threat assessment team is

responsible for the threat assessment and monitoring, and the ISST is

responsible for building a support plan.

The second major failure on threat assessment in this case was LPS’s failure to

validate its threat assessment tool and process. Without a validated threat

assessment tool, or a plan to validate the chosen tool, there is no way of

knowing if it actually predicted violence. As an analogy, a physician would not

give a child a medication that was not tested and proven effective by the


12

Federal Drug Administration. Similarly, a threat assessment tool that has not

been tested and proven effective should not be used to evaluate a student’s

level of concern.

Systems Thinking

High schools include many systems designed to produce graduates with the

intellectual and social skills needed to prepare students for the rest of their

lives. In The Logic of Failure, Dietrich Dörner (1996) argues that systems fail in

small incremental steps, not with one catastrophic error. AHS and LPS’s

system failed at many points to get a handle on KP’s problems, in spite of the

fact that there were many warning signs and many opportunities. The list on

the following page captures the many small errors made prior to the shooting.


13

Systems Thinking Failures: Decisions Made Prior to the Shooting

Decision to not build a safety and support action plan for KP after incidents of violence in elementary

school, when early violence is clearly a strong risk factor for later violence (see Appendices 1 and 5)

Decision to ignore the possible impact of his parent’s divorce

Decision to not enlist the help of one adult at AHS that KP trusted in his safety and support action plan

Decision to not have a safety and support action plan (e.g., mental health referrals, follow-up meetings)

when KP yelled “fuck” in class and was suspended

Decision to not follow-up on KP’s use of inappropriate “penis” line in debate competition

Decision to not follow up on KP’s claims of being bullied by others and being a bully to others

Decision to not empirically validate LPS’s threat and risk assessment tool

Decision to not treat KP’s violation of the Assistant Principal’s request that he not attend speech and

debate team practices as evidence of “boundary probing”

Decision to only use two threat assessment team members in the threat assessment process, despite

state and federal guidelines

Decision to leave the School Resource Officer out of the threat assessment process

Decision to not assign a staff member to serve as the “information vortex” for KP during the threat

assessment

Decision to not have a district-level Safe2Tell training policy for high schools

Decision to not forward KP’s threat assessment to the district for review

Decision to not thoroughly check the facts and collect collateral information on KP in the threat

assessment process

Decision to not tell a student’s teachers the reason for a threat assessment, detention, or suspension

Decision to train threat assessment using only didactic and audio visual resources (see Appendix 6)

Decision to not formally suspend KP for his threat to “kill” Mr. Murphy

Decision to not formally suspend KP for his outburst in Ms. Lombardi’s Spanish class

Decision to not obtain video surveillance footage of KP making a threat about Mr. Murphy in parking lot

Decision to allow KP to return to school without the threat assessment team obtaining release of records

from KP’s private therapist

Decision to allow KP to stay in school, when requested release of mental health records was not

provided, as requested

Decision to only have one follow-up meeting to discuss KP’s progress with the safety and support action

plan

Decision to not recommend a Student Intervention Team (SIT) to support KP when his grades began to

decline

Decision to not inform the threat assessment team about KP’s viewing of guns and mass shootings on his

laptop

Decision to not search KP’s computer, locker, or possessions for confirmation of his viewing of guns and

mass shootings

Decision to not report KP’s purchase of a gun or interests in guns, as well as his anger problems, to

Safe2Tell

Decision to not re-open KP’s threat assessment case after being told he had an angry outburst in class

and had a gun


14

Not one of these decisions by themselves caused the shooting, but together

they compounded upon each other in a system ill-equipped to prevent them,

leaving almost no barriers to KP’s plans. In short, AHS and LPS lacked the

infrastructure to adequately evaluate, respond to and follow-up on students in

crisis. Responsibilities for information sharing, threat assessment, and follow-

up were spread across several people within LPS and AHS and not officially

assigned to anyone.

The evidence of faulty systems thinking within AHS and LPS included a

tendency for groupthink, a reluctance to reflect on and admit failure, and the

minimization of sincere concern. These findings represent the most

challenging and the most important of the problems to solve, because

information sharing and threat assessment cannot overcome an unhealthy

organizational system. According to research from a wide variety of fields

(e.g., the criminal justice system, hospitals, and aerospace engineering),

organizational errors do not occur as the result of one major mistake or one

bad apple employee (Dörner, 1996; Doyle, 2010). Instead, organizational errors

occur with “a small mistake here, and a small mistake there, and these mistakes

add up” (Dörner, 1996, p. 7). With a complex problem like school safety,

organizational errors prove difficult to resolve. Costa (2012, p. 179) suggests

that, under these conditions, “We need a short term plan to stay alive long

enough to have a permanent cure.” The findings indicate that, in the short

term, schools and districts should implement a continuous improvement model

of error review. In the long term, schools and districts should adopt Dörner’s

five steps for addressing the complex problem of school safety.


15

Major Recommendations

This section highlights 14 of the 32 recommendations presented in this report.

The goals of the arbitration were to provide information on how to identify

students in crisis, support students in crisis, and develop protocols for

responding to students in crisis. To reach these goals and to help prevent

future tragedies, schools and districts must first build safe school climates (see

Fein, et al., 2002). A safe school climate is one where “students view teachers

as being fair, the rules are universally enforced and students feel welcome, are

engaged in activities and know a teacher they can talk to about a problem”

(Elliott, 2009, p. 54). These recommendations seek to promote safety and

prevent violence in all school settings (Nekvasil & Cornell, 2015). While the

findings come from AHS and LPS, the recommendations may apply to many

schools and districts in Colorado.

The institutional barriers within schools, districts, and our culture will need to

be dismantled, including the belief that schools are powerless to manage

mental health issues. Schools can manage mental health and social support

issues. The task is complicated but it is not impossible. The promotion of

school safety will require the implementation of multiple mitigations in parallel.

Costa (2012) calls this “parallel incrementalism,” a mitigation strategy whereby

the cumulative effect of several incrementally useful strategies implemented in

parallel is exponentially more effective than one strategy implemented at a

time. The authors recommend that the following strategies be implemented in

parallel:

1. Recommend that principals, assistant principals, teachers, counselors,

psychologists, coaches, and School Resource Officers (SROs) consistently

use a student information system (e.g., Infinite Campus) to document


16

matters of a “public safety concern,”3 including student behavior concerns,

conduct violations, interventions, academic concerns, threat assessment

results, and safety and support action plans.

2. Recommend that schools and districts promote Safe2Tell in formal

trainings to students and staff each year, using skills practice, one-on-one

feedback, and coaching (see www.Safe2Tell.org and Appendix 6: Skills

Training with Guided Practice) and emphasizing the three core principles:

a. No one will know; Safe2Tell is an anonymous reporting system.

b. When someone could be hurt or injured, you have a duty to report

the concern to authorities and break the code of silence.

c. Safe2Tell is not limited to student reporting; the system is

available to all students, teachers, parents, staff, and community

members, and they also have a duty to report any safety concern

to either authorities or Safe2Tell.

3. Recommend that school districts complete an Interagency Information

Sharing Agreement with community agencies, including law enforcement

agencies, mental health service providers, social services agencies, and the

criminal justice system, as recommended by the Columbine Review

Commission, stated in C.R.S. § 22-32-109.1(3), and outlined by the Colorado

Attorney General’s Office. To facilitate this reform, it is recommended that

the words “if possible” be removed from C.R.S. § 22-32-109.1(3).

3IntheColoradoAttorneyGeneral’s“JuvenileInformationExchangeLaws:AModel forImplementation,”“PublicSafety

Concern” Information, HB 00-1119 creates a category of information that is now available to schools (see § 19-1-

303(2)(b)(I)C.R.S). It is crucial that local jurisdictionsadoptacommondefinition forwhen informationgives rise toa

“publicsafetyconcern”fortworeasons.First,thedatathatcanqualifyasa“publicsafetyconcern”isatthediscretionof

theagency.Second,alotofdatacanfallwithinthiscategory,becauselocalstandardsvary.Thefollowingprovidesanon-

exhaustive list ofwhat types of informationor incidents local jurisdictions can include in such a definition: any act of

violence or intimidation on school grounds or at a school sponsored event; any act that compromises school or

communitysafety(e.g.,threatsorexpresseddesirestocommitviolenceataschool);anyactorthreatthatinvolvesriskof

injury to multiple people, a student, or a school employee; any act involving a firearm or explosive device; any act

involving sexual assault; any act involving arson; any act involving cruelty to animals; any act of violence executed

pursuantadvanceplanning;anyactinvolvingthedistributionofnarcotics;informationconcerningastudent’saffiliation

withagang;informationconcerningastudentwithahistoryofactsfallingwithintheabovecategories.


17

4. Recommend that schools and districts install a validated threat assessment

process, by either using the Virginia Student Threat Assessment Guidelines

(V-STAG), by using a different validated threat assessment process, or by

validating the current threat assessment process with similar outcome

measures to V-STAG (see Appendix 8).

5. Recommend that schools and districts install a validated risk assessment

process, such as the Structured Assessment of Violence Risk in Youth

(SAVRYTM

) or the Risk and Resiliency Check Up (RRCU). Use the results

from the risk assessment to build a safety and support plan for any student

who has a threat assessment. Risk assessments incorporate both risk and

protective factors in the plan for the student.

6. Recommend that, during a threat assessment, the Secret Service’s six

principles and 11 questions be used to gather and evaluate the early

warning signs, threat factors, risk factors, and protective factors. The

process should emphasize an “investigative, skeptical, inquisitive mindset”

for each factor until a clear yes or no is found (Fein, et al., 2002, p. 29). All

threat assessment team members, and if needed the ISST members and

peers, should be included in the process (see Appendix 3).

7. Recommend that schools and districts train in a validated threat and risk

assessment process using a one-on-one cognitive behavioral training

standard (see Appendix 6). Adopt a formal training curriculum for threat

and risk assessment. Train all teachers and staff in the overall process, and

train principals, assistant principals, counselors, and SROs in a minimum of

one-day hands-on scenario driven training curriculum.

8. Recommend that an information vortex coordinator (from the threat

assessment team) be assigned to every threat assessed student; the


18

information vortex coordinator should be noted in the student’s profile

within the student information system so that when a concern arises, all

teachers and other staff can easily identify and communicate with the

coordinator. In addition, it should be the proactive duty of the information

vortex coordinator to continue to seek out and evaluate information about

a threat assessed student and recall the threat assessment team if new risk

or threat factors are revealed.

9. Recommend that the Colorado School Safety Resource Center (CSSRC)

audit any school or district requesting an audit for proper use of V-STAG

(or other validated threat and risk assessment process). Any school or

district that has implemented a validated process and receives a “high

pass” in an audit of that process could use the results as an affirmative

defense in any proceeding under SB 15-213. The audit process and

implementation guidelines should be reviewed by CSPV.4

10. Recommend that the threat assessment and support teams produce a

formal safety and support plan for every threat assessed student, relying

on Individual Educational Plans (IEP) and Student Intervention Teams (SIT)

as models. ISSTs build and monitor the plan for threat assessed students

and revise the assessment and plan whenever a new threat or risk factor

appears (see Appendix 3: Child in Crisis Assessment Recommendation).

11. Recommend that each threat assessed (or red flag) student be paired with

an adult in authority, ideally within the school, who can build a trusting and

positive relationship with that student.

12. Recommend that the Attorney General annually update the Colorado

School Violence Prevention and School Discipline Manual on school safety

4Inordertoavoidaconflictofinterest,theCSSRCshouldnotbereceivingsignificantfundingfromanyschool,district,or

school-basedassociation.


19

statutes, FERPA, and their application to school districts. Additionally,

recommend that school districts conduct an annual training on all statutes

related to school safety and violence prevention and produce an annual

compliance report.

13. Recommend that schools and districts conduct an established school

climate survey of students and staff every one to two years and when the

findings exceed established norms, select and implement experimentally

proven interventions, programs, and practices.

14. Recommend that schools and districts create a continuous improvement

model of error review committee to promote a culture of safety (and

minimize groupthink), whereby staff can report concerns about

organizational errors and near misses and staff can openly discuss, reflect

upon, and address concerns and mistakes without formal or informal

penalty. This committee should help develop short and long term plans for

school safety reform. Dörner’s (1996) five steps can help with long term

planning.

PART 1: BACKGROUND


20

Shooting at Arapahoe High School

On December 13, 2013, 18 year old senior KP shot and killed classmate Claire

Davis and then himself at Arapahoe High School (AHS). The Arapahoe County

Sheriff’s Office investigation revealed that KP displayed inappropriate and

concerning behavior at AHS and at AHS-sponsored events on several

occasions over a two-year period (see Appendix 1: Chronological List of KP’s

Concerning Behaviors and Appendix 5: Timeline of KP’s Concerning

Behaviors). In April 2013, KP received a one-day suspension from school for

yelling “fuck” in response to a bad grade in math class and “fuck you” to a

student in that class. In September 2013, he was removed as captain of the

Extemporaneous Team of the Speech and Debate Team, and yelled “I’m going

to kill that guy,” referring to Tracy Murphy, Speech and Debate Coach and

Head Librarian at AHS. KP was not formally suspended for the threat; instead,

after a phone call with his mother, Assistant Principal Kevin Kolasa agreed that

KP could stay home for three days.

When he returned to school on September 9, 2013, Kolasa and AHS

psychologist Dr. Esther Song performed a threat assessment on KP with his

parents present, and he was labeled a “low-risk” threat. The assessment

recommended a follow-up meeting on September 26, 2013, but the follow-up

meeting appears to have been brief and perfunctory. AHS administrators and

counselors did not insist on a records release from KP’s outside therapist prior

to his return, which has been permitted in at least one other school district in

Colorado. In addition, they did not discuss KP’s progress with anger

management counseling. In the months following the shooting, the ACSO

investigation revealed that shortly after his September 9th

threat assessment,

KP began writing a diary, where he described his hate for others, satirized the

ineffectiveness of his medication and therapy, and outlined his plan for an

attack at the high school during finals week in December. The diary appears

to have been created simply to document his ability to avoid detection.


21

Following the “cultural script” created by other school shooters (see Newman,

et al., 2004), KP wanted fact-finders to read the diary, discuss it, and wonder

why.

On December 11, 2013, just two days prior to his December 13th

attack, KP had

an outburst in Spanish class. A classmate locked him out of Victoria

Lombardi’s Spanish class, and KP responded by banging very loudly on the

door, scaring Ms. Lombardi and her students. When KP was let into the

classroom, Ms. Lombardi asked him if he was serious, he replied “serious as a

heart attack,” further startling her (Lombardi Deposition, p. 47). Ms. Lombardi

asked KP to gather his things and leave the classroom. Campus Security

Officer Cameron Rust found him and brought him to the office of Assistant

Principal Kevin Kolasa, who had participated in his September 9th

threat

assessment meeting. KP gave a statement to Mr. Kolasa; they called his

mother and he was sent home for the remainder of the day. KP was not

formally suspended. He returned to school the next day, December 12, 2013

and apologized to Ms. Lombardi for his outburst. On December 13, 2013, KP

entered the school through an unlocked door at the north entrance near the

Trophy Hallway, armed with a shotgun, hunting knife, three Molotov cocktails,

and several rounds of shotgun ammunition. Witnesses’ accounts indicate that

he was looking for Tracy Murphy, but when Claire Davis saw him shooting in

the hallway, she asked him what he was doing, and he shot her in the head.

She fell to the ground. KP then entered the library, apparently looking for

Tracy Murphy, and he stood between two bookcases and shot himself. Claire

succumbed to her injuries eight days later on December 21, 2013.

To understand how similar school shootings might be prevented, the

Arapahoe High School Community Fund Honoring Claire Davis, a donor-

advised fund of The Denver Foundation approached the Center for the Study

and Prevention of Violence (CSPV) at the University of Colorado Boulder to


22

conduct a fact-finding investigation of the events and circumstances leading

up to this tragic event. The purpose was to understand the school’s threat and

risk assessment procedures and responses, the school’s approach to safety

and climate, and the lessons that may be learned from this incident that could

improve youth violence prevention in school settings in the future. We

recognize that we cannot eliminate all youth violence, but the findings and

recommendations shared here present excellent opportunities for reducing

youth violence in school settings.

Prior Research

School Safety

This report builds on the larger research literature on shooting events and

violence prevention in school settings. This literature provides a framework for

understanding and interpreting the events leading up to the Arapahoe High

School shooting. Colorado residents have been witness to several mass

shootings in the last seventeen years, including Columbine High School (1999)

and Aurora Theater (2012). Following the 1999 shooting that left 13 dead at

Columbine High School, Governor Bill Owens organized the Columbine Review

Commission. The Commission’s final report outlined specific recommendations

for both police response and violence prevention in schools in Colorado

(Erickson, 2001). Seven of those recommendations focused on efforts to

promote violence prevention in schools in Colorado, including clear guidelines

for School Resource Officers (SROs), tools and procedures for identifying

concerning students, the development of an anonymous reporting system

(Safe2Tell), the implementation of bullying prevention programs, and the

creation of interagency information sharing agreements.

The Commission suggested three potential models for addressing school

violence: (1) the Safe Communities Safe Schools Model, (2) the John Nicoletti

Model: Violence Goes to School, and (3) the FBI Approach to Threats of


23

School Violence. The Colorado legislature passed HB 00-1119 and SB 00-133 to

facilitate the exchange of information about adolescents across agency

boundaries (e.g., law enforcement, schools, and mental health providers). The

legislature sought to encourage “open communication . . . to assist disruptive

children and to maintain safe schools” §19-1-302(1)(b) C.R.S. In 2000, the

Colorado General Assembly passed a mandate that:

Each board of education shall cooperate and, to the extent possible, develop written agreements with law enforcement officials, the juvenile justice system, and social services, as allowed under state and federal law, to keep each school environment safe. § 22-32-109.1(3) C.R.S.

Law enforcement agencies have made progress in implementing emergency

response protocols to reduce fatalities in school shootings (Elliott, 2009).

However, challenges have emerged in the effort to implement the

Commission’s recommendations on Safe2Tell, Interagency Information Sharing

Agreements and threat assessment procedures. Many districts and schools

have yet to formally draft an Interagency Information Sharing Agreement to

facilitate the exchange of data across agencies on cases of public safety

concern. In addition, districts continue to use threat assessment screening

tools that have not been empirically tested or validated, creating concern

about the quality of the assessment and support process for students in crisis

(Elliott, 2009).

Threat and Risk Assessment

Threat and risk assessment theory and practice has evolved significantly over

the last 17 years since Columbine, starting with the U.S. Secret Service’s series

of reports on threat assessments and continuing with empirical studies of

various threat and risk assessment tools. In the Threat Assessment in Schools:

A Guide to Managing Threatening Situations and to Creating Safe School

Climates, the U.S. Secret Service defined threat assessment as the effort to

“identify, assess, and manage” an individual who may pose a public or school


24

safety concern (Fein, et al. 2002, p. 4). Based on their review of 37 incidents

of school violence in the U.S. between 1974 and 2000, Fein and colleagues

identified ten key findings for threat assessment protocols (Fein, et al. 2002, p.

17; also cited in Exhibit 4: LPS’s Threat Assessment Training PowerPoint):

U.S. Secret Service’s Ten Key Findings for Threat Assessment Protocols

1. School shootings are rarely sudden, impulsive acts

2. Most attackers engaged in some concerning behavior prior to the incident

3. Most attackers had difficulty coping with losses and failures

4. Many attackers felt bullied, persecuted, or injured

5. Most attackers had access to weapons prior to the attack

6. Most shootings were stopped by a non-law enforcement intervention

7. Most attackers did not directly threaten their targets prior to the attack

8. In most cases, others knew about attacker’s idea or plan prior to the attack

9. There is “no accurate or useful profile of students who engage in targeted school violence

10. Other students were often involved in the attack in some way

Source: Fein, et al., 2002


25

Drawing from these ten key findings, the Secret Service (Fein, et al., 2002, p.

55-57) developed six principles and 11 questions for use in threat assessments.

The six principles for threat assessment include:

U.S. Secret Service’s Six Principles for Threat Assessment Protocols

1. Targeted violence is the end result of an understandable, and often times

discernable process of thinking and behavior.

2. Targeted violence stems from an interaction among the individual, the situation, the setting, and the target.

3. An investigative, skeptical, inquisitive mindset is critical to successful threat assessment.

4. Effective threat assessment is based upon facts, rather than on characteristics or “traits.”

5. An “integrated systems approach” should guide threat assessment inquiries and investigations.

6. The central question in threat assessment inquiry or investigation is whether a student poses a threat, not whether the student has made a threat.



26

The Secret Service outlined 11 questions that should be asked during the threat

assessment process, including:

U.S. Secret Service’s 11 Questions for Threat Assessment Protocols

1. What are the student’s motives and goals? 2. Have there been any communications suggesting ideas or intent to

attack? 3. Has the subject shown inappropriate interest in any of the following (e.g.,

school attacks, school attackers, weapons, mass violence events)? 4. Has the student engaged in attack-related behaviors (e.g., developing a

plan, acquiring or practicing with weapons, rehearsing attacks, casing sites and areas)?

5. Does the student have the capacity to carry out an act of targeted

violence?

6. Is the student experiencing hopelessness, desperation, and/or despair?

7. Does the student have a trusting relationship with at least one responsible adult?

8. Does the student see violence as an acceptable or desirable or the only

way to solve problems? 9. Is the student’s conversation and “story” consistent with his or her

actions?

10. Are other people concerns about the student’s potential for violence?

11. What circumstances might affect the likelihood of an attack?


Similarly, after conducting a review of the research on school shootings, Bondu

and Scheithauer (2011) identified seven warning signs and risk factors for

school shooting offenders, including planning the attack, leaking the plan,


27

enjoying violent fantasies (or violent media), displaying narcissistic personality

traits (but not psychotic symptoms), experiencing peer rejection (e.g.,

bullying), experiencing a significant loss (e.g., heartbreak, college non-

admittance), and having a negative school climate (e.g., highly competitive)

(see also, Meloy, et al., 2004; Meloy, et al., 2012; Rappaport & Thomas, 2004).

Bondu and Scheithauer (2011) listed three stages in the path toward a school

shooting (1. biopsychological risk factors, 2. social risk factors, and 3. structural

risk factors); these stages prove helpful in explaining the missed opportunities

to intervene with KP.

Early research suggested that a threat assessment could be completed using

assessment tools to evaluate risk for general violence (see Cornell, 1990;

Gladwell, 2015). General violence prediction tools relied on a community base

rate of violence for comparison. However, in cases where targeted violence

(e.g., assassination, school shooting) is the concern and the focus is on one

group or individual, the base rate of general violence is often too low for

accurate validation. Cornell (1990) found that, compared to juveniles referred

for larceny, juveniles referred for homicide were less likely to have had a

history of mental illness, prior arrest, a juvenile facility placement, or school

adjustment problems (Reddy, et al., 2001). Thus, murderers often have minimal

criminal histories, single targets, and psychopathic tendencies, which tends to

make them more difficult to identify (Cornell, 1990). Therefore, it appears that

both a threat assessment (i.e., focused on a person or group) and risk

assessment (i.e., focused on general violence) are needed in most situations

where a safety concern arises with an individual.

A problem arises in assessment processes when threat and risk assessment indicators are comingled into a “threat assessment.” Recent literature on threat and risk assessment identifies one validated measure for threat assessments and at least two validated measures for risk assessments. It is important to note that threat assessments only get so far in evaluating a student; a threat assessment is enriched with a risk assessment, because a risk assessment can more carefully identify the risk and protective factors that can mitigate or enhance threat.


28

Moreover, risk factors have more evidence of predicting violence for certain types of shooters. Risk assessment tools, such as the Structure Assessment of Violence Risk in Youth (SAVRYTM) and the Risk and Resiliency Check-Up (RRCU), accurately predict violence. The differences between threat and risk assessments are relative, not categorical, as seen in the table below and also Appendix 2 (D. Cornell, personal communication, November 22, 2015).

Differences Between Tools for Risk Assessment and Threat Assessment

Validated Risk Assessment (e.g., SAVRY

5/RRCU

6)

Validated Threat Assessment (e.g., V-STAG)

Purpose

• Identify risk and protective

factors for intervention

• Build a plan to manage the individual based on the identified risks and protective factors

• Respond to threat posed

• Build a plan to mitigate threat (e.g., when boundary probing, threat assessment response is defined and acted upon)

Intended Victim • Not specified, general • Usually identified

Timeframe • Open-ended • Relatively short, unless new

risk or threat factors identified

Intervention Strategy • Mitigation and/or support • Problem resolution

Goal • Accurate Prediction • Prevention

Social Ecology • Not considered • Goal to improve climate

The examples used here, SAVRYTM and RRCU, are validated risk assessment

tools; each indicator within these tools has a quantified and anchored

definitions. These anchors provide tool users with guidance on how to score

each indicator, and they prove critical to the reliable scoring of an individual’s

overall risk. When a risk assessment tool does not provide these anchors or

definitions for the “threat factors,” “early warning sign factors,” and “at-risk

factors,” an unreliable assessment of the student occurs. Without anchors or

proper training, the scoring of these items can be very subjective and

5TheSAVRYisrecommendedherebecauseinacomparisonofnineriskassessmenttoolsitprovidedthebestpredictorof

violence(Singh,2011).6TheRRCUisrecommendedhereasoneexamplefromaclassofriskassessmenttoolsdesignedtopredictfuture

violence.TheRRCUisoneofveryfewinstrumentsthatincludescoringfor“protectivefactors”andistheonlyinstrument

thatcalculatesaresiliencyscore(i.e.,RRCU=riskscore–protectivefactorscore).


29

therefore in a “reliability” test of the threat assessment tool, it is doubtful that

ten people scoring the same item for the same individual would agree. At this

time, there is only one empirically validated threat assessment tool, the Virginia

Student Threat Assessment Guidelines (V-STAG) (Reddy, et al., 2001). Until

such time as alternative threat and risk assessment tools have been validated,

the research strongly supports the use of SAVRYTM, RRCU, and V-STAG, along

with the Secret Service’s threat assessment principles and questions, in threat

assessment processes.

More recent research on school shootings identifies the conditions and

warning signs for a school shooter. In Rampage: The Social Roots of School

Shootings, Katherine Newman and colleagues (2004) relied on an intensive

study of two school shooting cases (from Heath, Kentucky in 1997 and

Westside, Arkansas in 1998) to develop a list of five necessary but not

sufficient conditions for a school shooting. They tested and found support for

these five conditions using three data sets with a total of 74 school shooting

cases. The first condition is that school shooting offenders tend to feel

marginalized in the school or community and could be described as loners.

Second, these offenders frequently have individual vulnerabilities, including

family problems, suicidal ideation, mental illness, and depression. Third,

offenders tend to follow a cultural script, such as discussing the Columbine or

Newtown shootings, expressing a desire for a masculine exit, or indicting a

desire to send a message about an injustice. Fourth, the offenders tend to “fly

under the radar” with a minimal disciplinary record and low achieving

academic record. They may write violent words or texts in assignments. In

fact, two-thirds of the school shooting attackers identified by Newman and

colleagues (2004) had never been in trouble. Finally, access to and

knowledge about guns represented the fifth and final necessary but not

sufficient condition for a school shooting. While not a validated tool,


30

Newman’s (2004) conditions can inform the collection and interpretation of

student information during a threat assessment.

Recently, forensic psychologist Peter Langman (2009) looked at ten shootings

and their shooters to identify three types of shooters: (1) traumatized, (2)

psychotic, and (3) psychopathic. Traumatized shooters suffered abuse, and

they had at least one parent who was a substance abuser and at least one

parent with a criminal history. The psychotic shooters came from intact

families with no abuse or trauma history, but they exhibited symptoms similar

to schizophrenia or schizotypal personality disorder (e.g., paranoid delusions,

delusions of grandeur, and auditory hallucinations). The psychopathic

shooters also came from intact families and had no abuse or trauma history,

but they demonstrated narcissism, sadistic behavior, lack of empathy, and lack

of conscience. However, these typologies are not recommended for use in a

threat assessment because a threat assessment does not diagnose mental

illness and some shooters may follow the cultural scripts from prior shooters

(see Newman, et al., 2004). The FBI’s expert on criminal profiling concluded

that profiling was not an appropriate method for preventing school shootings

(Borum, et al., 2010). However, the literature on these three typologies may be

useful in building an action plan to support a student in crisis, if a diagnosis of

one of these conditions is received. For instance, if traumatized, the student

would need counseling, therapy, support, and protection. If psychotic, the

student’s action plan may require medication and long-term treatment. If

psychopathic, the student’s action plan may require external support and

controls. Thus, this information is beneficial for building a safety and support

plan, but these typologies should not be used for assigning a threat level.

The threat assessment team should use threat and risk assessment tools to

help the Interagency Social Support Team (ISST) build the safety and support


31

plan for the student. In many jurisdictions, the threat assessment team and the

ISST have the same membership.7

Systems Thinking

Doyle (2010) has recently argued that institutions (e.g., criminal justice system,

hospitals, and schools) should develop regular routines for reflecting on major

errors, near misses, and other mistakes in the management of individual cases

(e.g., wrongful conviction, eyewitness misidentification, medical mistake, and

shootings). Evidence from medicine and aviation indicates that errors in case

management do not arise from one person’s bad judgment or one procedural

misstep. Instead, errors in case management arise from a series of smaller

level errors combined with a system’s reluctance for self-examination, and

these smaller errors, along with a reluctance for self-examination, can lead to

major problems or events (Chassin & Becher, 2002). Doyle (2010) notes the

dramatic changes in the ways that aviation and medicine now conduct “error

reviews” of airplane disasters and surgical mistakes (respectively). These

industries have sought to move away from adversarial models of error review

to implement continuous improvement models of error review. This change

requires the promotion of a “culture of safety” and a willingness to evaluate

procedures and practices in a critical manner.

7Forreasonsofconfidentiality,ifateacherorpeerprovidesinformationforanindividual’sstudent’sthreatassessment,

they should only be present for that individual student’s threat assessment, not for subsequent threat assessment

meetings.


32

According to James Reason (1997, cited by Doyle, 2010, p. 137-138), an

organization with a culture of safety:

James Reason’s (1997) Characteristics of a “Culture of Safety”

1. Is informed about current knowledge in its fields

2. Promotes the reporting of errors and near misses

3. Creates an atmosphere of trust in which people are encouraged to report safety-related information

4. Remains flexible in adapting to changing demands (by, for example, shifting from steeply hierarchical modes into “flatter” team-oriented professional structures)

5. Is willing and able to learn about and adjust the functioning of its safety systems

The shift toward a culture of safety in organizations does not come easily, but

it can prove critical to the improvement of safety and service. We believe a

similar approach to “error review” should be taken to study school shooting

events and to improve school safety. The investigators critically reviewed the

tools, procedures, and cultural climate in place at AHS and LPS at the time of

the shooting.

The prevention of school violence is a complex problem that requires an

integrated multi-tiered solution. It’s not just about information sharing, mental

health care, or target hardening. It’s about creating a positive school climate

(i.e., high academic standards, clear rules, fairly enforced discipline, community

and school partnerships, promotion of good citizenship and character) that

encourages information sharing about and supportive responses for students

in crisis (Elliott, 2009).


33

Project Data

Depositions

Depositions (Conducted From June to November 2015)

Name Description

Jeff Corson

Social Studies teacher at Arapahoe High School from 2006 to Present,

who also taught International Relations, Psychology, and Western

Civilization.

James Englert

Deputy with the Arapahoe County Sheriff’s Office and School

Resource Officer assigned to Arapahoe High School from 2007 to

Present, responsible for safety and security of the building and being

accessible to students and the staff.

Guy Grace

Director of Security and Emergency Planning for Littleton Public

Schools from 1999 to Present, responsible for security cameras,

emergency planning, emergency drills, and incident command

systems.

Kevin Kolasa Assistant Principal for Arapahoe High School from 2011 to 2013.

Victoria Lombardi Spanish teacher at Arapahoe High School from 2008 to Present.

Darrell Meredith

Assistant Principal at Arapahoe High School from 2007 to Present,

responsible for 9th grade attendance and behavior, safety and

security, and building maintenance.

Scott Murphy

Superintendent of Littleton Public Schools from 2006 to 2015,

responsible for the operations of the school district, including

operations, security, instructional programs, and facilities.

Tracy Murphy Librarian at Arapahoe High School from 2005 to Present, who also

served as the Debate Coach for three years from 2011 to 2014.

Rodney Mauler Campus Supervisor for Security at Arapahoe High School from 2009

to Present, responsible for campus and building safety.

Natalie Pramenko

Principal at Arapahoe High School from 2012 to Present, who was

responsible for overseeing the training of staff and operations of the

school.

Dr. Esther Song

School psychologist for Arapahoe High School from 2008 to 2014,

responsible for meeting with students to discuss difficulties,

consulting with special education and general education teachers

about students, working with counselors to conduct suicide and

threat assessments, and making recommendations for outside

referrals for students and families.

Nathan

Thompson

Coordinator of Student Support Services for LPS from 2009 to 2013

and Director of Social, Emotional, and Behavior Services for LPS from

2014 to Present. As the Coordinator, Thompson was responsible for

overseeing mental health programming and intensive needs

programming for kids with emotional and behavioral issues in LPS, as

well as the crisis response team for suicide and threat assessments.


34

Supporting Documents

Project Data Supporting Documents

Title Description

Arapahoe County Sheriff’s Office

(ACSO) Report

The ACSO’s investigation produced a 37-page report

compiled by Investigator Kristin McCauley (Investigative

Report, Arapahoe High School, Case # CT13-44545); the

report relied on 27 separate PDF documents that

contained more than 4,000 pages of text in the case.

Littleton Public School (LPS)

Responses

In Answers and Responses to Claimants’ First Set of

Interrogatories and Requests for Production of

Documents, LPS produced a 27-page response and 171

separate PDF documents, which included more than 4,215

pages of text (LPS 00001 – LPS 04215).

Deposition Exhibits

Through the course of the depositions, 64 exhibits were

produced and introduced during testimony. Those

exhibits came from the above-mentioned ACSO Report

and LPS Responses.

Risk Assessment Models

Two risk assessment models in the U.S. provide empirically

tested and validated tools for evaluating the potential risk

an individual presents to the self, others, or the community:

Structure Assessment of Violence Risk in Youth (SAVRYTM

)

and Risk and Resiliency Check Up (RRCU).

Threat Assessment Tools

One empirically validated threat assessment tool provides

a comprehensive and evidenced-based method for

evaluating the threat an individual presents to the self,

others, and the community: Virginia Student Threat

Assessment Guidelines (V-STAG).

Family Educational Rights and

Privacy Act (FERPA)

FERPA (20 U.S.C. § 1232g; 34 CFR Part 99) represents

federal legislation that seeks to protect the privacy of

students’ education records.

Biography of the Shooter

This biography only covers the aspects of KP that were observed by the

witnesses deposed in this process and interviewed by the Sheriff’s Office; thus,

it primarily focuses on his behaviors and attitudes while at AHS. It does not

reflect every aspect of KP, who was loved by his family and friends.

In their study of school shooters, Newman and colleagues (2004) concluded

that school shooters often share similar characteristics and behaviors, but they

are never identical to each other. KP had some similarities to other school

shooters, and the evidence suggests that he had some knowledge of the facts

surrounding other school shootings (e.g., Sandy Hook, Columbine).


35

Ten years prior to the shooting, KP exhibited troubling behavior in elementary

school, when he hit two students with his lunchbox in November 2003 and he

kicked another student in the stomach and hit another student in the head in

December 2003 (see Exhibit 24 and Appendix 1).

At AHS, KP was an intelligent and achieving student, and after his junior year,

he had a GPA just below a 3.0 (ASCO, p. 202) and ACT scores in the 68th

percentile (ASCO, p. 248). It was in his sophomore year that KP began to

exhibit more concerning behaviors. In November 2011, he told a classmate to

“just go gut himself” (see Exhibit 19). International Studies teacher Jeff Corson

believed that KP had a “need for attention [and] the need to be perceived as

smart. . . I could tell he needed that approval” (Corson Deposition, p. 47).

Several people described KP as awkward and outspoken, and some

considered him strange, arrogant, and even narcissistic. AHS Debate Coach

Tracy Murphy described KP as “the stereotypical . . . nerdy kind of kid, lacking

self-confidence” (Tracy Murphy Deposition, p. 13). He was also fascinated by

politics. Corson attributed KP’s success in his International Studies class to his

interest in the topic and opportunities for class discussion. KP liked to provoke

others with controversial or uncomfortable images or comments. For

example, the image on his computer background was a swastika (ASCO, p.

1417), and he often wore a red t-shirt with a yellow hammer and sickle

emblemizing the USSR to suggest he was a communist. Some peers

suggested, however, that his true political ideology did not match the shirt

(ASCO, p. 1962). Instead, he may have used controversial t-shirts, symbols,

and comments to illicit reactions from others.

The ACSO investigation reflected two different opinions of KP’s popularity at

AHS. Some peers suggested that KP had friends and was well known at AHS.


36

In fact, one student told an ACSO investigator that KP belonged to an

exclusive clique (ASCO, p. 1974). However, other students described him as

having anger problems and as only feeling accepted by the debate team

(ASCO, p. 1688). Some evidence suggests that KP felt bullied by his peers,

while other evidence suggests that he bullied peers. For instance, in November

of 2011, KP justified an incident where he told a fellow classmate to “go gut

[himself]” by saying, “why wouldn’t I make him my bitch after [all] that has

been done to me” (ASCO, p. 198). The investigation never revealed what, if

anything, had actually been done to him.

Seven of the ten key findings for threat assessment protocols identified in the

U.S. Secret Service’s report on threat assessment protocols (Fein, et al. 2002,

p. 17) appeared in KP’s case and are marked below with a u.

U.S. Secret Service’s Ten Key Findings That Appeared in KP’s Case

1. School shootings are rarely sudden, impulsive acts u

2. Most attackers engaged in some concerning behavior

prior to the incident u

3. Most attackers had difficulty coping with losses and

failures u

4. Many attackers felt bullied, persecuted, or injured u

5. Most attackers had access to weapons prior to the

attack u

6. Most shootings were stopped by a non-law

enforcement intervention u

7. Most attackers did not directly threaten their targets

prior to the attack u

8. In most cases, others knew about attacker’s idea or

plan prior to the attack

9. There is “no accurate or useful profile of students

who engage in targeted school violence”

10. Other students were often involved in the attack in

some way

PART 2: FINDINGS


37

Major Finding 1: Information Sharing

This chapter describes the failures in information sharing among Arapahoe

High School staff and even students prior to the December 13, 2013 shooting.

Research on school shootings consistently finds that sharing of information

about students of concern remains key to the promotion of a positive school

climate and the promotion of school safety (see Erickson, 2001; Fein, et al.,

2002; Pollack, et al., 2008; Vossekuil, et al., 2002). The Secret Service found

that someone knew about the shooter’s intention prior to the attack in 81% of

the school shootings that occurred between 1970 and 2000 in the U.S.

(Pollack, et al., 2008). As with Columbine in 1999, a strong code of silence was

in place at AHS at the time of the shooting, among administrators, counselors,

security personnel, and students.

The deposition testimony, ACSO Report, and LPS interrogatory responses

revealed numerous instances when AHS administrators did not document,

share or act on information about KP’s concerning behaviors in the months

leading up to the shooting. In addition, when two campus security officers and

the assistant principal responsible for safety learned that KP had been viewing

guns on his computer in the school cafeteria just one month after his threat

assessment, KP was not contacted or questioned about the matter, and no

effort was made to re-evaluate the threat he posed to himself or others. In

addition, KP’s interest in and experience with guns was not documented in

KP’s files or addressed by the school psychologist.

Students also failed to share information about KP’s interest in guns and

problems with anger in the months leading up to the shooting. According to

the Secret Service, students or peers were most frequently the individuals who

knew the plan for the shooting prior to the event (Vossekuil, et al., 2002;

Pollack, et al., 2008). In fact, 14 years ago, the Columbine Commission


38

recommended that schools work to change the “code of silence” prevalent

within student culture (Erickson, 2001).

ACSO investigators interviewed 14 of KP’s peers (see ACSO Report, pp. 11-12).

Their testimony revealed that prior to the shooting, nine of them knew that he

had an anger problem and six of them knew he had a gun; two of the 14 peers

interviewed knew that KP had both an anger problem and a gun. None of

these students contacted Safe2Tell about KP’s possession of a gun and

machete. According to the ACSO investigation, one peer told Dr. Esther Song,

the AHS school psychologist on December 12, 2013, the day before the

shooting, that KP owned a gun (see ACSO Report, p. 1784-1785). However, Dr.

Song denied recalling that conversation and no follow-up action was taken

(Song Deposition, p. 200).

AHS administrators did not use the information sharing tools they had at their

disposal, including Infinite Campus and Safe2Tell. In addition, they failed to

develop and complete an Interagency Information Sharing Agreement, as

authorized under Colorado law (§ 22-32-109.1(3) C.R.S.). As a result, AHS

administrators had no way of building an information vortex (or coordinator)

to catalog the concerns that campus security officers, teachers, KP’s mother,

and a student shared about the threat KP posed. School Resource Officer

(SRO) James Englert said:

[I wish] we had more information given to us about students. . . like an information vortex. . . where everything [is] brought together and where law enforcement [is] involved, the therapist outside of the school [is] involved. . . [T]he information needs to be shared with everybody. Everybody needs to be brought in, and it’s frustrating for me. . . [T]he school is concerned about a certain kid, but they are holding back [on sharing information] because of fears. (Englert Deposition, p. 136)

Evidence suggests that AHS administrators feared negative publicity and

FERPA violations. FERPA refers to the Federal Education Rights and Privacy

Act, which protects students’ right to privacy. The Act and Colorado law


39

“afford parents, guardians, or legal custodians (“parents”) and students over 18

years of age (“eligible students”) certain rights with respect to the student’s

education records” (LPS, p. 1118). FERPA guidelines, however, stipulate that

school staff can document and disclose information about disciplinary actions

taken with a student for behavior that presents a safety concern for the

student, other students, or the community (see CCCOES Legal Summit). In

testimony provided by Dale King, the Director of the Family Policy Compliance

Office in the U.S. Department of Education, not only do faculty, staff and the

SRO have a right to share of information about a student, but FERPA

guidelines are clear that the Department of Education will not substitute its

own judgment for a school’s decision to share information about a student, as

long as there is a reasonable basis8 for that sharing of information. In LPS’s

Threat Assessment Training presentation (Spring 2011), Nate Thompson stated:

FERPA (Family Educational Rights and Privacy Act) – allows schools to convey disciplinary information regarding significant risk to those with a legitimate interest (see Exhibit 4, p. LPS 00480).

Yet several AHS staff, teachers, and the SRO indicated that they could not

discuss a student’s concerning behaviors with other teachers or staff prior to

the shooting because AHS administrators had told them that FERPA

guidelines prohibited it. Spanish teacher Victoria Lombardi and SRO James

Englert explained:

[B]efore the shooting, if I had an issue with a student, I couldn’t go to another teacher and say, ‘Hey, do you see the same behavior, because this is concerning me.’ (Lombardi Deposition, p. 23) The biggest obstacle [to keeping AHS safe] is just information sharing. The school is somewhat confused on what FERPA is. (Englert Deposition, p. 137)

8AnthonyB.Dyl,SeniorAssistantAttorneyGeneralandAssistantSolicitorGeneral,ColoradoDepartmentofLaw,Officeof

theAttorneyGeneral.2015.PersonalE-mailCommunicationtoWilliamWoodward,October21,2015.“Areasonablebasis

isdefinedasanyconceivablereasonfordoingwhatyoudid...Anactionmeetsthe“rationalbasis”testwherethereisany

reasonably conceivable state of facts that could provide a rational basis for the challenged action” [See] Qwest

Corporationv.ColoradoDivisionofPropertyTaxation,DepartmentofLocalAffairs,StateofColorado,304P.3d217(Colo.

2013).


40

Indeed, FERPA appears to have been widely misinterpreted by AHS

administrators prior to the shooting, leading them to discourage teachers, the

SRO, and others from discussing a student’s behavior problems or discipline

record. In a statement to an ACSO investigator, math teacher Michelle

Crookham said:

[T]he AHS administration will not tell the teachers anything about student discipline, as it is a violation of the student’s privacy rights. (Exhibit 16)

AHS’s practice of not sharing information about a student’s disciplinary issues

with teachers or staff contradicted the LPS’s Student Code of Conduct policies

for 2013-2014 and 2012-2013, which stated:

In accordance with state law, the principal or designee is required to communicate disciplinary information concerning any student enrolled in the school to any teacher who has direct contact with the student in the classroom and to any counselor who has direct contact with the student. The purpose of this requirement is to keep school personnel apprised of situations that could pose a risk to the safety and welfare of others. (see LPS 01085 for 2013-14 Student Code of Conduct and LPS 01014 for 2012-13 Student Code of Conduct)

In fact, under the section “Regulation for Board Policy JK” LPS’s Student Code

of Conduct stipulated that:

To assure that information is shared with the professional staff that may be important to understanding the particular needs of individual students and any potential risk that a student might pose to the safety or welfare of others, state law requires that the principal take steps to communicate this information to teachers and counselors who have direct contact with the student (LPS 01086 and LPS 01015).

Yet the principal, assistant principals, counselors and teachers declined to

share information about the risks KP posed to himself and others when he was

suspended in March 2013 and after his threat assessment in September 2013.

In response to the question, “[D]o you believe that there were impediments or

obstacles to the effective flow of information at Arapahoe High School about

KP [prior to the shooting]?” School psychologist Dr. Esther Song said, “Yeah”

(Song Deposition, p. 210).


41

At the time of the shooting, AHS administrators did not notify teachers, the

SRO, or other staff about the conduct violations leading to a student’s

suspension, even when that violation occurred in the teacher’s class or when

the violation involved a safety risk for the student or others (see Lombardi

Deposition, p. 17; Corson Deposition, p. 44). Jeff Corson explained:

[U]sually what happens is I’ll get an email with a form I can print out, and it will say, “This student has been suspended.” It will not say why. (Corson Deposition, p. 44)

When an AHS student received a suspension, the student’s teachers received a

note from the assistant principal describing the dates of suspension and

requesting the homework assignments for the student. Some teachers viewed

these restrictions on information sharing as problematic, because they worked

more closely with students than administrators and could better identify a

conduct relapse or safety risk than administrators. Spanish teacher Victoria

Lombardi said:

I think it takes all of us to keep the school safe . . . and information is important and communication is important. . . I think there should be a way that we know [about] every student in trouble in that school. (Lombardi Deposition, pp. 19, 85)

Even in October 2015, more than 18 months after the shooting, school

psychologist Dr. Esther Song said:

I don’t know what information can be relayed about each student to general staff and teachers, because I think that there has to be some protection of confidentiality to protect that student’s rights. (Song Deposition, p. 211)

AHS administrators’ concerns about potential violation of FERPA guidelines

are unfounded, as no school or district has ever been financially penalized for a

FERPA violation. “In Gonzaga University v. Doe, 536 U.S. 273 (2002), the U.S.

Supreme Court made clear that students and parents have no private right of

action against schools for unauthorized disclosure of education records.

Schools cannot be held liable for damages for improper disclosure of student

information” (Michael Roche Testimony, October 27, 2015). The court stated,


42

“FERPA’s nondisclosure provisions fail to confer enforceable rights” on

students or parents (Gonzaga University vs. Doe, 2002). [In addition,] “[T]he

defunding mechanism has never been used [to penalize a school or district]

under FERPA” (Michael Roche Testimony, October 27, 2015). During a press

conference for ACSO’s Final Report, LPS Superintendent Scott Murphy refused

to answer questions about KP, inappropriately citing FERPA restricts, which

do not apply to deceased students (The Denver Post, October 18, 2014). As

with other school shootings, like at Virginia Tech University and Columbine

High School, school officials failed to share information in the weeks and

months before KP shot and killed Claire (Virginia Tech Review Panel, 2007;

Erickson, 2001). LPS’s policy on FERPA reflects a misunderstanding of state

and federal FERPA guidelines, which led to failures in information sharing in

KP’s case and thus may have facilitated Claire’s death. When asked “[D]o you

know whether LPS had a specific policy or practice during your time leading

the district on the sharing of information among schools and staff members

about dangerous conditions or people?” Superintendent Scott Murphy

(Deposition, p. 67) said:

I don’t recall one off hand. My guess is there was at some time, but it may have been conversational or training consultation.

These widespread failures in information sharing and documentation meant

that not one AHS administrator, teacher, or counselor knew KP’s complete

history of behavioral problems, threat assessment results, or weapons history.

At least three teachers and two campus security officers expressed concern

about or described problems with KP in the three months prior to the

shooting, but not one teacher, administrator or counselor had all of that

information and not one individual appears to have accepted responsibility for

monitoring KP’s progress or decline following his September 9, 2013 threat

assessment. Tracy Murphy said:

I was working in isolation in the fall of 2013 when all of this was happening [with KP]. And it appears that Vicki Lombardi was having problems with Karl and that Dan Swomley had had some previous


43

problems and Michelle Crookham had some issues. None of us knew this, none of the faculty that had direct interaction with the student was aware . . . I wasn’t aware of the problems that Dan had had, that Michelle had had, that Vicki had had. . . I value student privacy, but I also value student safety. (Tracy Murphy Deposition, p. 218)

Thompson said that:

I agree that we need a central point of contact [for information sharing at each school]. [Up to now] we have made the decision not to push the issue of making one person a community-wide contact for each school . . . We try to use the natural systems that [schools] have. (Thompson Deposition, p. 168)

The problem with the “natural systems” approach is that not one administrator

(at AHS or other LPS campuses) takes responsibility for being the information

vortex that identifies, evaluates, and monitors a student of concern. Darrell

Meredith, the Assistant Principal in charge of AHS’s Safety and Security, said,

“It’s a shared responsibility.” Unfortunately, shared responsibilities meant that

no one was responsible, leading to information failures, as no one is or can be

held accountable.

When asked “[H]ow is it made known community-wide that if there is a

concern about student X, this is where you go with it?” Nathan Thompson,

the Coordinator of Student Support Services for LPS, said:

[T]hat’s not clear at this point in most of our schools. There is not a designated one person [for reporting safety concerns]. (Thompson Deposition, p. 168-169)

Indeed, in deposition testimony, AHS and LPS administrators frequently

provided “shared responsibility” and “expectations” as the reason why one

specific AHS administrator did not serve as a clearinghouse (or vortex) for

information about students of concerns or students in crisis in general and

why one specific AHS administrator was not responsible for following up with

KP on his threat assessment action plan in particular. After the September 26,

2013 threat assessment follow-up meeting with KP, not one AHS assistant


44

principal, teacher, or counselor asked KP, his mother, or his father about KP’s

progress with anger management, therapy, or academic performance. Not

one AHS staff person sought him out or asked him how he was doing. When

asked if he, Ester Song, and the other counselors at AHS “were, in a sense,

waiting for Karl to come to you if he had issues?” Kevin Kolasa said, “Yes”

(Kolasa Deposition, p. 149).

Finding 1a: Infinite Campus

Infinite Campus, a comprehensive student information database system, serves

as a portal for recording and sharing information about students, and AHS

used Infinite Campus prior to and after the shooting. The system allows school

administrators, teachers, and staff to document a student’s academic

performance and behavior concerns, monitor student academic and behavioral

improvements or declines, and evaluate overall patterns in student

performance and behavior. According to LPS testimony, AHS school

counselors and psychologists used the Contact Log feature within Infinite

Campus to document student behavior concerns, class schedule changes and

staff-parent communications. AHS administrators, primarily assistant

principals, used the Behavioral Detail Report in Infinite Campus to document

student conduct violations, punishments and staff-parent communications.

The findings revealed two serious problems with the way AHS administrators,

counselors, and psychologists used Infinite Campus as a tool for sharing

information prior to the shooting: (1) inconsistency in use and (2) restrictions in

access. First, Infinite Campus was not consistently used to document student

behavior concerns at AHS. Deposed witnesses, as well as AHS files, indicate

that KP exhibited problematic behavior that caused moderate and eventually

serious concern prior to the shooting, but the information about his behavior

was not consistently recorded in AHS’s hardcopy files or the Infinite Campus

database. LPS’s interrogatory responses listed 14 instances where KP’s


45

behavior raised concerns but only five of those instances appeared in Infinite

Campus. Nine of the 14 instances – including KP’s suspension in March 2013

and KP’s viewing of guns on his computer after his threat assessment in

October 2013 – were not documented in Infinite Campus’s Contact Log or

Behavioral Detail Report (see LPS’s Answers and Responses; Exhibits 19 and

24). Kevin Kolasa’s deposition (p. 37) revealed:

Mr. Michael Roche: Karl’s suspension in March of 2013 is not listed in his behavioral detail report, is it? Mr. Kevin Kolasa: It’s not. Roche: Whose job was it to input the information about that suspension into Karl’s behavioral detail log? Kolasa: That was my responsibility. Roche: And why didn’t that happen? Kolasa: I just forgot. I didn’t do it.

When asked, “When you got this news from Cameron Rust and Christina Kolk

that they thought Karl was looking at guns on his computer [in the cafeteria],

did you tell either Kevin Kolasa or Esther Song [who had performed the threat

assessment] about it so that they could decide whether to follow-up on it?”

Assistant Principal Darrell Meredith said:

I don’t think I did. (Meredith Deposition, p. 172)

In addition, referral forms (i.e., the three-page behavior misconduct reports,

which are completed by teachers with one copy getting sent to the assistant

principal and another copy getting sent home to parents) were not recorded

in Infinite Campus prior to the shooting (Lombardi Deposition, p. 22). Spanish

teacher Vicki Lombardi reported that referral form information is still not

noted in Infinite Campus (Lombardi Deposition, p. 22).

In September 2013 of his senior year, KP’s problematic behavior began to raise

more serious concerns, particularly among his International Studies teacher


46

Jeff Corson, Debate Coach Tracy Murphy, and Spanish teacher Victoria

Lombardi. KP started to disregard responsibilities, fail classes, verbally bully

classmates, and display anger.

In an effort to hold KP accountable for his irresponsible behavior, Debate

Coach Tracy Murphy (Murphy Deposition, pp. 72-74) made the decision to

remove KP from the position of Extemporaneous Team Captain of the Speech

and Debate Team at AHS on September 3, 2013. To relay the news, he

scheduled a one-on-one meeting with KP and his mother, Barbara Pierson.

Murphy described:

I was very concerned [about his behavior] and [I explained to KP and his mother] that I felt that the best thing to do at that point was to keep him on the team but demote him. . . I wasn’t sure what his response was going to be. . . but he blew up. . . He started screaming at me. . . He couldn’t believe it, he’s yelling at me, “What would [Principal] Pramenko think about me demoting the only member of the team that made nationals?” and. . . He wasn’t seeing that there was more to being a leader on the team than his own personal success.

Murphy continued (Murphy Deposition, p. 79): [In 28 years of being an educator] I’ve never had an interaction with a student where a student has interacted with me that way and looked at me that way. It was chilling.

Following that meeting, KP and his mother walked out to the school parking

lot, and KP yelled, “I’m going to kill that guy [Murphy]” (Tracy Murphy

Deposition, pp. 103-14). The outburst was overheard and seen by KP’s teacher

Mark Loptien and reported to Kevin Kolasa, which he documented in the

Behavioral Detail Report of Infinite Campus (see Exhibit 24). It is important to

note that while this incident was noted in Infinite Campus, only AHS

administrators and counselors could see the report; teachers and the SRO did

not have access to conduct information in Infinite Campus prior to the

shooting, and even today, the SRO does not have access to conduct

information in Infinite Campus.


47

Thus, in November 2013, when Spanish teacher Victoria Lombardi began to

have concerns about KP’s grades in her class, she did not know about KP’s

history of anger management problems or his threat to Tracy Murphy

(Lombardi Deposition, pp. 55). In addition, she did not know that KP had

undergone a threat assessment in September 2013. She only knew that his

Spanish grades were declining and he had made inappropriate remarks about

drinking tequila in her class. Even when he had an enraged outburst in her

classroom on December 11, 2013, Lombardi was not told about KP’s threat

assessment. Lombardi described what happened when KP got locked out of

her classroom on December 11, 2013 (Lombardi Deposition, pp. 46-47):

I was up in front of the room, the bell had rang, maybe a couple minutes before that, and all of a sudden there was this loud banging on the door that was so loud on the glass it really scared me. It scared my whole class. It was inappropriate banging. . . Someone let him in. I was still at the front, and he came in and he screamed, “You locked me out.” And I said, “Karl. . . no one locked you out, but your response is inappropriate” or something like that. And he said it again, screaming at me. And I said, “Are you serious Karl?”. . . And he stared at me and said, “As a heart attack.”

This incident was noted in Infinite Campus (Exhibit 24), but it was not

conveyed to Dr. Esther Song or SRO James Englert (Englert Deposition, p. 92;

Song Deposition, p. 196). No one followed-up on it, and KP was not

disciplined. This incident should have been a red flag for further follow-up,

having come after a threat assessment. As suggested in the U.S. Secret

Service’s (Fein, et al., 2002) findings, most school shooters exhibit concerning

behavior prior to the attack and most school shooting events are not sudden

or impulsive acts. KP was no different.

In summary, information about KP was not consistently maintained in hard-

copy files or AHS’s Infinite Campus database. Not one AHS teacher,

administrator or staff person at AHS had a complete record of KP’s history of

concerning behaviors over his more than three years at AHS, making it

challenging to adequately assess the threat he presented. If AHS staff had


48

consistently documented his behaviors, a pattern of “boundary testing” would

have been more apparent. The benefit of using Infinite Campus to consistently

document student concerns and crises is that Katherine Newman’s (2004) five

necessary but not sufficient conditions for a school shooter would have been

easier to identify (i.e., student feels marginalized; student has family problems

or history of depression history; student has interest in Columbine, Newtown

or a masculine exit; student has minimal disciplinary record or a low achieving

academic record; and student has access to or interest in guns).

The problematic behaviors that KP exhibited that could have been

documented in Infinite Campus included:

KP Behaviors That Could Have Been Documented In Infinite Campus

Spring 2013: Opened with statement “I woke up this morning and realized my

penis had fallen off” in debate competition.

August 21, 2013: Told a classmate “that’s stupid” and “verbally bullied”

classmates in Jeff Corson’s class.

September 10, 2013: Disregarded Kevin Kolasa’s request that he not attend

speech and debate practices following his threat to “kill that guy [Murphy]”

and asked to leave practice by Murphy.

September --, 2013: Received “F” on Michelle Crookham’s math test and

wrote “KMFDM” on top of test referring to German band “No Pity for the

Majority.”

October 2013: Observed by Christina Kolk and Cameron Rust viewing

pictures of guns and the Newtown shooting in the cafeteria.

November 1, 2013: Asked Vicki Lombardi “when can we drink tequila” in

Spanish class.

December 12, 2013: Told peers about his new shotgun “Kurt Cobain.”

The second problem with Infinite Campus related to AHS staff members’

access. Prior to the shooting, teachers, SROs, and campus security officers did

not have access to the Contact Log or Behavioral Detail Report sections of the

Infinite Campus database. AHS teachers could not: (1) document concerns


49

about a student’s behavior in Infinite Campus or (2) review a student’s history

of behavior concerns in Infinite Campus (Lombardi Deposition, p. 21-22).

Lombardi said:

Before the shooting, we didn’t have any way to put notes in Infinite Campus about a student. (Lombardi, p. 22)

After the shooting, teachers – but not SROs or campus security officers – were

given a “faculty tab” in Infinite Campus so that they could record information

on academic concerns, behavior concerns, and parent contacts; they can now

also view another teacher’s notes on a student in Infinite Campus. The SRO

James Englert was not permitted access to Infinite Campus prior to the

shooting, and at the time of his deposition in July 2015, SRO Englert reported:

We don’t have access to [Infinite Campus]. We’ve gone to the school to ask for them to give us access to the student information sheets [and Infinite Campus], and the school has not given that to us, law enforcement. (Englert, p. 133-134)

A few months later, SRO Englert reported:

AHS has given me access to the demographic information on students in Infinite Campus, such as their home address and age, but I still cannot access information on a student’s behavioral concerns. (Englert, AHS Campus Tour, October 13, 2015)

There is a challenge in documenting every student infraction, but certainly

major incidents (e.g., suspensions, hate language, recommendations for anger

management) should be documented in Infinite Campus. This issue is not

about detailing every single student encounter or student behavior concern;

the issue is documenting most of them and sharing that information about

students in crisis with other school staff. Infinite Campus should serve as the

effective intelligence gathering system (e.g., identify and address early

warning signs and assess school climate) that facilitates a culture of

information sharing to promote safety, mental health, and violence prevention.


50

Finding 1b: Safe2Tell

Safe2Tell is an anonymous reporting system created in Colorado following the

Columbine Review Commission’s Report (Erickson, 2001), and it provides a

safe and easy way for students, school staff, parents, and community members

to anonymously report information about a safety concern or a student in

crisis to law enforcement officials using a toll free line (1-877-542-7233), a web

reporting feature (www.Safe2Tell.org), or two-way dialogue texting. Early

information about concerning or suspicious behavior is key to violence

prevention (Elliott & Kingston, 2013). As of 2013, Safe2Tell had prevented 28

planned school attacks by responding to more than 700 threats of violence

since 2004 (Elliott & Kingston, 2013). Safe2Tell trainers find that students and

school staff need to receive hands-on training in the system and the reasons

for using the system to fully understand the role it can play in prevention and

intervention.

Newman and colleagues (2004) found that students can find it challenging to

“rat” on a peer, due to an unspoken “code of silence.” Schools, however, can

build a culture and train students that reporting safety information is a duty.

Beverly Kingston has identified five ways to make it safe to tell:

Five Ways to Make it Safe to Tell

1. Teach students that reporting anything related to the safety of anyone is their responsibility.

2. Teach students how to tell a trusted adult that they have a safety concern and teach them to keep telling until someone takes action.

3. Create a climate in which students feel comfortable sharing sensitive information regarding a potentially threatening situation (e.g., social emotional connections and mutually respectful relationships).

4. Train adults on how to properly respond to students who provide information about a threatening situation.

5. Provide an anonymous way to report information safely.

Source: Kingston, 2013


51

At the time of the shooting, LPS and AHS administrators did not have a policy

regarding Safe2Tell training and did not require that students or staff receive

training on the Safe2Tell system. In fact, the information shared about

Safe2Tell at AHS was limited to a sticker on the back of student identification

cards and a PowerPoint slide displayed in the cafeteria. In addition, Safe2Tell

was not listed in the AHS Student Planner and Handbook or the AHS Faculty

Handbook at the time of the shooting (see Meredith Deposition, pp. 59-60;

LPS 01654 and 01668; Table 2). Even after the shooting, LPS did not

implement a policy on training high school students in Safe2Tell (see Grace

Deposition, p. 46, 159).

In response to several AHS students’ suicides, the 2013-14 AHS Student

Planner and Handbook listed information on a Teen Suicide Hotline (LPS

01654). Under the Student Safety Precautions section of the handbook, AHS

administrators advised:

Students are to report to the Attendance Office, or any available adult in the building, any unusual activity or questionable strangers on the campus or in the school vicinity. (LPS 01668)

The safety precautions section reflects a limited view of safety and crises,

limiting students’ understanding and awareness of danger and crisis. By 2014-

15, AHS administrators began listing the Safe2Tell phone number and website

in the AHS Student Handbook in a section on suicide prevention, but the listing

does not indicate what Safe2Tell is or how it can and should be used to

anonymously report concerning behaviors (see LPS 01692-01694). Deposition

testimony from LPS Director of Security and Safety Guy Grace revealed that

LPS’s training of middle school students on Safe2Tell has generated numerous

Safe2Tell reports. However, Grace did not train Safe2Tell at AHS, because it is

left up to the school’s discretion. When asked, “Now, you talked about

sometimes you're invited to the schools to talk about Safe2Tell; right?” He

said: “Um-hum.” He was asked, “I take it that's just sort of at the discretion of


52

the school administrators at that particular school?” He explained, “It's at their

discretion, yes, it is.” He said:

I have never been brought to the [Arapahoe High] school to present there [on Safe2Tell]. (Grace Deposition, p. 48)

The deposition testimony and LPS interrogatory responses revealed that

students, teachers, and administrative staff were not regularly informed of or

trained in Safe2Tell. Several AHS administrators, teachers, and the SRO

reported being unclear about the extent of students’ knowledge of or the

training procedures when asked “[W]hat in the 2013 time period did Arapahoe

do to train the students on the Safe2Tell Program?” Darrell Meredith, the

Assistant Principal responsible for school safety, said (Meredith Deposition, pp.

58-59):

I don’t specifically remember anything. . . except for mentioning it in class meetings at the very first day of school and then posters around the school. . . [a sticker] may have been on the back of the student ID [in 2013-14]. . . [and it was in] daily announcements [that] were always in the cafeteria projected on a large screen . . . Safe2Tell was a slide [in a PowerPoint type presentation]. (see also Mauler Deposition, p. 54)

When asked “What kind of training do students at Arapahoe receive on

Safe2Tell?” Principal Natalie Pramenko said (Pramenko Deposition, p. 45-46):

I wouldn't say it's explicit training on Safe2Tell, but we talk about it. . . At the very beginning of the school year, we have all-class meetings with each grade level separately on the very first day of school and we talk very much about how we appreciate that they're [our] eyes and ears on our campus. . . And we've added the number [for] Safe2Tell on the back of their student ID. It's also linked to all of the district websites, including Arapahoe High School's. . . [And] we have over 15 of those [Safe2Tell posters] posted up around our school.

Other deposition testimony revealed that there was some confusion between

LPS administrators and AHS staff about who was responsible for training

students on Safe2Tell. Guy Grace, Director of Security for LPS, said:

[S]chool Resource Officers. . . talk about the Safe2Tell [program] at their schools and use the same materials they’re provided by the Safe2Tell website. . . [but] it’s at [the school administration’s] discretion. . .


53

Typically, SROs are the ones that are responsible for [Safe2Tell] training. . . There is nothing from the district [providing a formal policy on Safe2Tell training]. (Grace Deposition, pp. 47-48)

AHS’s SRO James Englert, however, did not describe Safe2Tell as one of his

responsibilities (see Englert Deposition, pp. 9-16), and when asked about the

program he said:

Safe2Tell is a good program. We get a lot [of tips]; we probably get two a week maybe. And since the shooting, maybe three or four a week. . . It’s a good program, but. . . it does go back to training the kids on what to look for [and] what to report to us. (Englert Deposition, p. 141)

Dr. Esther Song said:

I know the [Safe2Tell] information was relayed [to AHS students in 2013]. I just can’t remember how. I don’t know if it was in a freshman seminar or if I’m confusing the – maybe it was health. But I know that it was given to students by – pretty sure by counselors, but that’s something that I feel like every year was told to students. (Song Deposition, p. 60)

When asked, “In the time period prior to December of 2013, what was LPS’s

policy on training students about the existence of Safe2Tell and how to use

it?” Nathan Thompson (Deposition, p. 40-41) said:

Well, we didn't have a policy at that time. I don't believe we had a policy.

No one at AHS or in LPS knew who was responsible for Safe2Tell training in

LPS high schools, and we could find no evidence that AHS students were

trained in Safe2Tell. The evidence indicates that AHS made some efforts to

inform students. Some LPS administrators assumed that AHS students would

know when and how to call Safe2Tell just from the posters, student ID stickers,

and projector slides. During a tour of the AHS campus on October 13, 2015, a

few 8.5 x 11 posters advertising Safe2Tell were evident. It is important to note

that other school districts in Colorado, including Jefferson County, regularly

provide students and staff with training in Safe2Tell. Training, however, does

not mean a PowerPoint slide in the cafeteria or a sticker on the back of


54

student identification cards. It means formal training modules, including one-

on-one coaching, role-playing activities, and scenario exercises.

The ACSO report and Safe2Tell data reveal that AHS students were not well

trained in the Safe2Tell system. After the shooting, calls to Safe2Tell about

AHS students increased from four in 2012-13 to 15 in 2013-14 (or more than

tripled). In 2014-15, Safe2Tell calls on AHS students increased to 24. Below

are examples of things that students or others shared with ACSO investigators

after the shooting that could have been reported to Safe2Tell and could have

made a difference in identifying KP as in need of support and follow-up

(Exhibit 14: ACSO Report, pp. 10-12):

Concerns About KP That Could Have Been Reported to Safe2Tell

[Student] was not in school on Friday, December 13, 2013. . . [and reported that] KP did have

anger problems. KP did show him a picture of a gun he bought. KP told him that he (KP) named his gun “Kurt Cobain.”

[Student said] KP showed him a picture of his gun and a machete he bought.

[Student said] KP told him that he (KP) was going to buy a shotgun. KP also told him about the

machete and showed him a picture of it on Thursday, December 12, 2013.

[Student said] KP began to withdraw about three to four months ago. In hindsight, he believed KP may have been thinking about conducting a school shooting before it occurred. He and KP

had a conversation about school shootings and he told KP it was unlikely to occur . . . KP showed

him a picture of a shotgun.

[Student said] KP told him that he (KP) should kill Tracy. He did not take KP’s threat seriously.

KP was quick to anger. He was aware that KP purchased his shotgun from Cabela’s around Saturday, December 7, 2013.

[Student said] KP was an aggressive, outspoken, atheist, liberal, impulsive, self-serving narcissist.

KP threated to burn down --- church because --- is Catholic. KP would often “blow a gasket.” He

heard KP say, “I’m going to kill Mr. Murphy; he is now on my list.” He heard KP call the list “the hit list.”

[Student said] one time KP said he should kill Tracy for kicking him off the team. KP “had it in” for

Tracy.

[Student said] KP told her that he becomes a monster when he is mad.

[Student] was in Spanish class with KP. . . KP was very angry about being locked out. KP scares

him. . .[in text exchanges he said] KP was “honestly scary, like he is going to hurt us, I’m a little nervous. He obviously has the potential to be a threat if little stuff like that makes him crazy.”

[Student said] KP had anger issues. KP was going to snap one day, but she didn’t think it would

be of this magnitude.

[Student said] KP seemed to become angry after his parents’ divorce. . . She thought KP had

some kind of mental health problem.


55

According to the ACSO Report (see ACSO pp. 1784-1785), one of these

students told the school psychologist, Dr. Esther Song, that KP had a gun he

called “Kurt Cobain” on December 12, 2013, but Dr. Song did not recall that

statement (Song Deposition, p. 200).

More than 18 months after the shooting in July 2015, Guy Grace, LPS’s Director

of Security, described LPS’s work to formalize the training of staff and

students in LPS on Safe2Tell “as a work in progress” (Grace Deposition, p. 173).

Finding 1c: Interagency Information Sharing Agreement

Following Columbine, the Colorado legislature passed HB 00-1119 and SB 00-

133 to facilitate the exchange of information about adolescents across agency

boundaries (e.g., law enforcement, schools, and mental health providers). The

legislature sought to encourage “open communication . . . to assist disruptive

children and to maintain safe schools” §19-1-302(1)(b) C.R.S. In 2000, the

Colorado General Assembly passed a mandate that:

Each board of education shall cooperate and, to the extent possible, develop written agreements with law enforcement officials, the juvenile justice system, and social services, as allowed under state and federal law, to keep each school environment safe. § 22-32-109.1(3) C.R.S.

However, challenges have emerged in the effort to implement the

Commission’s recommendations on information sharing agreements and threat

assessment procedures. Many districts and schools have yet to formally draft

an Interagency Information Sharing agreement to facilitate the exchange of

data across agencies on cases of public safety concern.

The Colorado Attorney General’s Office provides a Self-Assessment Checklist

for the development of an Interagency Agreement and Social Support Team

(see www.state.ago.co.us). The list provides questions for stakeholders to

answer to evaluate the level of agreement about the sharing of information

across agency lines (see Table 3: Interagency Agreement and Social Support


56

Team Self-Assessment Checklist). “These questions are designed as an aid to

create information sharing agreements among schools, law enforcement,

prosecution, courts, mental health, social services and other stakeholder

professionals. The goal is to assure a safe environment for students and staff,

provide a basis from which communities can organize Interagency Social

Support Teams (ISST) that are encouraged by the legislature and share

information mandated by statute (CRS 22-32-109.1(3) & CRS 19-1-303 and

304)” (see www.state.ago.co.us).

LPS did not create an Interagency Information Sharing Agreement, as

recommended by CRS § 22-32-109.1(3). LPS attorneys were not able to

provide documentation that an Interagency Information Sharing Agreement

had been in place between schools, mental health providers, and law

enforcement agencies prior to the shooting. According to Nathan Thompson

(Deposition, pp. 222-224) and LPS’s Superintendent Scott Murphy, no one

within LPS – including LPS’s attorney Steve Everall – knew of the Colorado

legislation recommending an Interagency Information Sharing Agreement for

all districts in the state, and Scott Murphy, Nathan Thompson and others’

comments suggest that they did not believe an agreement was relevant to

information sharing or school safety. The following exchange between Michael

Roche (attorney for the Davis family), Nathan Thompson, and Steve Everall

occurred during Thompson’s deposition:

Mr. Michael Roche: Does LPS have an interagency information sharing agreement with the Arapahoe County Sheriff's Office or the Littleton Police Department? Mr. Nathan Thompson: I believe there is some type of MOU, yes. Roche: And do you recall when that was signed? Thompson: I don't know. Roche: And do you have a work[ing] understanding of what that MOU permits LPS to do or share with those law enforcement agencies?


57

Thompson: There is nothing in there related to information sharing. I do know that we're in the process of reviewing and trying to draft new MOU's for more clarity. I know what our practice is, but I can't speak to what's in the agreement. LPS does have a form that law enforcement can use to request information or records. Roche: Sorry, I'm just stretching. Well, what is the practice at LPS as it related to information sharing with law enforcement? Thompson: Well . . . in the past, [SROs] have not had access to our Infinite Campus system. So they would have to ask a staff member, “Hey, can you look up this student's address or information -- or information for me.” This fall we did give them basic demographic access, so they can't necessarily see all of the kids' records, but they can look up contact information. But our practice has been at any time we can work with a law enforcement officer as a school-initiated investigation. So the way our procedures and policies work is if it is a school investigation and led by the administrator, we can request a school resource officer be there to help us search or be there to sit in when we investigate or interview a student. But the minute it becomes led by that officer, it now becomes a law enforcement investigation. And they have to meet all of the Miranda warnings and get all of the parents involved. I don't know if that's what you're looking for. Roche: I will ask again, can I get a copy of whatever that MOU is? I know a while back we had talked about that. Mr. Steve Everall: I sent it to you. Roche: No, what you sent me actually was an agreement between Arapahoe County and the school saying James Englert was authorized to be employed there. It was a different document entirely. Everall: I'll ask again. Roche: Okay. Thanks.

Scott Murphy (Murphy Deposition, p. 41-45) said:

Mr. Michael Roche: [D]id Littleton Public Schools have any kind of interagency information sharing agreement in place prior to the shooting in December 2013? Mr. Scott Murphy: I believe you’re speaking of a written document? Roche: Yes.


58

Murphy: I don’t recall any. . . I don’t know of any. . . Roche: [W]hy didn’t you direct your staff to – or somebody – to prepare a written information sharing agreement that you could send to the Arapahoe County Sheriff’s Office or the Littleton Police Department or any of the other law enforcement agencies whose jurisdictions overlapped with your school district? Murphy: I did not because I was not aware of the statute . . . frankly, the staff that works for me in a number of areas follow some of their requirements, and I don’t know why this wasn’t forwarded.

The Interagency Information Sharing Agreement, or an equivalent

memorandum of understanding (MOU) or court order, is a critical part of

keeping schools safe. Without the agreement many behaviors at school that

are of “public safety concern” will be missed. Unfortunately in this case, it was

ignored or at best forgotten. To ensure that district superintendents and their

staff are aware of statutes the Attorney General should annually update the

Colorado School Violence Prevention and School Discipline Manual on school

safety statutes, FERPA, and their application to school districts. Additionally,

school districts should conduct an annual training on all statutes related to

school safety and violence prevention and produce an annual compliance

report.

Major Finding 2: Threat Assessment

Today, most schools in Colorado are using a combined threat and risk

assessment tool, which seeks to follow the Secret Service’s six principles and 11

questions for completing a threat and risk assessment, but have not been

validated. In Colorado, this results in many different versions of threat and risk

assessment tools being used across the state (e.g., Jefferson County, Denver

County, Adams12, Cherry Creek, Littleton Public Schools, and Colorado School

Safety Resource Center). Each district is using a different mix of threat

factors, risk factors, and protective factors in their assessment tool. Each

district claims to produce a result which categorizes a student into three to


59

five levels of concern, threat, or risk, and with each level, there is a

corresponding set of follow-up actions. As far as the authors can determine,

not one of these tools has been validated using rigorous methodologies, as

described in the National Registry of Evidence-Based Programs and Practices

(NREPP) or Blueprints Program (www.colorado.edu/cspv/blueprints).

The findings indicate that the shooting may have been averted if the threat

assessment done on KP on September 9, 2013 had followed LPS’s threat

assessment training and policies and the Secret Service’s threat assessment

guidelines. To be clear, a poorly executed threat assessment did not cause

Claire’s death. KP caused her death. However, had the threat assessment been

executed properly (even using LPS’s un-validated threat assessment tool), and

a clear safety plan for follow-up executed, KP’s violent plans might have been

interrupted. A properly executed threat assessment could have revealed a

higher level of threat, and a higher level of threat should have prompted a

more serious action plan and more thorough action plan monitoring. If the

threat had been investigated more skeptically, a few highly qualified AHS staff

could have crafted a safety plan for KP that might have interrupted his plans.

This chapter discusses the deficiencies found in: (a) AHS’s execution of LPS’s

threat assessment guidelines, (b) the training in LPS’s threat assessment

procedures, and (c) the selection of valid and reliable risk and threat

assessment tools.

Finding 2a: Threat Assessment Guidelines

LPS’s threat assessment tool appears to have been crafted using some of the

Secret Service’s six core principles and 11 key questions. However, AHS’s

threat assessment team failed to follow LPS’s Threat Assessment Guidelines

(see Exhibit 4); they failed to thoroughly review and complete the checklist on

LPS’s threat assessment form (see Exhibit 35); and, they failed to apply the six

principles and 11 questions using an “investigative, skeptical, inquisitive,

mindset” with KP (Fein, et al., 2002, p. 29). The process of using the threat


60

assessment form to review the items on the checklist with KP was flawed. Dr.

Song explained her use of LPS’s threat assessment checklist with KP:

I was asking him questions off of this form [Exhibit 35]. When asked, “[W]ere [you] walking through all of these factors saying, for instance, ‘Does KP have access to weapons?’ And then the next item . . . ‘does KP has an ability to carry out his plan or was he serious about this?’

Song continued:

So, no, I didn't go through each check box, and in hindsight, I would have. (Esther Song Deposition, pp. 151-152)

If school administrators assume that the checklist is perfect, if the collection of

the data for each item is flawed, or the team does not go through each item,

then the checklist is irrelevant. The FBI’s The School Shooter: A Threat

Assessment Perspective (O’Toole, 2000, p. 1, emphasis added) states: “This

[threat assessment] model is not a ‘profile’ of the school shooter or a checklist

of danger signs pointing to the next adolescent who will bring lethal violence

to the school. Those things do not exist…Once a threat is made, having a fair,

rational, and standardized method of evaluating and responding to threats is

critically important (Cornell, Allen, and Fan, 2012).”

Continuing with the checklist discussion, in Mr. Kolasa’s deposition (pp. 138-

141), Attorney Michael Roche asked:

Mr. Michael Roche: Did you hear about any other incidents in which KP's anger had manifested itself besides the car accident, the Tracy Murphy threat, and the Dan Swomley incident? Mr. Kevin Kolasa: Not that I recall, no. Roche: Did you consider those three incidents in the span of just a few months to be a cluster of warning signs? Kolasa: I don't know if it was a cluster of warning signs or a cluster of incidences but -- because I don't know, warning signs leading to what? Roche: Well, do you understand in your role as, I guess, currently a principal of a middle school --


61

Kolasa: Assistant principal. Roche: Assistant principal. That one of the things that you're supposed to watch for in a threat assessment is a cluster of incidents? Kolasa: Right. Yes. Roche: And my question is real simple, did you consider those three things to be a cluster of incidents? Kolasa: Yes. Roche: And did that increase the threat level that you believed KP posed? Kolasa: Yes. Roche: As we continue down Exhibit 35 in the early warning sign factors, the very last item on the list is: “Does the student have a history or perception of being bullied or victimized by others.” Do you see that there? Right there. Kolasa: Got it. Yes. Roche: And we've looked at a number of documents and discussed a couple of incidents where KP expressed to you that all the teachers were out to fucking get him, that he had endured a decade of hell at the hands of his peers, those kind of things. Given that you knew about those incidents, why isn't this box checked? Kolasa: I don't know why it's not checked. Roche: Doesn't it seem like it should have been? Kolasa: If I was -- if I had the paper in front of me, yes, I would have checked it. Roche: Well, in fact, the incident that led to the paperwork about all the teachers being out to fucking get KP is described in that other relevant details section immediately below that unchecked box, isn't it? Kolasa: It is.


62

Roche: So it seems to me that what happened here is people just plain weren't paying attention, and that's why that box isn't checked. Is that what happened here? Kolasa: No, I don't feel like I wasn't paying attention. Roche: Okay. Well, let's go to the next part of this threat assessment document, and that's the at-risk factors, right? Kolasa: Yes. Roche: Do you see that? The very first one is what is the history of school discipline, right? Kolasa: Uh-huh. Roche: And the box oppositional misconduct is checked, right? Kolasa: Yes. Roche: And the box for suspension is not, right? Kolasa: Correct. Roche: And you personally had suspended this kid six months earlier, hadn't you? Kolasa: Yeah. Roche: And you didn't check this box either? Kolasa: Correct. Roche: You weren't paying attention, were you? Kolasa: I feel like I was, but I did miss those two things. Roche: Moving down the at-risk factors, another box asks whether or not the student externalized blame, right? Kolasa: Okay. Roche: Do you see that? Kolasa: No.


63

Roche: It's right there. Kolasa: Okay. Roche: Got it? Kolasa: Yep. Roche: Now, you just described to me that both KP and his parents felt it was unfair that he was being demoted from his captain's position on the debate team, right? Kolasa: Uh-huh. Roche: Wouldn't that qualify as KP externalizing blame? Kolasa: Yes. Roche: So why isn't that box checked? Kolasa: I don't know. Roche: Let's keep moving down to -- you'll see there's a section called drugs or alcohol concerns. Do you see that? Kolasa: Yes. Roche: And right below that it asks whether or not the student is sensitive to feedback or criticism, right? Kolasa: Correct. Roche: That box also isn't checked, right? Kolasa: That's correct. Roche: And isn't that exactly what you suspended KP for in March of 2013, an outburst because he was sensitive to criticism about having gotten a bad grade in Mr. Swomley's class?

Kolasa: Yes.


64

In a second example of the problems with AHS’s use of LPS’s threat

assessment checklist, the following exchange between Attorney Michael

Roche and Dr. Esther Song (Esther Song Deposition, pp. 155-156) proved

illustrative:

Mr. Michael Roche: I'm just trying to figure out which factors were increasing your level of concern and which ones were decreasing. And the fact that he had a long history of anger was one that increased your level of concern, right? Dr. Esther Song: Right. Roche: And what about in the margins here? You've written that KP understands his reaction was inappropriate, but does not seem to be remorseful or understanding of Tracy Murphy's feelings of being threatened. Did his lack of remorse and lack of empathy increase your concerns? Song: It did. Roche: Why? Song: Because I was concerned about the fact that he -- I mean, it was concerning that he didn't have that -- he didn't feel bad about Tracy feeling threatened. I don't know how else to explain it. Roche: Well, I'm certainly not a psychologist, and don't pretend to be. But I have read that lack of remorse and lack of empathy are two of the hallmarks of a sociopath. Am I right about that? Song: I'm sure they're characteristics of that. I don't know if it's hallmarks. Roche: Okay. Characteristics. Song: Okay. Roche: Did you ever have the thought that, “Okay, this kid shows poor impulse control, no remorse, and no empathy, those are characteristics of a sociopath?” Did you think about that as you went through this process? Song: I'm sure I did. Roche: Okay. I take it that increased your concern level as well?


65

Song: Yep. Roche: And – Song: And I think I was missing pieces of information as well. Roche: Agreed. Song: So, yeah, if I could look back and see all of those pieces, absolutely, my level of concern would be different, and we would probably be having a different conversation.

The evidence presented here and prior research indicates that there are

problems with the checklist approach. A checklist without an “investigative,

skeptical, inquisitive, mindset” and without a concerted effort to collect data

from teachers, parents, and peers is simply worthless (Fein, et al., 2002, p. 26).

It is true that most district-created threat assessment tools imply that the

more risk and threat factors “checked” on the form, the higher the probability

that targeted violence will occur, but this assumption has not been tested

empirically.9 Some may think if they complete a threat assessment and check

off all the boxes and the student is a low level risk, as occurred with KP, then

they do not have to worry about this person. With KP, the boxes were

checked, he was labeled a low risk; so, minimal follow-up was prescribed. That

low risk designation was based on checking off a list of threat and risk factors.

In fact, in a review of the 17 threat assessments conducted within LPS high

schools from 2011 to 2015, the number of risk factors, threat factors, and

protective factors checked was not correlated with the designated level of

concern. Thus, LPS’s assessment tool is not a reliable predictor of level of

concern.10 LPS’s checklist indicators are not summed or tallied to inform the

estimation of concern, which is why such tools are not recommended.

9Verlinden,Hersen,andThomas(2000)foundsomeconnectionsbetweenriskfactorsandlevelof threat;however,his

researchexaminedtenshootersandwasconducted“posthoc”aftertheshootingsandallinformationwasdiscovered.10Inaquantitativeanalysisof18threatassessmentscompletedatAHSfrom2011to2015,noclearassociationwasfound

betweenstudents’totaledassessmentscores[i.e.,assessmentscore=(threatfactors+riskfactors)-protectivefactors]

and theirdesignated level of concern (e.g., low,medium,high). In these calculations, the assessment scores for the18


66

But it appears that AHS believed that their three levels of threat were

connected to a valid actuarial scale that predicted outcomes (see Appendix 7).

For instance, they checked-in with and offered KP less support because he

was identified as a “low level concern,” but he was incorrectly assessed. As we

noted above, the levels of threat approach may be flawed. Once rated low,

their error was compounded by not being on the lookout for evolving risk

factors and by waiting for reports of problems. There was no proactive

inquisitive drive to keep track of his progress or lack of progress. The clinical

psychologist who completed the threat assessment on KP, Dr. Esther Song

(pp. 143-145) said, “We did our best.” She continued:

Mr. Michael Roche: Okay. Would it have been important to you to know that KP's anger could reach the point where he would run through a stop sign in a rage -- Dr. Esther Song: Absolutely. Roche: -- and wreck his car? Song: Absolutely. Roche: Okay. Now, the early warning sign factor that is there for violent/threatening themes conveyed in stories, diary entries, essays, letters, songs, drawings or videos, that's not checked, correct? Song: Correct. Roche: And part of why that's not checked is because you didn't look at any of those items, right? Song: Correct. Roche: And that's because nobody had told you that there was anything to see in his stories, diaries, essays, letters, et cetera, correct?

casesrangedinvaluefrom4to38,withameanassessmentscoreof16(withastandarddeviationof9).Inlookingatthe

levelofconcern,12studentswereidentifiedasalowlevelconcern,4studentswereidentifiedasamediumlevelconcern,

and2studentswereidentifiedasahighlevelconcern.Whencomparingtheassessmentscorewiththelevelofconcern,

questions arise about the ability of the assessment factors to reliably inform the level of concern. For instance, one

student received a 38 on the assessment score, the highest of the 18 cases, but was viewed as a “medium” level of

concern;while,anotherstudentreceiveda22ontheassessmentscore,butwasviewedasa“low”levelofconcern.The

disconnectbetweentheassessmenttotalsandthelevelofconcernisimportant,becauseLPS’stoolimpliesarelationship

betweenthethreat,risk,andprotectivefactorsandastudent’slevelofconcern.Onlythroughtheuseofavalidatedtool

cantheserelationshipsbemanagedtominimizelabeling(i.e.,falsepositives)andmissedthreats(i.e.,falsenegatives).


67

Song: Correct. Roche: And because you didn't talk to any of his teachers about whether there was anything concerning in any of those categories, right? Song: Correct, they usually -- so I'm not pointing a finger at the teachers, but usually they would bring us -- they bring it to us if there was something concerning, a student's writing or artwork. Roche: Okay. Well, the teachers weren't told that KP was going to be the subject of a threat assessment, were they? Song: Not that I'm aware of. Roche: And none of the other boxes in the early warning sign factors are checked, are they? Song: No. Roche: And that includes the early warning sign factor for a history of perception of being bullied or victimized by others, right? Song: Right. Roche: I take it that was another mistake? Song: I'm assuming so. Roche: Can you shed any light on why that isn't checked given the fact that KP said almost those exact words in the behavioral detail report we just looked at? Song: I mean, my -- honestly, and it's not – I don't want it to sound like I'm making excuses for anyone or anything that we did, but I feel like, yeah, if I could have gone back, I would have taken all the time in the world to sit there and make sure I was doing everything as -- sorry. Roche: It's okay. Song: But I feel like we did the best that we could in the moment, and it's not an excuse, but I feel like -- yeah, if there were boxes that were mischecked or not checked, it was my fault for not doing that, but I guess I just -- I don't really have an excuse for not having that done. I don't think I answered your question.


68

In summary, KP was scored “low” on the LPS threat assessment tool (ACSO, p.

193-196). However, without anchors for each item, it would be difficult to

repeat this scoring with a different set of scorers. In addition, using a tool that

has not been validated also makes the process suspect. Failing to follow the

Colorado School Safety Resource Center’s (CSSRC) process for an

interagency social support team, no matter the level of concern, is flawed. For

example, in the LPS threat assessment, the person completing the form is

asked to simply check off “Early Warning Signs and At Risk Factors.” There

are no scoring definitions of each of these items. Therefore, when should

“depression, self-harm, and/or suicide issues” be scored? – when one student

hears someone say “I feel depressed” or when three students say so? In

another example, the risk factor “student experienced rejection or humiliation”

was not checked for KP, yet the record is clear that he was “angry” when the

teacher he threatened to kill rejected him as a captain of the debate team. In

addition on March 15, 2013, when his grade was read out loud in class and he

said “fuck” and “teachers are out to get me” (ACSO, p. 190), he was suspended

for this outburst. But he was not scored as “experienced humiliation” in the

“At-Risk Factors” list (ACSO, p. 194; Exhibit 35).

In this case, it can be argued that KP fit the first four of Newman and

colleagues’ (2004) five “necessary but not sufficient” criteria for a school

shooter, and that when he showed others photos of his gun in the days before

the shooting, the fifth element was in place and an immediate intervention

should have occurred.11 Since the Rampage criteria have not been fully tested,

11NeitherLPS,norotherschoolsinthemetroarea,haveadoptedtheRampage(Newman,etal.,2004)criteriatoidentify

possible shooters. While the research is ongoing, it is important to note that the five necessary but not sufficient

conditions outlined in Rampage were consistent in the sample of 74 cases investigated by Newman. These criteria

included:(1)shootersperceivedthemselvesasextremelymarginal inthesocialworldsthatmattertothem(“…KPjust

struckmeasunsureofhimself,awkward,sociallyawkward. . . Iwouldventure todescribehimasabiton thesocially

ineptside”(TracyMurphyDeposition,pp.16-18);(2)shootersmustsufferfrompsychosocialproblemsthatmagnifythe

impact of marginality; (3) shooters believe that unleashing an attack on teachers and classmates will resolve their

dilemmas;(4)failureofsurveillancesystemsthatareintendedtoidentifytroubledteensbeforetheirproblemsbecome

extreme;(5)gunavailability.


69

schools should use them for data collection and to inform the threat

assessment team, but not as formal criteria for threat assessment.

In fact, the American Psychiatric Association warns its own psychiatrists that

they cannot predict violence 12 ; therefore, schools and communities must

provide threat assessment policies in their jurisdictions. This policy must

adhere to the basic principles of threat assessment promulgated by the Secret

Service.

There are many proposals to use a checklist for threat assessment. This notion

comes from aviation and medicine, but it does not work well with threat

assessment. Pilots and surgeons receive instant feedback about the success

or failure of their checklist, because if they miss something, the plane crashes

or the patient dies. This is not the case with threat assessments.

AHS failed to follow the six basic principles of threat assessment as noted by

the Secret Service (Exhibit 37) and outlined in LPS’s Threat Assessment

Training presentation (Exhibit 4). In this case KP received a “low” threat

assessment rating. By policy only the medium and high threat assessments

were reviewed at the District level in 2013 (post AHS shooting – all threat

assessments are reviewed at the District level by the Director of Security and

Emergency Planning).

12“Psychiatric expertise in the prediction of ‘dangerousness’ is not established and clinicians should avoid ‘conclusory

judgments in this regard.” Motion for Leave to File Brief Amicus Curiae and Brief Amicus Curiae for the American

Psychiatric Association, Supreme Court of the United States, October term 1979, Case No. 79-1127, "W.J. Estelle, Jr.,

Director,TexasDepartmentofCorrectionsv.ErnestBenjaminSmith,"pp.14-15.


70

The following findings are based on a thorough inspection and review of the

threat assessment of KP that occurred on September 9, 2013. Here, each

principle is reviewed, as it applied to KP’s case; these examples illustrate the

factors not thoroughly investigated:

Review of Secret Service’s Six Principles as Applied to KP’s Assessment

1. Process of Thinking

There was a lack of understanding that KP might have been following an often times discernible process of thinking and behavior. According to the threat assessment notes and

deposition testimony, the threat assessment team never took a proactive stance to find out about KP’s thinking on the subject of killing Mr. Murphy. They simply accepted his apology for his outburst, even though he was not remorseful. Kevin Kolasa said, “KP was very clear that that he didn’t care that Tracy Murphy was upset about what he said” (Kolasa

Deposition, p. 122). This disturbing news was never followed up on.

2. Interaction Between Individual, Situation, Setting & Target The record is silent on any proactive work to discern any planning on KP’s part, but he does have a plan mentioning all four of these items which was never uncovered by the threat assessment team or any other person.

3. Interaction Between Individual, Situation, Setting & Target The following are examples of a failure to apply this principle. a. There was no effort to identify or contact others who might have known about KP – no

students who knew him were sought, no teachers who might have had positive or

negative experiences with KP were invited, the SRO was not invited (see also Kolasa Deposition, p. 127). In fact, only two threat assessors participated in the evaluation of KP, while state standards require a minimum of three threat assessors on every team (see Exhibit 5, p. 3). In addition, there was no documented attempt to reach out and find

others for the threat assessment. b. When asked, “Did you ever consider the possibility that KP was lying to you about what

his motives and intentions and plans were?” Kolasa said: “Yeah. I didn’t feel like he was, but I know that. . . he could have been, yes.” When asked, “Did you do anything to try to verify what KP was telling you?” Kolasa said, “I don’t know how that would look” (Kolasa

Deposition, pp. 114-115). c. In a series of questions about the threat assessment, Dr. Song’s testimony revealed (pp.

175-176):

Mr. Michael Roche: Let’s . . . go back to the threat assessment document itself. . . And in KP’s case, the action plan is laid out on this document, correct?

Dr. Esther Song: Correct. . . Roche: No suicide risk screening [was] performed?

Song: Right.


71

Roche: No mental health evaluation [was] performed, right? Song: Right. Roche: And then there’s additional measures to ensure safety. That box is checked?

Song: Right.

Roche: And it says one time per week – psych office or psych appointment?

Song: Right. . . [which is a reference] to outside therapy.

Roche: Okay. And what did you or anyone at Arapahoe do to verify that KP was actually going to a weekly psych appointment? Song: I didn’t do anything to verify that.

4. Effective Threat Assessment The following items on the threat assessment were not checked under either early warning

signs or at-risk factors (see Exhibit 35), and there is evidence to suggest that they should have been checked if an investigative, skeptical and inquisitive mindset had been adopted: a. History of Discipline: Should have checked “suspension.” On March 15, 2013, KP was

suspended by Kevin Kolasa for using the word “fuck” and stating “teachers out to get me” (see Exhibit 32). When Kolasa and Song completed the threat assessment form, that suspension was not noted. Dr. Song said, “I don’t think he had been suspended.” When Dr. Song was asked, “[You] didn’t check the box for suspension, because you didn’t know that he had been suspended, is that right?” Dr. Song said: “Correct.” (Song

Deposition, p. 111 and 146)

b. Poor Student Achievement or Academic Progress: Should have been checked. In

September 2013, grades had not yet been given to students, but this should have been added to the assessment when it became apparent that KP was failing several classes in mid-November. However, because there was no proactive look at KP’s grades the AP failed to notice his dropping grades and if he did notice them he failed to realize that this item would raise KP assessed threat score (ACSO, p. 00187; Exhibit 21). In Fall 2013, he had two F’s and two D’s across his six classes, and on November 1, 2013, Vicki Lombardi

emailed KP’s mother expressing concern about his grades (Exhibit 21). In prior years, KP had less than “B” average (see Kolasa Deposition, p. 78; ACSO, p 7).

13 So, there was a

significant decline in achievement over a three-month period. Kolasa said: I remember him having . . . a D in AP Economics, a C in AP U.S. History, . . . an A in International Relations [with Jeff Corson],. . . and a B in weight training. . . [H]is F was in . . . World Lit. . . and he also had an F in Physics. (Kolasa Deposition, p. 79)

c. Violent Behavior Toward Others: Box not checked. The form includes two categories:

“1-2 incidents” and “significant history.” Significant History was checked, but not the overall category of “violent behavior towards others.” Because the overall category was

not checked, it does not stand out in the visual inspection of the form. The student contact log indicates that KP hit two students with his lunchbox and kicked and hit two others in November and December 2003 (see Exhibit 24). A handwritten note on the

13Inadepositionamendment,KevinKolasastated:“No,outofsixsemesters,halfofthemKarlhadaGPAbelow3.0.”


72

threat assessment form stated: “Mom reports ‘deep-seeded’ anger and KP agrees that he’s had anger management issues for a while.”

d. Student Externalized Blame: Box not checked. Clearly KP blamed Tracy Murphy for his

problems at school and threatened to kill him. Even his peers knew that KP blamed others for everything. In the ACSO investigation, an AHS student reported, “KP was the type of person that blamed others for everything” (ACSO Report, p. 10).

e. Peers are Fearful of Student: Box not checked. Claire was fearful of KP and so were

others according to Sheriff’s report. No students were interviewed for the threat assessment. However, in an incident after the September 9

th threat assessment and

September 26th

follow-up, KP scared his Spanish teacher and classmates when he

banged on the classroom door and screamed on December 11, 2013. When Lombardi notified Kolasa of KP’s behavior, the outburst should have triggered another threat assessment review.

During her deposition, Spanish teacher, Vicki Lombardi was asked, “Were the students in your classroom scared as well that you could see?” Lombardi said, “yes. Yes. . . I was just unsettled. I was like, this kid is yelling at me, and my kids are looking at me with these big eyes, and I just felt like I needed to get him out of my class” (Lombardi

Deposition, p. 49).

“I know [Kolasa] heard me because he said, ‘I can tell you’re scared, and I can tell your students are scared by the way they acted when I walked in’ ” . . . when asked, “Does it bother you that Mr. Kolasa either denies that or doesn’t remember you telling him [that you were scared]?” Lombardi said, “Very much so” (Lombardi Deposition, p. 57).

In other words, a lot of people were scared of KP in the days prior to the shooting, but

no one thought to redo the threat assessment or change his action plan.

f. Sensitivity to Feedback: Box not checked. We know of at least three instances when KP reacted very negatively to constructive criticism, including his “I’m gonna kill that guy”

reaction to being removed from debate team (Exhibits 18 and 24), his reaction to Jeff Corson about not calling classmates “stupid” was negative (ACSO, p. 5), and when his grade was read aloud in class, he yelled “fuck you” (Exhibit 32).

g. Student Tends to Hold on to Grudges and Resentments: Box not checked. He was very

angry with Tracy Murphy for removing him as a Team Captain in Debate (Tracy Murphy Deposition, p. 79). KP initially refused to apologize to Tracy (ACSO, p. 8) and he was not

remorseful or understanding of Tracy’s feelings of being threatened (ACSO, p. 194). In addition, he told several peers that he was going to get Murphy.

h. Student Recently Humiliated: Box not checked. KP was recently humiliated in class over

a failing grade – ‘girls were giggling at him about math grade’ (ACSO, p. 194).

i. Bizarre or Concerning Behaviors: Box not checked. For example, the “fuck you” incident

(ACSO, p. 9), and talked about dreaming his penis fell off at debate (ACSO, p. 5).

5. Integrated Systems Approach AHS failed to adopt an “integrated systems approach” to threat assessment, as recommended by the Secret Service, and AHS violated the CSSRC recommended minimum of three members to complete a threat assessment (see CSSRC’s, Threat Assessment, Section I, 2, e).


73

a. They did not have the minimum of three staff for KP’s threat assessment. Only two AHS

staff, Song and Kolasa, completed the threat assessment. They could not have possibly

known everything there was to know about KP. This was a violation of the CSSRC’s threat assessment guidelines.

b. In addition, they did not include other system players with knowledge of KP (e.g., Tracy Murphy,

14 Jeff Corson, Michelle Crookham, SRO James Englert, CSO Christina Kolk, or

peers) to attend or contribute to the threat assessment. In an “integrated systems

approach” to threat assessment, representatives from special education, law enforcement, district attorney’s office, DA, social services, and principal were not present.

c. The sheriff’s investigation found many students and teachers with knowledge of KP –

none of whom were asked for statements or to appear at the threat assessment (see

ACSO, p. 5; Tracy Murphy Deposition, pp. 21 and 61); Tracy Murphy, the teacher who KP threatened was not included in the threat assessment meeting and his concerns were ignored. If he had been included in the threat assessment, he could have provided the information below that he provided to other teachers, assistant principal, and principal.

6. “Poses” a Threat, Not Makes a Threat The fact that KP posed a threat was missed. Attention to the detail of the threat assessment and its probable upgrade to at least a medium threat would have caused KP’s threat assessment to go to the district level where it might have been reviewed in more detail.

In violation of the state guidelines for threat assessments, no teachers

knowledgeable about the student were included in the threat assessment

process. The Secret Service notes that:

Different people in the student’s life may have different and possibly small pieces of the puzzle. It is the responsibility of the threat assessment team to gather this information from what may be multiple sources, such as teachers, parents, friends, counselors, after-school program staff, part-time employees and others (Fein, et al. 2002, p. 35).

Overall, out of 24 possible risk factors on KP’s threat assessment, only five

were checked. This investigation revealed that seven to nine additional risk

14ThefactsthatTracyMurphycouldhavesharedifincludedinKP’sthreatassessmentincluded:(1)Tracyneverhadakid

lookathimwiththelookofhatredthatKPdidthatday.Itwas“haunting”howKPlookedathim(ACSO,p.5);(2)Tracy

toldAstridThurneauthathehadabad“gut’feelingaboutKP,andAstridtoldTracyhehadtogowithhis“gut”(ACSO,p.

6); (3) Tracy tried to speak to the Principal, butwas told to see Kevin Kolasa. He did not feel she took the situation

seriously(MurphyDeposition,pp.106-107)(ACSO,p.6);(4)TracywassoconcernedhespoketoKevinandaskedforthe

surveillancevideofortheafternoonofSeptember3,2013tobepulledtoverifytheincident.Thisdidnotoccur.(ACSO,p.

7);(5)Tracy,stillconcerned,spoketothesecretaryforAPDarrellMeredithtwoweekslaterandagainaskedforvideo,

andwastoldthatthevideowasnotpulledandthatithadbeenwrittenover(ACSO,p.7);(6)Tracyrelayedhisconcerns

aboutKPtoKevinandEsther.HeseriouslythoughtaboutresigningfromhispositionatAHSbecausehewantedtotake

himselfoutoftheschoolasatarget.TracybelieveditwouldbesaferforhimanditwouldprotectothersKPmighthurt

tryingtogettohim.Tracy’sfeelingthatKPwas“trouble”neverwentaway.


74

factors could have been checked if all threat assessment team members with

knowledge of KP had been recruited and there had been a clear definition of

each item. It appears that no investigation of the “un-checked factors” was

initiated even when there were indicators that many could have been checked.

If the seven to nine additional items had been added to the risk factors, it is

very possible that the concerns about KP’s threat level would have been

greater. If KP had been assessed as a medium or high level concern, his threat

assessment would have been sent to the district for review and it is possible

that more risk factors would have been uncovered, more time might have

been spent with KP, and his plan to kill people might have been uncovered.

In summary, AHS’s threat assessment process consistently failed to address

two important elements of the Secret Service’s Guidelines for threat

assessment procedures: (1) that there is enough reliable information to answer

the 11 key questions and (2) that the weight of the information is convincing

that the student does not pose a threat of targeted school violence. Once

these elements have been answered, then, the threat assessment team can

conclude the threat assessment inquiry (Fein, et al., 2002, pp. 56-57). The

Secret Service states: “Evaluation of information gathered from research and

interviews conducted during a threat assessment inquiry should be guided by

the following 11 key questions” (Fein, et al., 2002, pp. 55-57). In summary, AHS

did not follow LPS’s policy to use the Secret Service’s six principles and 11

questions in the completion of KP’s threat assessment.


75

The following figure details AHS’s errors in pursuing the Secret Service’s 11 key

questions for a threat assessment:

Secret Service’s 11 Questions as Applied to KP’s Threat Assessment

Questions AHS/LPS Error 1. What are the student’s motive(s) and

goals?15

Failure to dig deeper, the threat assessment team believed KP, without verification.

While testimony from Kevin Kolasa indicates that KP’s

apology to Tracy Murphy weighed heavily in the assessment (Kolasa Deposition, p. 177), the evidence reveals a failure to dig deeper into that grudge (e.g., “does not seem to be remorse and understanding of

Tracy Murphy’s feelings of being threatened,” see ACSO, p. 194).

Clearly, when KP did not feel remorseful, he had not

let go of his anger for Tracy Murphy’s decision. But there is no discussion or follow up on this item. And as the result of his being removed he started a journal specifically targeting Tracy Murphy.

2. Have there been any communications suggesting ideas or intent to attack? a. What, if anything, has the student

communicated to someone else (targets, friends, other students, teachers, family, others) or written in a diary, journal, or Web site concerning his or her ideas and/or intentions?

b. Have friends been alerted or "warned away"?

Failure to look at all possible communications including backpack, writings, and computer. No attempt was made to search his backpack, locker,

journals, or computer for any information about a pending attack, even though LPS Policy would have allowed it.(Meredith Deposition, p. 168). He did communicate with several students that he was going

to kill Murphy (ACSO Report, Exhibit 14).

3. Has the subject shown inappropriate interest in any of the following? a. School attacks or attackers b. Weapons (including recent

acquisition of any relevant weapon) c. Incidents of mass violence

(terrorism, workplace violence, mass murderers)

Failure to follow up on sighting him looking at weapons and other shootings and home search.

During the threat assessment, KP claimed no interest in weapons. It was subsequently determined that he

had (ACSO Report, Exhibit 14):

• Weapon proficiency from the Venture Crew summer program called RAMS

• Searched mass shootings and guns on the web

• Shared the Anarchist Cookbook with a peer

• Shared instructions on making Molotov cocktails with a peer

• Showed a picture of his shotgun to peer

This information should have triggered a reassessment of KP behavior and action plan.

15Fein,etal.(2002)elaborate:(a)Whatmotivatedthestudenttomakethestatementsortaketheactionsthatcausedhimor

hertocometoattention?(b)Doesthesituationorcircumstancethatledtothesestatementsoractionsstillexist?(c)Doesthe

studenthaveamajorgrievanceorgrudge?Againstwhom?(d)Whateffortshavebeenmadetoresolvetheproblemand

whathasbeentheresult?Doesthepotentialattackerfeelthatanypartoftheproblemisresolvedorseeanyalternatives?


76

4. Has the student engaged in attack-related behaviors? These behaviors might include: a. Developing an attack idea or plan b. Making efforts to acquire or

practice with weapons c. Casing, or checking out possible

sites and areas for attack d. Rehearsing attacks or ambushes

Failure to look for these items during the follow up period. During the Action Plan period after the threat assessment KP was noticed looking at guns online in the cafeteria. This information was passed on to Meredith, but did not evoke a reassessment of KP and

was never communicated to Kevin Kolasa (Kolasa Deposition, p.108, p. 12-15).

KP also came back to school on September 10, 2013

to attended Tracy Murphy’s debate club meeting after being told not to return. (T. Murphy Deposition, p. 179). This is clearly a boundary probing behavior and when reported to Kolasa should have triggered a

reassessment of KP’s behavior.

5. Does the student have the capacity to

carry out an act of targeted violence? a. How organized is the student’s

thinking and behavior? b. Does the student have the means

(e.g., access to a weapon) to carry out an attack?

Failure to find information about his weapons training, and his purchase of a shotgun just before the attack.

KP’s capacity to carry out an attack went unnoticed because no one followed-up with him. After the shooting it came to light that he had received weapons training. But no one took the time to

investigate and find out about this training. KP did have the means to carry out an attack. He showed his new gun to several students just before

the attack (ACSO, p. 10).

6. Is the student experiencing hopelessness, desperation and/or despair? a. Is there information to suggest that

the student is experiencing desperation and/or despair?

b. Has the student experienced a recent failure, loss and/or loss of status?

c. Is the student known to be having difficulty coping with a stressful event?

d. Is the student now, or has the student ever been, suicidal or “accident-prone”? Has the student engaged in behavior that suggests that he or she has considered ending their life?

Failure to probe hopelessness and despair.

Threat assessment indicated that KP was experiencing desperation and/or despair. He had experienced recent failures such as failure in

classes, and loss of status from demotion in debate team and inability to practice with the team. KP was having difficulty coping with his Dad’s

divorce, and he indicated that he had an anger problem (ACSO, p. 194)


77

7. Does the student have a trusting relationship with at least one responsible adult? a. Does the student have at least one

relationship with an adult where the student feels that he or she can confide in the adult and believes that the adult will listen without judging or jumping to conclusions? (Students with trusting relationships with adults may be directed away from violence and despair and toward hope.)

b. Is the student emotionally connected to–or disconnected from–other students?

c. Has the student previously come to someone’s attention or raised concern in a way that suggested he or she needs intervention or supportive services?

Failure to have one at school, and no one noticed – including the threat assessment team. It was a clear possibility that KP had a trusting relationship, as one teacher in international studies was able to connect with KP, but was never asked to come into the process. (Corson Deposition, p. 43).

There was no work done to determine whether KP was emotionally connected or disconnected from other students.

KP had come to someone’s attention but it was not discussed and when he started to fail in school, no new planning took place.

8. Does the student see violence as an acceptable–or desirable–or the only–way to solve problems? a. Does the setting around the

student (friends, fellow students, parents, teachers, adults) explicitly or implicitly support or endorse violence as a way of resolving problems or disputes?

b. Has the student been "dared" by others to engage in an act of violence?

N/A

9. Is the student’s conversation and “story” consistent with his or her actions? Does information from collateral interviews and from the student’s own behavior confirm or dispute what the student says is going on?

Failure to pursue “story”. No collateral interviews done. No follow-up on this item for the action plan. No attempt to confirm or

dispute except from Mom who said a psychologist said he was not dangerous to himself or others.

10. Are other people concerned about the student’s potential for violence? a. Are those who know the student

concerned that he or she might take action based on violent ideas or plans?

b. Are those who know the student concerned about a specific target?

c. Have those who know the student witnessed recent changes or escalations in mood and behavior?

Failure to probe concerns.

Several students said they were afraid of him in the ACSO report, but never shared that information with school administrators or teachers, nor with Safe2Tell.

Some who knew KP were concerned about a specific target (Murphy Deposition, p. 94, 194).


78

11. What circumstances might affect the likelihood of an attack? a. What factors in the student’s life

and/or environment might increase or decrease the likelihood that the student will attempt to mount an attack at school?

b. What is the response of other persons who know about the student’s ideas or plan to mount an attack? (Do those who know about the student’s ideas actively discourage the student from acting violently, encourage the student to attack, deny the possibility of violence, passively collude with an attack, etc.?)

Failure to recognize new risk factors. Obtaining a shotgun, incident in class about grades,

returning to Mr. Murphy’s class without his permission, October looking at weapons on computer in lunchroom, 11/1 Mr. Lombardi outburst.

Finding 2b: Threat Assessment Training Procedures

Teachers, some assistant principals, and the principal were not trained in threat

assessment. The principal and many assistant principals did not attend training

in threat assessment (LPS 00858- LPS 00861). All AHS assistant principals and

the principal have attended LPS’s threat assessment training since December

13, 2013. The records for AHS staff’s threat assessment trainings were

reviewed and revealed the following:

a. No AHS principal ever attended a threat assessment training from 2011

through 2013, according to official records (see LPS 00858-LPS 00861).

b. While mandatory in theory, few if any critical AHS personnel attended

threat assessment training prior to the shooting. However, since the

shooting one assistant principal said, “I believe mandatory is mandatory”

(Meredith Deposition, p. 69, line 15).

c. The SRO James Englert did not attend LPS’s threat assessment training and

had never received the training prior to the shooting; however, SROs are

now invited to LPS’s threat assessment trainings.


79

d. LPS’s threat assessment training was didactic and did not include scenario

practice sessions with one-on-one coaching and feedback. LPS’s training

had no role-playing, and participants did not actually complete a mock

threat assessment during the training. Research finds that the didactic,

reading, and audio visual presentation methods used by LPS in their threat

assessment training typically only yield a 20% retention among participants

(see Appendix 6: Skills Training with Guided Practice).

In summary, in 2011, five AHS staff were trained in threat assessment (two

assistant principals, one school psychologist, two counselors). In 2012, two

AHS staff were trained in threat assessment (two counselors). In 2013, one

AHS staff was trained in threat assessment (assistant principal). In 2014, 15

staff were trained in threat assessment (see LPS, pp. 00858-00861).

Finding 2c: Untested Threat Assessment & Risk Assessment Tools

Threat assessments are required for every threat, but risk assessments are

desirable for every student in crisis to help develop the student’s safety and

support plan. In other words, a risk assessment is invaluable for the

development of the safety and support plan. Currently, the literature identifies

several validated risk assessment tools, including (1) SAVRYTM and (2) Risk and

Resiliency Checkup (RRCU) (Singh, 2011; Turner, 2005). First, Randy Borum, a

co-author of the Secret Service’s Threat Assessment in Schools, created

SAVRYTM (see Fein, et al., 2002); it is a well-respected and validated

automated decision-making risk assessment tool that was born from the

Psychopathy Check List and the Risk Sophistication Treatment Inventory

(R.S.T.I) (Gladwell, 2015). However, many school districts express reluctance to

use SAVRYTM, because they view it as too complicated, too long, and not in

keeping with other risk factor analysis. Nathan Thompson, LPS’s Coordinator

for Student Support Services, explained:

I’ve never been trained on administering it, but my understanding is it’s more of a normed formal assessment tool that can be used to help. . . I


80

don’t know if they would say it predicts, but to give you more of a scored version of how at-risk is this kid for violence. . .We looked at [SAVRYTM]. We did look at that. We looked at the PETRA, which is another form of that kind of [threat] assessment. . . [we decided that] those aren’t necessarily effective and that they confuse the waters even more. (Thompson Deposition, pp. 248-249)

However, the empirical literature provides strong evidence for the validity of

SAVRYTM as a risk assessment tool. In a review of 68 studies and nine

commonly used risk assessment tools, Singh and colleagues (2011, emphasis

added) found:

A tool designed to detect violence risk in juveniles, the Structured Assessment of Violence Risk in Youth (SAVRYTM), produced the highest rates of predictive validity.

The RRCU is another validated risk tool that specifically details “protective

factors”, scored to insure that both risk and protective factors are taken into

account when calculating an overall resiliency. It is the only risk assessment

tool known to comprehensively investigate the protective factors at the same

level as the risk factors and create an overall resiliency score. The findings

indicate that the failure to carefully consider the enhancing protective factors

for KP may have reduced his chances of being deterred from committing a

school shooting. For example, KP’s International Studies teacher Jeff Corson

developed a positive relationship with KP, despite earlier instances where KP

bullied classmates in the fall of 2013, but Corson was not asked to be part of

the threat assessment or safety action plan. If an administrator had reviewed

KP’s grades and noticed that International Studies was the only class where KP

had an “A”, KP’s area of “strength” and Corson’s success with him in class

might have been identified. This protective factor, which could have been

identified using the RRCU, was a critical missed opportunity for positive

intervention with KP.

The Virginia Student Threat Assessment Guidelines (V-STAG), developed by

Dr. Dewey Cornell is a validated and reliable threat assessment tool. The


81

National Review of Evidence-Based Programs and Practices (NREPP) lists V-

STAG as an evidence-based practice. Cornell and colleagues (2009)

compared 95 public high schools using V-STAG to 131 public high schools

using a locally developed threat assessment program and 54 high schools

using no threat assessment program. The results indicated that:

Compared with 9th graders in high schools that used a locally developed threat assessment program, 9th graders in high schools that used V-STAG had less [of a] bullying school climate, less bullying victimization, and less criminal victimization…compared with 9th graders in high schools that had no threat assessment program, 9th graders in high schools that used V-STAG had less of a bullying school climate. (Cornell, et al., 2004 and 2009; SAMHSA, 2015).

In summary, according to empirical research, V-STAG represents the best

threat assessment tool and SAVRYTM produces “the highest rates of overall

predictive validity” (Singh, et al., 2011, p. 9). Most school districts have

developed their own untested threat assessment tools using a combination of

threat factors and risk factors from several different sources, such as the FBI,

Secret Service, and Surgeon General. These various lists include: (1)

overlapping risk and threat factors, making it confusing for districts to create a

perfectly inclusive list, (2) very few of these agencies’ lists include risk and

protective factors for violence that have been validated, and (3) the risk and

threat factors often do not have operational definitions (or anchors) for each

indicator. To add to the confusion, the tools include indicators for: threat

factors, early warning sign factors, at-risk factors, and protective factors.

School districts may assume that the longer their list, the more thorough the

threat assessment. However, an untested and inappropriately used tool can

never accurately predict violence. Thus, when districts create their own threat

assessment tool, it may or may not predict violence. Districts need to validate

the tool’s ability to predict the threat and risk of violence before it is used for

intervention, planning, and support. Surprisingly, the flaws with this make-

your-own-list approach are not addressed in the threat assessment or school


82

safety literature. Schools are making the decision to invent a tool, when they

really don’t have the empirical research to support their inventions. It’s like

inventing one’s own test for intelligence, reading skill, or cancer diagnosis; it is

a difficult and complicated process.

The LPS threat assessment instrument used on KP is located in Appendix 7.

The instrument included five sections (see also Exhibit 35 and ACSO, pp. 193-

196):

LPS’s Threat Assessment Instrument Sections

1. Make sure all students and staff are safe

2. Make immediate notifications

3. Review threat assessment factors (e.g., threat factors, early warning signs factors, at-risk factors, protective factors)

4. Review finding with building team and determine level of concern

5. Develop an action plan (e.g., safety measures, discipline and monitoring, notifications, and parent/guardian follow-up steps)

According to the available evidence, there is no research being conducted to

validate LPS’s current threat assessment tool, nor is there research being

conducted in the metro area on any other threat assessment tool. In addition,

according to the best available information, no Colorado schools are using an

empirically validated risk or threat assessment tool at this time. Examples of

validated risk and threat assessment tools, according to the research literature,

are SAVRYTM, RRCU, and V-STAG (described earlier).

The problem-solving approach taken by V-STAG and recommended by the

Secret Service is more appropriate because of the empirical research


83

supporting it and because it removes the need for schools to judge the level of

risk. In addition, this leads the threat assessment team to focus its time and

energy on what may be the most important part of the assessment process -

identifying and supporting a student in crisis. The true purpose of any threat

and risk assessment is to determine if a threat is posed and to design a set of

interventions and follow-up reassessments appropriate to the student and

school’s safety as well as the social emotional climate of the school.

Finding 2d: Threat Assessment Safety & Support Plan

At the time of the shooting and even in the year following the shooting, LPS’s

threat assessment protocol did not allow for the development of a separate

team to support students in crisis (see Exhibit 17: Administrative Review of LPS

Threat Assessment Protocols, June 2014). Instead, LPS and AHS tried to use

the threat assessment team of two people to provide the intervention and

support for KP, but they failed. They only conducted one follow-up meeting

with KP in the months after the September 9, 2013 threat assessment. The

following exchanged occurred during Kevin Kolasa’s (p. 148-149) deposition:

Mr. Michael Roche: So during this meeting on September 9, you told Karl and his parents that if he was having anger issues, he could come and talk to you, right? Mr. Kevin Kolasa: Yes. Roche: And you told him he could go and talk to Dr. Song? Kolasa: Uh-huh. Roche: And you told him he could go and talk to the other counselors in the building, whether that was Astrid Thurnau or Kelly Talen, when she got back from maternity leave? Kolasa: Correct. Roche: Did Karl ever do that to your knowledge? Kolasa: No.


84

Roche: Did you do anything to check whether Karl reached out to Esther Song or Astrid Thurnau or Kelly Talen to discuss his wellbeing? Kolasa: No, I don't recall. Roche: So would it be fair to say that you and Esther Song and the other counselors in the building were, in a sense, waiting for Karl to come to you if he had issues? Kolasa: Yes.

Later in his testimony, Kolasa (Deposition, p. 81) was asked:

Roche: When you saw that Karl had D or F grades in several classes in the fall of 2013, what did you do? Kolasa: I don't -- I don't remember it being several, but I remember him being on the D and F list, and I don't recall -because what Kelly Talen and I would typically do is sit together, the counselor, and we would go through and say, “Okay, you talk to these kids, I [will] talk to these kids,” because we were working hard to get [to] those kids to make sure that, you know, they could graduate on time. So, I don't recall whose responsibility it was to talk to Karl.

Kolasa (Deposition, p. 85) continued:

Roche: Well, can you point me to any action you took to investigate or ameliorate the situation that existed when Tracy Murphy came to you and said, “I'm concerned about Karl's grades?” Kolasa: I really don't remember what I did after that. I do know when Tracy said that, you know, “look at his grades.” And I forget exactly what he had at that time, but the classes that Karl was struggling in, I understood why he was struggling in those classes. And Karl also, in working with him the year previous, did have grades that would fluctuate a lot.

The work of the threat assessment team should inform the work of the

student’s ISST. In this case, there was no proactive follow-up on KP’s failing

grades, even when the assistant principal was informed about those grades.


85

Major Finding 3: Systems Thinking

This section catalogs the problems in systems thinking within AHS and LPS in

the months leading up to the shooting (see Senge, 1990). These findings

represent the most challenging and the most important of the problems to

solve, because information sharing and threat assessment cannot overcome an

unhealthy organizational system. According to research from a wide variety of

fields (e.g., the criminal justice system, hospitals, and aerospace engineering),

organizational errors do not occur as the result of one major mistake or one

bad apple employee (Dörner, 1996; Doyle, 2010). Instead, organizational errors

occur with “a small mistake here, and a small mistake there, and these mistakes

add up” (Dörner, 1996, p. 7). With a complex problem like school safety,

organizational errors prove difficult to resolve. Costa (2012, p. 179) suggests

that, under these conditions, “We need a short term plan to stay alive long

enough to have a permanent cure.” The findings indicate that, in the short

term, schools and districts should implement a continuous improvement model

of error review. In the long term, schools and districts should adopt Dörner’s

five steps for addressing the complex problem of school safety. The following

outlines the continuous improvement model of error review for the short term,

the characteristics of a complex problem for diagnosis and understanding, and

the five steps to solving a complex problem for the long term.

Finding 3a: Using a Continuous Improvement Model (Short

Term)

To foster a culture of safety, Doyle (2010) argues that organizations should

develop a continuous improvement model of error review. This model requires

routine investigation of and reflection upon major errors, near misses, and

other mistakes in the management of individual cases (e.g., wrongful

conviction, eyewitness misidentification, medical mistake, and school

shootings). Organizations unwilling to engage in error review perpetuate

groupthink (Doyle, 2010). Groupthink discourages workers from openly


86

reflecting upon and criticizing the work of superiors, co-workers, and the

group. Organizations must strike a balance between questioning on the one

hand and making decisions on the other. Two symptoms of groupthink are: (1)

the belief that failure is unacceptable and (2) open communication about

problems is discouraged. When asked about AHS’s perception of failure,

Tracy Murphy (Deposition, pp. 125-126) said:

You know, nobody likes bad news. . . We're a school of 2100 kids, you know, a large suburban high school. . . [W]e have to confront the fact that, you know, not every kid at Arapahoe High School is the cream of the crop. . . [I]t would be healthier at Arapahoe High School to . . . admit that, you know, it's not perfect here, that there's always room for improvement, that mistakes are made, and that you can learn from those mistakes. And we tell kids all the time that [but] sometimes I wonder how true it is, it's okay to fail, it's okay to make mistakes, but then we don't let them.

Mr. Murphy’s statement, and other case evidence, suggests that AHS’s climate

was unhealthy.

The second symptom of unhealthy schools and districts is when obstacles to

open communication exist between administrators, teachers, and students.

When Tracy Murphy repeatedly raised questions about KP’s concerning

behaviors (e.g., “penis” comment, threat to kill, and academic performance)

and the administration’s limited response, he was marginalized. In a discussion

with Kevin Kolasa about KP’s declining grades in November 2013, Murphy said:

So, I brought it to his attention. He told me he was aware of it, [and] the counselors were aware of it. I think I may have said something to the effect [of]. . . “This is a big red flag.” And he kind of shrugged his shoulders and kind of brushed it off saying, “Let him hang himself” kind of thing.

Murphy’s concern was not taken seriously, and he felt disappointed with and

even “astonished” by the way Kolasa responded. Murphy explained:

He had a 3.3, 3.4 average. He. . . was a B student. Now . . . he's running the risk of not graduating. That's huge. . .That's tremendous. . .[And Kevin Kolasa] let me know [that] he was aware of it [but] that was


87

pretty much it. . . [And] I was [disappointed and] kind of astonished [with his response].

Other teachers expressed similar concern with the administration’s openness

to feedback about the handling of KP and other students. When another

student exhibited concerning behavior in Spring 2014, Spanish teacher Victoria

Lombardi asked that Principal Natalie Pramenko and Assistant Principal Darrell

Meredith intervene, but she was disappointed with the response. When asked,

“[D]id that incident give rise to concerns in your mind that Mr. Meredith or

other members of the administration were still not taking student threats or

student safety as seriously as you would have liked?” Lombardi (Deposition, p.

80) said, “Yes. I was upset.”

In Tracy Murphy’s deposition (p. 134), Attorney Michael Roche asked, “Did you

agree with Mr. Kolasa's conclusion that Barbara Pierson's decision to keep KP

out of school for three days obviated the need for a suspension?” Murphy

said, “I wasn't really in a position to agree or disagree. All I could do was

accept it.” Mr. Murphy’s response indicates that he felt it was unacceptable to

voice his concern about the discipline applied. In fact, of the three teachers

deposed in this case, two of them – Tracy Murphy and Victoria Lombardi –

tried to question the administration’s response to KP; their concerns were

minimized. These findings indicate that AHS teachers were not encouraged to

question administrators, and when Murphy and Lombardi did question

administrators’ decisions, they were not taken seriously. Despite the fact that

the Columbine shooting that left 13 dead in 1999 happened just eight short

miles from AHS, the feeling that a “shooting could not happen here” appeared

to have remained prevalent among school staff and students.

A comparison of AHS teachers’ TELL Survey responses to questions about

AHS’s school leadership revealed a dramatic decline in teachers’ perceptions

of trust and respect from 2013 to 2015 (see http://www.tellcolorado.org/). In

2013, 93% of AHS teachers either agreed or strongly agreed that, “there is an


88

atmosphere of trust and mutual respect [with school leadership] in this

school.” By 2015, only 57% of AHS teachers agreed or strongly agreed with

that statement. It is important to recognize that high levels of trust may

reflect a positive organizational climate; however, when reviewed in the

context of deposition testimony and in comparison to other schools’ climate

data, it should be considered a possible symptom of groupthink. Too much

agreement is suspicious, as it may indicate that discussion and reflection on

errors is unacceptable. The extremely high level of trust in AHS leadership

(93%) in 2013 reinforces the finding that groupthink may have been in place at

AHS prior to the shooting; the dramatic decline in trust in AHS leadership after

the shooting suggests a shift in awareness and an awakening to a troubled and

dysfunctional school system. By comparison, in 2013, 67% of teachers in

Colorado high schools agreed or strongly agreed with the statement.

Recently, Principal Natalie Pramenko has become aware of the problems

between AHS administrators and teachers. In her deposition testimony,

Pramenko (Deposition, pp. 202-203) stated that she believed that

communication patterns would change once she became principal. She said:

[T]here's just been a culture for so long that we [teachers] don't talk to the administration. There was this wall [between teachers and administrators]. . . [And] I thought that wall just automatically came down when I became the principal, and it didn't. And I know I still have work to do. . . I think we're starting to build some trust amongst the staff . . . When the staff are in a better place and [there is] more trust. . . then the kids [benefit] as well. So, I think continuing to open those avenues of communication and showing the staff that we really are there for them [is important]. And I hope to keep doing that and keeping that culture alive, alive and moving in that direction.

AHS administrators, teachers, and students had systemic communication

problems. Danner and Coopersmith (2015) argue that organizations that adopt

a “failure is not an option” approach discourage open communication and

encourage deception and cover-up, leading to larger mistakes.


89

To combat the tendency for a “failure is not an option” approach,

organizations should routinely require error review, which improves the impact

of lessons learned, reduces individual and organizational resistance to error

investigation, and builds a culture of safety within the organization (Costa,

2012; Doyle, 2010). The discussions that followed a landmark study of 28

wrongful convictions corrected by DNA evidence in the criminal justice system

revealed that many prosecutors, detectives, and judges had grown

accustomed to seeing small mistakes within the system. These mistakes had

become a problem they lived with, not a problem they tried to solve (Doyle

2010). Doyle (2010) highlights the dramatic change in the ways that aviation

and medicine now conduct error reviews of airplane disasters and surgical

mistakes (respectively). The evidence suggests that LPS and AHS have been

reluctant to adopt a continuous improvement model of error review since the

shooting. When asked whether a “debriefing [or error review was ever] done

at either Arapahoe High School or at the district level about what went well

and what went not so well in the handling of Karl, threat assessments and his

other behavioral issues,” Superintendent Scott Murphy (Deposition, pp. 93-95)

said:

As far as I know, they did not, but they may have. . . I did not do any. . . We did [our debriefing] . . . through the sheriff’s office. We didn’t know all of the pieces. . . [But] I don’t recall receiving a document that said, “Here is a debriefing; here is what went well; here is what didn’t.” I don’t recall if there was [a critical review or a report done] along that line.

Without open dialogue about the mistakes in a tragic case like this one,

business continues as usual. When asked “[W]as [there ever] a debrief either

at Arapahoe or on a district-wide level on the efforts that occurred to try to

prevent the shooting and the handling of Karl?” Principal Natalie Pramenko

(Deposition, p. 192-193) explained:

Well, I don't think that . . . we've had specifically [a debrief] going backwards in time and trying to prevent it, but in moving forward [we’re] talking about changes in either practice or process to prevent it again from happening, so learning from what we've gone through. And I would say that's ongoing and continuous conversation.


90

Dörner (1996), Doyle (2010), and others argue that organizations cannot move

forward after a catastrophic error like the mishandling of KP at AHS without

first “going backwards in time” to do an error review. Organizations cannot

know what changes to make in practice or process without candidly

discussing and documenting what exactly went wrong and how it went wrong.

The dramatic decline in teachers’ trust in AHS leadership after the shooting

may have been the direct result of a failure to do a full debrief on the shooting

(http://www.tellcolorado.org/).

Nathan Thompson reported that LPS’s leadership team did not conduct a

review of the threat assessment process on KP until Spring of 2015, more than

a year after the shooting (Nathan Thompson Deposition, p. 227):

[T]he group I think came to [the] consensus that our [threat assessment] process is good, that . . . our training is effective. I guess effective is probably not the best word. Our training is appropriate. And based on, you know, the kind of current research and practice, and in terms of data and those things, we didn't have a specific comparison. So, we didn't have information to compare our district to other districts or anything like that. . . We definitely talked about John Nicoletti's four-stage model, which is part of what resulted in this, as well as some of our district-wide planning and district review team.

More than two years after the shooting, LPS continues to rely on a slightly

modified version of the threat assessment training protocol, data collection

process, and safety and support plan procedures used with KP. Costa (2012, p.

73) says that “When faced with complexity, our first response is to retreat to

the familiar, even if the familiar means failure.” During his deposition, Nathan

Thompson was asked if the execution of the threat assessment on KP raised

any concerns for him about LPS’s threat assessment training process.

Thompson (p. 230) explained:

I'm sure we had some conversation about the form and some about the training. Again, I think the general concepts of this is that it's a fine balance between trying to train people to do a form exactly perfectly and get the concept of what you're looking for. And so I don't think anybody in the room felt like, you know, spending hours and hours in a


91

training and making sure they know exactly how to fill out every box and does the evidence is the point of the training. The point of the training is: “Can we help people get those big principles? Do we help them understand how to get the information they need and where to look, what questions to ask?”. . . [But] this administrative review did not look at the details in depth of this incident. . . .It was not specific to this incident [with KP].

Unfortunately, AHS and LPS have not yet embraced a continuous

improvement model of error. Of course, the shift to a continuous

improvement model does not come easily, but it is critical to improving school

safety.

Finding 3b: Understanding a Complex Problem

The promotion of a safe culture and positive climate in large middle and high

schools represents a complex and ever-changing social problem. To develop a

longer-term solution, the complexity of the problem must be understood in

depth. Dörner’s (1996) four characteristics of a complex situation provide a

helpful framework for examining the findings on systems thinking in this case:

Dörner’s Four Characteristics of a Complex Situation

1. The complexity of the factors involved and the experience level of the

staff (e.g., identifying the problem, developing effective interventions, staff experience level)

2. The dynamic (or changing) nature of individual and organizational

problems 3. The intransparent (or unknown) information in the case

4. The presence of untested or mistaken hypotheses

Here, these four characteristics are applied to the evidence in this case.


92

Characteristic 1: Complexity of Factors

First, the complexity of the factors inherent in school safety can produce a

large number of different outcomes, requiring a greater need for decision-

makers to gather extensive information, critically evaluate that information,

and implement effective interventions. Research indicates that the level of

experience of those decision-makers can either increase or decrease the

complexity of the task (Dörner, 1996). Simply, more experienced staff will

have more success with decision-making and find the tasks less complex than

less experienced staff. As one example, the AHS staff who conducted the

threat assessment on KP had a two-hour training and very little practical

experience with the process. Prior to KP’s threat assessment on September 9,

2013, school psychologist Dr. Esther Song (Deposition, p. 21) had completed

“anywhere from five to ten” threat assessments and assistant principal Kevin

Kolasa (Deposition, p. 15) had completed two threat assessments. Thus, Kolasa

and Song had a theoretical understanding of the problems that threat

assessed students posed, but they could not really imagine the utility of the

assessment process or the potential harm a student could cause. The district’s

didactic (i.e., lecture and reading) approach to training did not help staff

imagine the harm.

Kolasa and Song did not have much experience in collecting information on a

student, critically evaluating that information, and conscientiously following up

with that student in the weeks and months after the assessment. In addition,

AHS and LPS had no systematic method for overseeing or evaluating the way

school staff conducted a threat assessment and follow-up plan. For example,

when asked whether any AHS staff were reprimanded or disciplined for

shortcomings in their threat assessment work with KP, Nathan Thompson (p.

61-62) said:

Not to my knowledge. . .[Because] my understanding [was that] there [were] not violations of district policy to the point of reprimand.


93

When LPS conducted an “Administrative Review of LPS Threat Assessment

Protocols” in June 2014 (see Exhibit 17, LPS 04049), LPS staff reported

multiple challenges with the threat assessment process, including:

• [It’s] difficult to choose a level of risk [because it] feels uncomfortable

• [The] mental health staff feel like they bear a hard burden in [threat

assessment] decisions

• [There are] issues with how to explain to parents and what to share or

not [to] share

• [There is] confusion about notifying teachers and other staff (who,

when, how)

• [There are a] lack of options when parents or students don’t want

mental health care

• [There are] logistic[al] challenges [with] implementing a tight

safety/supervision plan

LPS staff felt overwhelmed by the responsibility of assessing a student and

assigning a risk level, and these feelings may have contributed to a sense of

helplessness and even subtle cynicism about the value of the process. This

cynicism meant that key decision-makers never considered pursuing

alternative solutions to supporting and intervening with KP (e.g., calling

Safe2Tell, asking KP about his viewing of gun photos, enlisting KP’s

International Relations teacher in the support plan, and talking with KP’s

teachers).

In addition, LPS had no policy for either auditing the accuracy of the threat

assessment or providing feedback to threat assessment team members on

their work (see Nathan Thompson Deposition, pp. 62-63). This makes the

need for a well-trained standardized threat assessment team for each campus

critical to the reform of school safety (see also Erikson, 2001). The complex

nature of school safety protocols means that districts should provide an

accountability system for ensuring that each school follows the CSSRC’s


94

guidelines for the training and membership of the school’s threat assessment

team (e.g., eight hour training; membership - three core members, one

concerned teacher/staff, one supportive teacher/staff). The district should

also have a system for auditing the quality of each school’s threat assessment

procedures, outcomes, and safety and support plans. Regular external audits

should be conducted of schools and districts to monitor compliance with

threat assessment guidelines.

Characteristic 2: Changing Conditions

Second, the dynamic nature of individual and organizational problems

presents a challenge to decision making. Dörner (1996) argues that this fact

requires that organizational actors anticipate where the individual or situation

is going, not just where it is at one point in time. Consideration of possible

changes represents a difficult task, but it can be improved with the consistent

collection and analysis of individual and school level data. The individual data

can be obtained through threat and risk assessment tools and can help identify

the most appropriate evidence-based interventions to support a student in

crisis. The school data can be obtained through student and staff climate

surveys, and can determine the levels of bullying, drug use, violence, and trust.

The survey findings can help identify the most appropriate evidence-based

programs for the school’s needs, and when collected regularly, the survey

findings can also provide information on the effectiveness of those programs

in addressing the previously identified problems (see

http://www.colorado.edu/cspv/safeschools).

For individual level data, AHS and LPS did not have adequate systems in place

to recognize and evaluate the changing nature of a threat-assessed student’s

mental health, social isolation, support plan, and academic performance. In

KP’s case, consideration of the types of future events that may have hindered

his recovery could have included: (1) rejection from first choice university, (2)


95

rejection from his girlfriend, (3) decline in academic performance, and (4)

ridicule or rejection from peers. Thus, schools and districts should build in

continuous follow-up measures to support a threat-assessed student, to train

school staff on the early and imminent warning signs, and to monitor the

climate and culture of a school.

The assessment procedures in place within AHS and LPS approached the

threat assessment process as a one-time static event, an event that required

no additional data or information, an event that required no additional support

or intervention. However, both school climate and student success are

dynamic situations, which require continuous and conscientious monitoring,

support, and re-evaluation. Schools’ systems must be built to account for

changing safety conditions.

For school level data, AHS and LPS did not have adequate systems in place to

regularly collect and evaluate information on students’ and staff members’

perceptions of the climate. To monitor the culture of a school, Colorado’s

Department of Education recommends that schools conduct a climate survey

of students on an annual or biennial basis; these surveys measure the

frequency of bullying, presence of a code of silence, use of alcohol and drugs,

and perceptions of discipline practices (see

http://www.cde.state.co.us/pbis/measuringschoolclimatetoolkit). According to

deposition testimony and interrogatory responses, AHS had last completed a

school climate survey of students in 2010 (see Meredith Deposition, p. 142).

However, AHS administrators could not recall the findings from that survey or

the ways the findings were used to guide school programs and safety

interventions at AHS (Meredith Deposition, pp. 141-142; Pramenko Deposition,

p. 208). Natalie Pramenko (p. 208) said:

No, we haven’t [conducted a climate survey since I became principal], and I think that would be fabulous. . . It’s definitely been on my radar, [but]. . . I have never done one as principal.


96

Because the climate of a school can change from one year to the next, climate

surveys prove critical to capturing the dynamic nature of school settings.

School administrators can use the findings from a climate survey to inform

decision-making on what programs and services to offer students (e.g., drug

and alcohol awareness, bullying prevention, and suicide prevention). LPS did

not require schools to collect climate data or address climate findings. Nathan

Thompson (p. 82) explained:

I don’t know if it was four or five years ago [from 2015 but] the school improvement plans required a section [on] school climate and culture. . . When that template changed, there wasn’t as much data required to support [the question of] “How are you doing in that area?” So, that certainly did take away some of the . . . impetus to do that on an annual or every other year basis. In our district, it’s been established that it’s a site-based decision around when that happens.

To evaluate staff’s perception of a school’s climate, the Colorado Department

of Education uses the TELL Survey (http://www.tellcolorado.org/). TELL

measures teachers’ perceptions of and satisfaction with communication,

students, administration, and work. This survey, or a similar staff climate

survey, should be used to assess communication patterns, leadership, and

system quality at AHS and the findings should be used to improve conditions.

Characteristic 3: Intransparence

Third, intransparence refers to the fact that some of the information needed to

make a decision is not easily available or even apparent. Thus, intransparence

can create uncertainty in decision-making, and this uncertainty can lead to

feelings of helplessness and cynicism among staff. Certainly, when evaluating

the risk and threat for a student in crisis, a great deal of intransparence may

exist. School staff may not know the student’s true level of rage, parental

support, and mental state. School staff may also not know the climate of the

school. Thus, when some information is unknown, there is a greater need for a

skeptical investigative approach to the risk and threat assessment process

(see Fein, et al., 2002). In assessment meetings and follow-up discussions,


97

school staff must work hard to dig up as much information as possible about

the student and the situation and to thoughtfully reflect upon and question the

meaning of that information (e.g., dropping grades, viewing guns in cafeteria,

deep seeded rage). No one should assume that they know the meaning of the

information obtained (e.g., not a big deal, probably nothing). Instead, they

should assume that they do not know what that information means, they

should ask more questions, and they should collect more data. In short, they

should adopt an “investigative, skeptical, inquisitive, mindset” (Fein, et al.,

2002, p. 29).

In this case, Kevin Kolasa and Dr. Esther Song did not know many things about

KP during and in the weeks following their threat assessment of him, including

his diary entries describing his plan for the attack, his declining academic

performance in Spanish and other classes, his positive relationship with his

International Relations teacher Jeff Corson, and his lack of progress with his

outside therapist. The administrators making decisions about KP made fewer

decisions (e.g., did not gather more information on KP, did not reflect on the

meaning of KP’s declining grades, did not revisit threat assessment after

outburst in Spanish class), asked fewer questions (e.g., how is KP doing, why

are KP’s grades falling, what kind of therapy did he receive, why did KP ask for

an IQ test, what did his mom mean by a request for an IEP), and made faulty

assumptions. AHS administrators and counselors took everything they knew

about KP at face value, which we now know was not very much.

As is common with students in crisis, what school staff need to know about a

student may not be easily available, which can create uncertainty in the

assessment process and with the safety and support plan. In KP’s case, Kolasa

and Song’s uncertainty about how to actually conduct a threat assessment,

gather information in support of a threat assessment, and provide ongoing

support for a student in crisis appeared to have created a sense of


98

helplessness. Reports on school shootings and research on the school safety

have identified some of the strategies for reducing intransparence, but even

those strategies can feel overwhelming to school staff when they are already

stressed and ill-trained. Individual staff represent one element of a larger

systemic problem. In “Error in Medicine”, Lucian Leape (1994, p. 1852) has

explained:

While proximal error leading to an accident is, in fact, usually a ‘human error,’ the causes of that error are often well beyond the individual’s control. All humans err frequently. Systems that rely on error-free performance are doomed to fail.

Costa (2012) argues that when people do not have efficient processes for

thinking about and solving systemic problems, they feel drawn to simpler

explanations and behaviors. It is critical to the improvement of school safety

to avoid the usual response to this complex problem, including an irrational

opposition to new ideas, the personalization of blame, and a tendency for silo-

thinking.

Characteristic 4: Untested Hypotheses

Finally, Dörner (1996) argues that mistaken hypotheses represent the fourth

condition frequently present when organizations fail. He argues that

organizations need knowledge of the institution’s structure, current status, and

how certain actions will yield particular outcomes. Organizational actors need

to assume that the information they have is incomplete and that their

hypotheses about a case or a situation are incorrect. These allowances,

however, do not come naturally to organizational actors; these allowances

have to be cultivated and practiced. As the evidence in this report indicates,

AHS staff held countless untested hypotheses about KP, the school’s climate,

threat assessment, Safe2Tell awareness, and LPS policy (on FERPA and the

Student Code of Conduct).


99

The untested hypotheses among AHS and LPS staff in the months and weeks

prior to the shooting included that:

• Students would report safety concerns to school staff

• Students knew about Safe2Tell and how to use it

• Administrators and teachers understood the meaning and application of

FERPA

• LPS’s threat assessment tool provided a valid measure of risk and threat

• AHS and LPS could not make a mental health records release a

condition of return to school

• Teachers knew who to talk to about a student who exhibited concern

behavior

• Counselors and assistant principals knew how to conduct a threat

assessment

• KP would contact an assistant principal or counselor with any problems

or concerns

• KP’s falling grades signaled nothing about his wellbeing and success

AHS and LPS staff never really tested these (and other) hypotheses, and there

are countless ways that they could have tested them. They could have:

• Surveyed students on their comfort with reporting safety concerns

• Surveyed students on their knowledge of Safe2Tell

• Requested Safe2Tell data on the number of AHS concerns reported

• Discussed the staff’s knowledge of and concerns about FERPA

• Reviewed the research on validated threat assessment tools in school

settings

• Requested district permission to make a student’s re-admittance

conditional upon a release of medical records

• Asked teachers if they knew who to contact when they had concerns

about a student


100

• Contacted KP’s parents to inquire about his likelihood of reaching out to

an assistant principal or counselor for help

• Asked one of KP’s teachers to serve as his trusted adult within AHS

• Questioned KP about the meaning of his declining grades in November

When campus security personnel learned that KP was looking at photos of

guns and school shootings on his laptop in the AHS cafeteria and reported

that to an assistant principal, the assumption was that nothing could be done

about that event (e.g., no review of the student handbook, no search of KP’s

computer, no conversation with KP, no discussion with parents). No one

documented that event. No one questioned it. No one asked KP’s mom about

it. No one suggested or requested a second threat assessment.

After the perfunctory September 26, 2013 threat assessment follow-up

meeting with KP, Kevin Kolasa, Esther Song and Astrid Thurneau never

proactively reached out to KP. Not even once. Not when his grades began to

plummet in early November or when he had an enraged outburst in his

Spanish class on December 11th. When his outburst occurred in Spanish class –

with a teacher known for having fantastic classroom management skills (see

Pramenko Deposition, p. 132; Lombardi Deposition, pp. 32, 100) – it was viewed

as another example of an anger management problem, not an indicator of an

escalating anger management problem. They did not test numerous

hypotheses about KP and school safety.

Finding 3c: Solving a Complex Problem (Long Term)

In The Watchman’s Rattle, Rebecca Costa (2012) argues that the most

persistent and dangerous problems are systemic, and the solutions to a

systemic problem prove challenging to imagine, develop, and implement.

There are cognitive limits to what we can understand; sometimes the big


101

picture is just too big.16 The findings from this report – and other reports on

school shootings – reveal that traditional problem-solving methods are no

longer sufficient for addressing the complex problem of school safety (see

Costa, 2012). Of course, it is not going to be easy. The solution will require that

districts develop policies to clarify responsibilities for information sharing (e.g.,

Safe2Tell training), threat assessment, and error review.

The evidence indicates that LPS had the “expectation” that school

administrators promote safety but they had no district-level mandates or

accountability measures for ensuring that this expectation was met. Site-

based decision-making affords school administrators a great deal of

autonomy, but it does not make for a consistent and strategic approach to

school safety. As one example, site-based decision-making on the

identification of and support for students in crisis created problems with the

execution of threat assessments at AHS. No systematic, clear procedures or

protocol existed in LPS for: (1) the training of school personnel (e.g., assistant

principals, counselors, teachers, and security personnel) in threat assessment,

(2) the execution and auditing of a threat assessment with a student in crisis

(e.g., did you ask this question for this indicator?, did you use the six key

criteria or 11 key questions?), and (3) the safety and support plan follow-up for

a threat assessed student. Admittedly, it is difficult to mandate such policies

at the district level, as school administrators, counselors and others feel

overwhelmed by other demands (e.g., educational testing, educational

intervention evaluations). In addition, organizational leaders and staff may

express resistance to the implementation of new practices and an “irrational

opposition to new ideas” (Costa 2012, p. 73). Certainly, the view that mental

16Costa (2012) suggests that when the complexity of the social problem exceeds the ability of the human brain, five

supermemes(i.e.,apervasivethought,belief,orbehaviorthatcontaminatesorsuppressesallotherbeliefsandbehaviors

insociety)develop,including:(1)anirrationaloppositiontonewideas,(2)thepersonalizationofblame,(3)counterfeit

correlations,(4)silo-thinking,and(5)extremeeconomics(i.e.,acost/benefitanalysisthatvaluesprofitmorethanhuman

progress). Silo-thinking (i.e., compartmentalized thinking and behavior) can lead humans to oversimplify the problem

intosmallmanageabletasks.


102

health and threat assessments are not something that schools should handle

represents one example of an irrational opposition.

No one at AHS or LPS has ever said that they made a mistake or that they may

have failed in their duty. In fact, AHS and LPS did not conduct a debrief on the

threat assessment process in KP’s case until Spring 2015, more than a year

after the shooting. Following the shooting, the district appears to have taken

a defensive “we did the best we could,” not an inquisitive and reflective “what

can we learn from this tragedy” response.

Dörner’s (1996, p. 43) five steps for solving a complex problem prove helpful

for addressing school safety and violence prevention:

Dörner’s Five Steps for Solving a Complex Problem

1. Formulating goals 2. Formulating models and gathering information 3. Predicting and extrapolating 4. Planning actions, making decisions about actions, and executing actions 5. Reviewing the impact of actions and revising goals, models and actions

(see Appendix 9: CSPV’s Safe Communities Safe Schools Model)

Step 1: Formulate the Goal

In the months and years ahead, AHS, LPS and Colorado lawmakers will need to

establish a goal for school safety in Colorado. “Clear goals give us guidelines

and criteria for assessing the appropriateness or inappropriateness of

measures we might propose” (Dörner, 1996, p. 44). The goal should be to

create a “culture of safety” within schools and districts. A culture of safety (1)

stays educated about its current knowledge in the discipline; (2) encourages


103

the reporting of errors; (3) fosters an environment of trust; (4) remains

amenable to changing demands and organizational structures; and (5) learns

from and adjusts safety systems (Doyle, 2010, citing Reason, 1997).

Step 2: Gather Information and Formulate Models

The second step involves collecting information and building models.

Research consistently identifies two critical approaches to school safety and

violence prevention: (1) building effective intelligence gathering systems (at

the student, staff, and school level) to support the identification of students in

crisis (Elliott, 2009) and assess school climate (Hernandez & Seem, 2004; Fein,

et al., 2002) and (2) creating and maintaining a safe and positive school

culture (Elliott, 2009; Thapa, et al., 2012). A data collection system helps

identify the school’s climate and safety needs, provide early identification of

students in crisis, and monitor the effect of programs and trainings on school

safety outcomes (e.g., Safe2Tell, bullying prevention programs, drug use,

suicide, bullying, suicide prevention programs). This data can be used to build

a model that promotes and sustains a positive school climate.17

CSPV’s Safe Communities Safe Schools Model – which will be tested in 32

Colorado communities over four years through a National Institute of Justice

funded study – encourages the development of an Effective Intelligence

Gathering System (EIGS) at both the school and student level to use data to

prevent violence and other problem behaviors (e.g., drug use, suicide, bullying)

(Elliott, 2009). The data gathered informs programming, training, and

interventions at the school, staff, and student levels. In addition, given the

communication and systemic problems within AHS and LPS, survey data

should be collected from school and district staff on the morale, attitudes, and

communication of administrators, teachers, and staff.

17Apositiveschoolclimate includes:(1)astrongacademicorientation, (2)respectful teachersandpeers, (3)students’

positiveattitudesaboutschool,(4)perceivedrewardforeffort,(5)respectforauthority,(6)acleanandorderlycampus,

(7)highteachermorale,and(8)clearlyandfairlyenforceddisciplinarypolicies(Elliott,2009).


104

Step 3: Predict and Extrapolate

Third, when enough information is obtained about the current state of support

services and school climates (i.e., how do things look now?), the next question

is – what is likely to happen next? Using a Strengths, Weaknesses,

Opportunities, and Threats (SWOT) Analysis (www.mindtools.com),

organization leaders can sift through this data and prioritize the organization’s

needs. The first phase is to extract information learned from the data

collected (e.g., student climate survey, staff climate survey) to review, honor,

and prioritize the organization’s strengths and achievements. The second

phase is to extract information on the weaknesses revealed in the data and

prioritize those weaknesses. The third phase is to extract information on

opportunities and prioritize those opportunities. Finally, review the threats

found in the data and prioritize those threats. This analysis allows

organizations to anticipate the problems and opportunities on the horizon.

Step 4: Plan, Make Decisions, Execute

The fourth step involves making decisions about how to achieve the

organization’s goals and develop plans for action and implement those actions.

This is a program planning step, where each of the identified SWOT priorities

get analyzed. Here, organizations can make and implement data-based

decisions about short term tactics and long term strategies for success.

Step 5: Review the Effect of Actions and Revise the Goals, Models & Actions

Finally, once actions become reality, self-reflection and critique should follow,

and the results of that self-reflection and critique should inform the revision of

each step in the process, starting with the revision of the organization’s goals.

Prior research and the evidence presented here offers strong support for

schools and districts to move from an adversarial to a continuous

improvement model of error review for the promotion of school safety and the

prevention of school violence.

PART 3: RECOMMENDATIONS FOR REFORMS


105

Recommendations For Policy & Legislative Reforms

The goals of the arbitration were to provide information on how to identify

students in crisis, support students in crisis, and develop protocols for

responding to students in crisis. To reach these goals and to help prevent

future tragedies, schools and districts must first build safe school climates (see

Fein, et al., 2002). A safe school climate is one where “students view teachers

as being fair, the rules are universally enforced and students feel welcome, are

engaged in activities and know a teacher they can talk to about a problem”

(Elliott, 2009, p. 54). These recommendations seek to promote safety and

prevent violence in all school settings (Nekvasil & Cornell, 2015). While the

findings come from AHS and LPS, the recommendations may apply to many

schools and districts in Colorado.

The institutional barriers within schools, districts, and our culture will need to

be dismantled, including the belief that schools are powerless to manage

mental health issues. Schools can manage mental health and social support

issues. The task is complicated but it is not impossible. The promotion of

school safety will require the implementation of multiple mitigations in parallel.

Costa (2012) calls this “parallel incrementalism,” a mitigation strategy whereby

the cumulative effect of several incrementally useful strategies implemented in

parallel is exponentially more effective than one strategy implemented at a

time.


106

Recommendations on Finding 1: Information Sharing

u Indicates a Major Recommendation

Consistent Use of and Shared Access to Infinite Campus

1. Recommend that principals, assistant principals, teachers18

, counselors, psychologists, coaches, and SROs consistently use a student information system

(e.g., Infinite Campus) to document matters of a “public safety concern,” including student behavior concerns, conduct violations, interventions, academic

concerns, threat assessment results, and safety and support action plans. u

2. Recommend that principals, assistant principals, teachers, counselors, psychologists, coaches, and the SRO have access to information on a need to know

basis about a student’s behavior and discipline history in the school’s student information system (e.g., Infinite Campus), particularly when a public safety

concern exists.

3. Infinite Campus should serve as the primary tool for documenting and reviewing any and all public safety concerns raised about a student; it should serve

as the threat assessment team’s database.

4. If a threat assessment has been conducted on a student, the assessment information should be recorded in the student information system, along with the

safety and support action plan. For example, if a student is known to have had suicidal ideation or threats, the assessment results in the system could

inform coaches and teachers to watch for dropping grades, angry outbursts, social withdrawal, or isolation.

5. Through the student information system, all of the student’s teachers, coaches, as well as campus security personnel (e.g., principal, assistant principals,

campus security officers, and SROs), should receive active notification of a student’s threat assessment, disciplinary action, and the behaviors giving rise

to the threat assessment, along with potential warning signs of a reoccurrence (e.g., email, letter).

6. All school staff have a duty to report to the student’s information vortex coordinator (or case manager), identified in the student information system, the

occurrence of any warning signs, risk factors, and threat factors subsequent to a threat assessment.

7. In district policy, school staff manuals, student conduct codes, and staff trainings, the meaning and application of FERPA should be clearly communicated

to district staff, school staff, students, and parents, particularly as it relates to students in crisis and public safety concerns. FERPA permits the sharing of

student information when a “public safety concern” arises.

Train in and Promote Safe2Tell Among Students and Staff

8. Recommend that schools and districts promote Safe2Tell in formal trainings to students and staff each year, using skills practice, one-on-one feedback,

and coaching (see www.Safe2Tell.org and Appendix 6: Skills Training with Guided Practice) and emphasizing the three core principles*:

a. No one will know; Safe2Tell is an anonymous reporting system.

b. When someone could be hurt or injured, you have a duty to report the concern to authorities and break the code of silence.

c. Safe2Tell is not limited to student reporting; the system is available to all students, teachers, parents, staff, and community members, and they also

have a duty to report any safety concern to either authorities or Safe2Tell. u

9. Recommend that schools and districts advertise and promote Safe2Tell – throughout the academic year and during breaks – in morning announcements,

school news bulletins, email blasts, and email signature lines to students, staff, and parents.

Implement an Interagency Information Sharing Agreement

10. Recommend that school districts complete an Interagency Information Sharing Agreement with community agencies, including law enforcement agencies,

mental health service providers, social services agencies, and the criminal justice system, as recommended by the Columbine Review Commission, stated

in C.R.S. § 22-32-109.1(3), and outlined by the Colorado Attorney General’s Office. To facilitate this reform, it is recommended that the words “if possible”

be removed from C.R.S. § 22-32-109.1(3). u

11. To clarify for all schools and districts, the Colorado Attorney General’s Office should prepare an AG opinion on the application of FERPA to information

sharing within schools, districts, and to external agencies.

18

ItisimportanttonotethatAHSaddeda“FacultyTab”toInfiniteCampusinJanuary2014toallowteachers(butnotSROs)todocumentstudentconcernsandparentcommunications

(seeMeredithDeposition,p.200).


107

Recommendations on Finding 2: Threat and Risk Assessment

Overall, a threat assessment and risk assessment process should be used to develop a plan of action to manage the student who makes a threat in the most comprehensive way possible, provide appropriate social/emotional components, ensure the safety of the student and others, and reassess whenever a new risk, threat, or protective factor appears. The action plan should include components that address all three typologies of aggressive and violent behavior (e.g., traumatization, psychotic, or psychopathic) if appropriate. In this way, schools are not diagnosing (or labeling) students with a mental health issue; instead,

they are taking those possible conditions into account when developing a safety and support action plan for the student in crisis.

Consistently Implement Threat Assessment Guidelines

12. Recommend that, during a threat assessment, the Secret Service’s six principles and 11 questions be used to gather and evaluate the early warning signs, threat factors, risk factors, and protective factors. The process should emphasize an “investigative, skeptical, inquisitive mindset” for each factor until a

clear yes or no is found (Fein, et al., 2002, p. 29). All threat assessment team members, and if needed the ISST members and peers, should be included in the process (see Appendix 3). u

13. Recommend that the CSSRC define the similarities and differences in the responsibilities of the Multijurisdictional Threat Assessment Team (MTAT) and the Interagency Social Support Team (ISST) in the threat assessment and support process.

19 In some cases, the two teams are combined, but their

responsibilities in the threat assessment and support process should be distinct.

14. Recommend that schools and districts develop a threat assessment manual which clarifies the definition of each factor with anchors (i.e., concept definitions and/or behavior statements) for assessors to better determine the presence of each factor (e.g., V-STAG). These factors include 10 threat factors, 6 early warning sign factors, 20 at-risk factors, and 5 protective factors.

15. Recommend that an information vortex coordinator (from the threat assessment team) be assigned to every threat assessed student; the information vortex coordinator should be noted in the student’s profile within the student information system so that when a concern arises, all teachers and other staff can easily identify and communicate with the coordinator. In addition, it should be the proactive duty of the information vortex coordinator to continue to seek out and evaluate information about a threat assessed student and recall the threat assessment team if new risk or threat factors are

revealed. u

16. Recommend that schools and districts conduct an internal audit of their threat and risk assessment processes and report the findings to the school board

on an annual or biennial basis.

17. When additional factors (e.g., risk, threat, or protective) appear or become known about a student, the threat assessment team should be reconvened and a new threat assessment should be conducted and updated and the ISST should develop a revised safety and support action plan.

18. Recommend that the Colorado School Safety Resource Center (CSSRC) audit any school or district requesting an audit for proper use of V-STAG (or other validated threat and risk assessment process). Any school or district that has implemented a validated process and receives a “high pass” in an audit of that process could use the results as an affirmative defense in any proceeding under SB 15-213. The audit process and implementation guidelines

should be reviewed by CSPV. u

19. Recommend that the threat assessment and support teams produce a formal safety and support plan for every threat assessed student, relying on

Individual Educational Plans (IEP) and Student Intervention Teams (SIT) as models. ISSTs build and monitor the plan for threat assessed students and revise the assessment and plan whenever a new threat or risk factor appears (see Appendix 3: Child in Crisis Assessment Recommendation). u

20. Recommend that each threat assessed (or red flag) student be paired with an adult in authority, ideally within the school, who can build a trusting and positive relationship with that student. u

19TheInteragencySocialSupportTeam(ISST)isdiscussedintheCSSRCGuidelinesandtheColumbineReviewCommission(Erickson,2001).HB04-1451maybethevehicleforthisteam.

Usingmodelsforacademicintervention,suchasIndividualEducationalPlans(IEP)andStudentInterventionTeams(SIT)19,ISSTsbuildandmonitortheplanforthreatassessedstudents

(seeAppendix3).


108

Recommendations on Finding 2: Threat and Risk Assessment Continued

21. Recommend that schools and districts train in a validated threat and risk assessment process using a one-on-one cognitive behavioral training standard

(see Appendix 6). Adopt a formal training curriculum for threat and risk assessment. Train all teachers and staff in the overall process, and train

principals, assistant principals, counselors, and SROs in a minimum of one-day hands-on scenario driven training curriculum. u

22. Recommend that principals, assistant principals, and counselors receive certification in threat and risk assessment processes by CSSRC at least once

every three years.

23. Security staff within the district should become active members of The Colorado Association of School Security and Law Enforcement Officials

(CASSLEO) to maintain and improve threat and risk assessment protocols, using information obtained from other schools. Active membership within a

“community of practice” improves current knowledge and improves organizational operations.

Implement Validated Risk and Threat Assessment Tools

24. Recommend that schools and districts install a validated threat assessment process, by either using the Virginia Student Threat Assessment Guidelines

(V-STAG), by using a different validated threat assessment process, or by validating the current threat assessment process with similar outcome

measures to V-STAG (see Appendix 8). u

25. Recommend that schools and districts install a validated risk assessment process, such as the Structured Assessment of Violence Risk in Youth

(SAVRYTM

) or the Risk and Resiliency Check Up (RRCU). Use the results from the risk assessment to build a safety and support plan for any student who

has a threat assessment. Risk assessments incorporate both risk and protective factors in the plan for the student. u

Recommendations on Finding 3: Systems Thinking

26. Recommend that the Attorney General annually update the Colorado School Violence Prevention and School Discipline Manual on school

safety statutes, FERPA, and their application to school districts. Additionally, recommend that school districts conduct an annual training

on all statutes related to school safety and violence prevention and produce an annual compliance report. u

27. Recommend that schools and districts create a continuous improvement model of error review committee to promote a culture of safety

(and minimize groupthink), whereby staff can report concerns about organizational errors and near misses and staff can openly discuss,

reflect upon, and address concerns and mistakes without formal or informal penalty. This committee should help develop short and long

term plans for school safety reform. Dörner’s (1996) five steps can help with long term planning. u

28. Recommend that schools and districts conduct an established school climate survey of students and staff every one to two years and

when the findings exceed established norms, select and implement experimentally proven interventions, programs, and practices. u

29. Recommend a formal debrief of what went well and what did not go well in violence prevention and threat assessment after any school

shooting or school violence event, giving all system actors the opportunity to participate. This represents a critical recommendation for

AHS and LPS.

30. Recommend that CSSRC review this report and coordinate a working group to improve information sharing, threat assessment, and

systems thinking in school safety and violence prevention.

31. Recommend that CSSRC help districts establish an accountability system for ensuring that each school follows their guidelines for the

training and membership of threat assessment and Interagency Social Support Teams (ISST).

32. Recommend that districts develop policies to clarify school site’s responsibilities for information sharing (e.g., Safe2Tell training, FERPA,

Interagency Information Sharing Agreement), threat assessment, and error review.

REFERENCES


109

Adverse Childhood Experiences Reported by Adults—Five States (2009).

Morbidity and Mortality Weekly Report CDC, December 17, 2010 /

59(49); 1609-1613.

Anda, R. F., Felitti, V. J., Bremner, J. D., Walker, J. D., Whitfield, C. H., Perry, B.

D., & Giles, W. H. (2006). The enduring effects of abuse and related

adverse experiences in childhood. European Archives of Psychiatry and

Clinical Neuroscience, 256(3), 174-186.

Blair, J. P., Martiandale, M.H., & Nichols, T. (2014). Active Shooter Events from

2000 to 2012. FBI Law Enforcement Bulletin, January.

Bondu, R. & Scheithauer, H. (2011). Explaining and preventing school shootings:

Chances and difficulties of control. In W. Heitmeyer, et al. (Eds.), Control of

Violence (pp. 295-314). NY: Springer.

Borum, R., Cornell, D.G., Modzeleski, W., & Jimerson, S.R. (2010). What can be

done about school shootings? A review of the evidence. Educational

Researcher, 39(1), 27-37.

Borum, R., Lodewijks, H., Bartel, P. A., & Forth, A. E. (2010). Structured

Assessment of Violence Risk in Youth (SAVRY).

Chassin, M. R. & Becher, E.C. (2002). The wrong patient. Annals of Internal

Medicine, 136, 826-833.

Cornell, D.G. (1990). Prior adjustment of violent juvenile offenders. Law and

Human Behavior, 14, 569-577.

Cornell, D.G., Allen, K., & Fan, X. (2012). A randomized controlled study of the

Virginia Student Threat Assessment Guidelines in kindergarten through

grade 12. School Psychology Review, 41(1), 100-115.

Cornell, D.G., Sheras, P.L., Gregeory, A., & Fan, X. (2009). A retrospective study

of school safety conditions in high schools the Virginia Threat Assessment

Guidelines versus alternative approaches. School Psychology Quarterly,

24(2), 119-129.


110

Cornell, D.G., Sheras, P.L., Kaplan, S., McConville, D., Douglass, J., Elkon, A,

McKnight, L., Branson, C., & Cole, C. (2004). Guidelines for student threat

assessment: Field-test findings. School Psychology Review, 33(4), 527-546.

Costa, R. (2012). The Watchman’s Rattle: A Radical New Theory of Collapse.

Philadelphia, PA: Vanguard Press.

Danner, J. & Coopersmith, M. (2015). The Other “F” Word: How Smart Leaders,

Teams, and Entrepreneurs Put Failure to Work. Hoboken, NJ: Wiley & Sons.

Denver Post. 2014. “Why Were Warning Signs Ignored Before Arapahoe

Shooting?” October 13, 2014.

http://www.denverpost.com/editorials/ci_26720829/why-were-warning-

signs-ignored-before-arapahoe-shooting?source=pkg.

Denver Post. 2014. “Phony Excuse for District’s Silence on Arapahoe

Shooting.” October 18, 2014

http://www.denverpost.com/editorials/ci_26748502/phony-excuse-

districts-silence-arapahoe-school-shooting?source=pkg.

Dörner, D. (1996). The logic of failure: Recognizing and avoiding error in

complex situations. Reading, MA: Addison-Wesley.

Doyle, J. M. (2010). Learning from error in American criminal justice. The

Journal of Criminal Law and Criminology, 100: 109-147.

Elhai, J. D., Layne, C. M., Steinberg, A. M., Brymer, M. J., Briggs, E. C., Ostrowski,

S. A., & Pynoos, R. S. (February, 2013). Psychometric properties of the UCLA

PTSD reaction index. part II: investigating factor structure findings in a

national clinic-referred youth sample. Journal of Traumatic Stress, 26(1), 10-

8.

Elliott, D. (2009). Lessons from Columbine: effective school-based violence

prevention strategies and programmes. Journal of Children's Services, 4(4),

53 – 62.

Elliott, D. & Kingston, B. (2013). Statement for the youth violence summit.

Washington, D.C.: U.S. House of Representatives.


111

Erickson, W. (2001). The Report of Governor Bill Owens’ Columbine Review

Commission. Denver, CO: The State of Colorado.

Fein, R.A., Vossekuil, B., Pollack, W.S., Borum, R., Modzeleski, W., & Reddy, M.

(2002). Threat Assessment in Schools: A Guide to Managing Threatening

Situations and to Creating Safe School Climates. Washington, D.C.: United

States Secret Service and United States Department of Education.

Gardner, W., Lucas, A., Kolko, D. J., & Campo, J. V. (2007). Comparison of the

PSC-17 and alternative mental health screens in an at-risk primary care

sample. Journal of the American Academy of Child & Adolescent

Psychiatry, 46(5), 611-618.

Gladwell, M. (October, 2015). Thresholds of violence: How school shootings

catch on. The New Yorker. Retrieved from

http://www.newyorker.com/magazine/2015/10/19/thresholds-of-violence

Goodman R., Meltzer, H, & Bailey, V. (1998). The Strengths and Difficulties

Questionnaire: A pilot study on the validity of the self-report

version. European Child and Adolescent Psychiatry, 7, 125-130.

Hernandez, T.J., & Seem, S.R. (2004). A safe school climate: A systemic

approach and the school counselor. Professional School Counseling, 7(4),

256-262.

Kelleher, I, Harley, M., Murtagh, A. & Cannon, M. (2011). Are screening

instruments valid for psychotic-like experiences? A validation study of

screening questions for psychotic-like experiences using in-depth clinical

interview. Schizophrenia Bulletin, 37, 362-369.

Kingston, B. (2013). Let’s make it safe to tell. Education Development

Center. Retrieved from

http://preventingbullying.promoteprevent.org/let%E2%80%99s-make-it-

safe-tell.

Langman, P. (2009). Rampage school shooters: A typology. Journal of

Aggressive and Violent Behavior, 14, 79-86.


112

Langman, P. (July 2014). Rampage school shooters: A typology.

www.schoolshooters.info, Version 1.1.

Leape, L. (1994). Error in medicine. Journal of the American Medical

Association, 272(23), 1851-1857.

Loewy, P., Vinogradov, B. & Cannon, T. D. (2011). Psychosis risk screening with

the prodromal questionnaire- Brief version (PQ-B). Schizophrenia

Research, 129(1):42-6.

Meloy, J. R., Hempel, A.G., Thomas, B. G., Mohandie, K., Shiva, A. & Richards, T.

(2004). A comparative analysis of North American adolescent and adult

mass murders. Behavioral Sciences and the Law, 22, 291-309.

Meloy, J. R., Hoffmann, J., Guldimann, A., & James, D. (2012). The role of

warning behaviors in threat assessment: An exploration and suggested

typology. Behavioral Sciences and the Law, 30, 256-279.

Nekvasil, E.K. & Cornell, D.G. (2015). Student threat assessment associate with

safety in middle schools. Journal of Threat Assessment and Management,

doi: 10.1037/tam0000038.

Newman, K., Fox, C., Hardin, D.J., Mehta, J., & Roth, W. (2004). Rampage: The

social roots of school shootings. NY: Perseus Books.

O’Toole, M. (2000). The school shooter: A threat assessment perspective.

Quantico, VA: Critical incident response group, National Center for the

Analysis of Violent Crime, FBI Academy.

Pollack, W.S., Modeleski, W. & Rooney, G. (2008). Prior Knowledge of

Potential School-Based Violence: Information Students Learn May Prevent a

Targeted Attack. Washington, D.C.: U.S. Secret Service and Department of

Education.

Rappaport, N. & Thomas, C. (2004). Recent research findings on aggressive

and violent behavior in youth: Implications for clinical assessment and

intervention. Journal of Adolescent Health, 35, 260-277.

Reason, J. (1990). Human error. NY: Cambridge University Press.


113

Reddy, M., Borum, R, Berlund, J, Vossekuil, B. Fein, R., & Modzelski, W. (2001).

Evaluating risk for targeted violence in schools: Comparing risk assessment,

threat assessment, and other approaches. Psychology in the Schools, 38(2),

157-172.

Substance Abuse and Mental Health Services (SAMHSA). (2015). National

registry of evidenced-based programs and practices (NREPP): Virginia

Student Threat Assessment Guidelines. Retrieved from

http://youthviolence.edschool.virginia.edu/threat-assessment/home.html.

Sandy Hook Advisory Commission. (2015). Final Report. March 6, 2015.

REtreived from

http://portal.ct.gov/Search.aspx?q=Sandy%20Hook%20advisory%20report

Schonfeld, D. (October, 2015). Colorado Safe Schools Summit. Thorton,

Colorado.

Singh, Jay P., Grann, M., & Fazel, S. (2011). A comparative study of violence risk

assessment tools: A systematic review and metaregression analysis of 68

studies involving 25,980 participants. Clinical Psychology Review,

doi:10.1016/j.cpr.2010.11.009.

Senge, P.M. (1990). The fifth discipline: The art & practice of the learning

organization. NY: Doubleday Press.

Steinberg, A. M., Brymer, M. J., Kim, S., Briggs, E. C., Ippen, C. G., Ostrowski, S.

A., Gully, K. J., & Pynoos, R. S. (February, 2013). Psychometric properties of

the UCLA PTSD reaction index: part I. Journal of Traumatic Stress, 26(1):1-9.

Thapa, T., Cohen, J., Higgins-D’Alessandro, A. & Guffey, S. (2012). School

climate research summary. National School Cimate Center. Retrieved from

https://www.schoolclimate.org/climate/documents/policy/sc-brief-v3.pdf.

Turner, Susan, Terry Fain and Amber Sehgal. Validation of the Risk and

Resiliency Assessment Tool for Juveniles in the Los Angeles County

Probation System. Santa Monica, CA: RAND Corporation, 2005.

http://www.rand.org/pubs/technical_reports/TR291.html.


114

U.S. Department of Health and Human Services. (2001). Youth Violence: A

Report of the Surgeon General. Rockville, MD: U.S. Department of Health

and Human Services, Centers for Disease Control and Prevention, National

Center for Injury Prevention and Control; Substance Abuse and Mental

Health Services Administration, Center for Mental Health Services; and

National Institutes of Health, National Institute of Mental Health.

Verlinden, S., Hersen, M., & Thomas, J. (2000). Risk factors in school settings.

Clinical Psychological Review, 20(1), 3-56.

Virginia Tech Review Panel. (2007). Mass shootings at Virginia Tech.

Blacksburg, VA.

Vossekuil, B., Fein, R.A., Reddy, M., Borum, R. & Modzeleski, W. (2002). The

final report and findings of the Safe School Initiative: Implications for the

prevention of school attacks in the U.S. Washington, DC: U.S. Secret Service

and Department of Education.

APPENDIX 1: CHRONOLOGICAL LIST OF KP’S CONCERNING BEHAVIORS

Report on the Arapahoe High School Shooting 115

Sandberg Elementary School Brief Description Action Taken Noted In Who Knew Exhibit

11/24/03 Hit students with lunch box because they weren’t fast enough in lunch line; asked to write an apology letter

Hit peers with lunchbox

Required to write apology

IC-BDR ES 24

12/18/03 Kicked student in stomach and hit another student in head; asked to write an apology letter

Kicked and hit peers

Required to write apology

IC-BDR ES 24

Arapahoe High School (2011-12) Sophomore Year Brief Description Action Taken Noted In Who Knew Exhibit

11/16/11 Told peer to just “go cut yourself” in Jackie Price’s class Told peer “go cut

yourself”

Called father IC-CL JP, KT, ES 19

11/28/11 Told Jackie Price “he has always been someone’s bitch” and other kids are mean to him; said “why wouldn’t I

make him my bitch after that has been done to me?”; was “extremely angry” in meeting

“Make him my bitch”

Held meeting; discussed anger

management

IC-CL JP, KT, ES 19

Arapahoe High School (2012-13) Junior Year Brief Description Action Taken Noted In Who Knew Exhibit

3/15/13 Yelled “fuck” in response to C- grade in Dan Swamley’s class; said “teachers out to get me” and “my peers have

often pushed me. . . one outburst for a decade of hell is unfair”; signed statement “Ides of March”

Yelled “fuck” in math

Met with KK; suspended for one

day

Not noted in IC-BDR

or IC-CL ; hardcopy

DS, KK 32

Date Unknown

Opened with the statement “I woke up this morning and realized my penis had fallen off” in a debate competition

“Penis” line in speech

None not noted in IC

TM 34 (p. 4-5)

Arapahoe High School (2013-14) Senior Year Brief Description Action Taken Noted In Who Knew Exhibit

8/11/13

Ran stop sign, hit another car, and totaled car after leaving work angry

Totaled car Mother reported during threat

assessment

Not noted BP, ES, KK 33

8/21/13 Told another student “that’s stupid” and verbally bullied” classmates in Jeff Corson’s class; Corson consulted with

Murphy about problem

Bullied peers verbally

JC consulted TM; JC enlisted KP as

expert

Not noted JC, TM 13

9/3/13 Removed as captain of the AHS Extemporaneous Team

of the Speech and Debate by Murphy during meeting with mother; did not respond well; stared at Murphy with a “haunting” look and was later heard yelling “I’m going

to kill that guy [Murphy]” in the parking lot by Mark Loptien

Yelled “going to

kill” Murphy

Mother kept home

for 3 days; threat assessment scheduled for

9/9/13; no suspension

IC-BDR;

TA and Action Plan

TM, ML, DM,

KK, ES, JE

19, 35

9/5/15 Documented threat with ACSO Threat noted in

police report

ACSO Report ACSO

Report

TM, ML, DM,

KK, ES, JE

18

9/9/13 Assessed for threat by Kevin Kolasa and Esther Song with parents (Mark and Barbara Pierson) present;

described as apologetic but not remorseful; labeled a “low risk”; requested to not attend speech and debate meetings for 2-3 weeks

AHS threat assessment

performed

Not permitted to attend speech and

debate practices

IC-CL; IC-BDR; TA

and Action Plan

KK, ES, BP, MP

19, 24, 35

9/9/13 Assessed at Highland Behavioral Health; described as not a threat to self or others

Private mental health assessment performed

None TA and Action Plan

KK, ES, BP 35

9/10/15 Disregarded Kolasa’s request to not attend speech and debate practice; asked to leave practice by Murphy

Boundary probing Asked to leave Not noted TM, KK Murphy Depo, p.

170-1

APPENDIX 1: CHRONOLOGICAL LIST OF KP’S CONCERNING BEHAVIORS CONTINUED


9/--/13 Received F on Michelle Crookham’s math test and wrote “KMFDM” on top of test, referring to German band “No

Pity for the Majority” reported incident to Kolasa

Wrote KMFDM on test

None Not noted MC, KK 16

9/17/13 Diary entry: outlined “project Saguntum, a 10 year

subconscious project to . . . shoot up my school. . . before year is over . . .I am a psychopath with a superiority complex”

Started diary and

planning attack

None Not noted No one 14

9/22/13 Diary entry: “I am filled with hate, I love it. . . I feel like a bomb. . . When I do commit my atrocities, I want conversation to be about elementary school teasing.

Words hurt, can mold a sociopath, and will lead someone a decade later to kill”

Described self as sociopath

None Not noted No one 14

9/26/13 Conducted threat assessment follow-up meeting with

Thurneau, Kolasa, Murphy, Karl and parents

AHS conducted

threat follow-up

None taken IC-CL

AT, KK, TM 19

9/30/13 Diary entry: “I feel like a bomb. . . it is important to note I rarely take my meds”

Feel like bomb None taken Not noted No one 14

10/--/13 Observed looking at pictures of guns and mass shootings on computer in cafeteria by Cameron Rust and Christina

Kolk, which they reported to Darrell Meredith

Viewed guns/shootings in

cafeteria – reported to AP

None taken Not noted CK, CR, DM 27

10/1/13 Diary entry: “Saguntum is the project to shoot up (and

maybe bomb) Arapahoe High School”

Planned to shoot

up AHS

None taken Not noted No one 14

10/3/13 Diary entry: “since day 1, my job has been to . . . shoot up the school. . .date is set for mid-November, I need time to

build my arsenal”

Set attack date for mid-Nov


10/11/13 Diary entry: “had a shrink appointment. . . massive waste of time”

Wasted psych meeting


10/15/13 Diary entry: “shooting up [place where I had] psych evaluation. . . lied through my teeth through the test”

Lied in psych evaluation


10/26/13 Diary entry: “the 13th

of December is a great date, as the 347

th . . . date of the year. . . it is a day of gore”

Set attack date for Dec 13

th


11/1/13 Asked “when can we drink tequila” in Vicki Lombardi’s

Spanish class; Lombardi emailed mother with concern about behavior and grades

Tequila incident None taken Not noted

in IC; email

VL, BP 21

11/6/13 Diary entry: “December 13 date I chose is perfect. . . 38

days”

38 days None taken Not noted No one 14

11/24/15 Diary entry: “It’s weird going through life knowing that in

19 days, I’m going to be dead”

19 days None taken Not noted No one 14

11/26/13 Diary entry: “I can’t believe in a fortnight, I’ll be dead. . .I had no friends at Arapahoe, and I was trying to fit in”

No AHS friends None taken Not noted No one 14

12/6/13 Purchased shotgun Bought gun None taken Not noted 6 peers 14

12/11/13 Locked out of Lombardi’s classroom by a classmate; banged on door and when asked if he was serious, said

“serious as a heart attack”

Banged on classroom door

Sent home; not suspended

IC-BDR VL, KK 24

12/12/13 Observed pacing near library Acted suspiciously None taken Not noted Peer ACSO

p.1785

12/12/13 Told peers and teacher about his new shotgun “Kurt Cobain” in hallway; said to peer “Don’t make me show

Showed pictures of gun

None taken Not noted Peers, BM ACSO p. 1785

APPENDIX 1: CHRONOLOGICAL LIST OF KP’S CONCERNING BEHAVIORS CONTINUED


you Kurt Cobain”; teacher Brad Meyer warned about suspension for threat

12/12/13 Student reported Karl’s possession of a gun to Song Peer reported gun purchase to

counselor

None taken Not noted Peer; ES-denied

ACSO p.1784-

1785

12/12/13 Purchased shotgun shells and belt at Cabela’s Purchased ammunition

None taken Not noted No one ACSO p.1954

12/13/13 Diary entries end Last diary entry None taken Not noted No one 14

12/13/13 Shot Claire Shot Claire

Initials Glossary: AT: Astrid Thurneau; BM: Brad Meyer; BP: Barbara Pierson; CK: Christina Kolk; CR: Cameron Rust; DM: Darrell Meredith; ES: Esther Song; JC: Jeff Corson; KK: Kevin Kolasa; ML: Mark Loptien; TM: Tracy Murphy; VL: Victoria Lombardi

APPENDIX 2: DIFFERENCES BETWEEN RISK & THREAT ASSESSMENT


Differences Between Tools for Risk Assessment and Threat Assessment

Validated Risk Assessment

(e.g., SAVRY/RRCU) Validated Threat Assessment

(e.g., V-STAG)

Purpose

• Identify risk and protective factors for intervention

• Build a plan to manage the individual based on the identified risks and protective

factors

• Respond to threat posed

• Build a plan to mitigate threat (e.g., when boundary probing, threat assessment response is defined and

acted upon)

Intended Victim • Not specified, general • Usually identified

Timeframe • Open-ended • Relatively short, unless new

risk or threat factors identified

Intervention

Strategy • Mitigation and/or support • Problem resolution

Goal • Accurate Prediction • Prevention

Social Ecology • Not considered • Goal to improve climate

APPENDIX 3: CHILD IN CRISIS ASSESSMENT RECOMMENDATION

Prepared by Dr. Monica Fitzgerald, Center for the Study and Prevention of Violence, University of Colorado-Boulder


An analysis of populations of rampage shooters or school shooters in the U.S.

indicates that there is substantial heterogeneity in these youth’s histories,

including their family backgrounds, personalities, and behavior (Langman,

2009). This heterogeneity has led Langman (2009) to identify typologies of

school shooters; these typologies can potentially be used with other prominent

social factors and trends to develop a threat assessment process (Langman,

2009; O’Toole, 2000; Verlinden, Hersen, & Thomas, 2000; Gladwell, 2015).

Langman’s (2009) analysis of ten cases of school shooters led to the

identification of three typologies: traumatized, psychotic, or psychopathic. It is

important to note that profiles or typologies should not be used alone to

identify students who pose a risk for targeted school violence, given their

imprecision and the risk for false negatives. However, these typologies

combined with additional information regarding types of student behaviors

and communications provides valuable threat assessment information (Fein, et

al., 2002). What is concerning is that the severity of the escalating

psychological and behavioral problems experienced by school shooters in

many cases was not identified and their mental health needs went unmet. In

many school shooting cases, youth sent clear signals to others regarding their

problems and thus were not “invisible” but did not receive an effective

response (Fein, et al., 2002).

The limited yet valuable data about youth who engage in targeted violence

guides our recommendations for threat assessment in schools. The data

suggests that it is important to screen “youth of concern” for: (a) child abuse

and trauma history, including emotional abuse, physical abuse, sexual abuse,

neglect; (b) household dysfunction and childhood stressors, such as mental

illness in a household member, absence of a parent due to divorce, domestic

violence, substance use, and parental criminal history; (c) psychotic symptoms,

traits, and behaviors, such as auditory or visual hallucinations, bizarre,

disturbed thoughts, paranoia, fantasy/delusional thinking, odd social behavior,



verbalizations, and appearance, and other characteristics of schizophrenia-

spectrum disorders; and (d) psychopathic traits and behaviors, including a lack

of empathy, narcissism, sense of superiority and contempt for others, blatant

disregard for human life, verbalizations about hurting or killing others, lack of

guilt and remorse, and other mean-spirited and sadistic behaviors. Other

psychopathic traits and behaviors of concern include blatant violation and

rejection of traditional values, laws, social norms, or morality. Other factors

such as family structure, peer influence, and role models have been highlighted

as important to assess.

The literature highlights the importance of assessing individual psychological

factors contributing to engaging in targeted violence, rather than over-

focusing on social factors (e.g., media violence). It is strongly recommended

to develop a comprehensive, school-wide system for recognizing and

promoting youth’s social, emotional and behavioral health and development,

family background and level of support, as well as peer interaction (Schonfeld,

2015) in order to effectively identify appropriate supports and intervention

strategies and prevent future violence. A stepped process is recommended for

identifying youth with psychological and behavioral health problems and

assessing threat:

1. School staff (e.g., teachers, principal, administrative, lunch servers) receive

psychoeducation and training to identify psychological and behavioral

health problems as a first step of identifying “youth of concern” and refer to

mental health school staff. Students also receive developmentally

appropriate psychoeducation about emotional and behavioral signs,

communications, and social dynamics of concern in their friends and/or

peers, and provide comfortable ways to share this information with adults,

and schools share this type of information with parents. This is critical for

early identification and prevention because peers knew about the attacker’s



idea and/or plan in most shooting incidents, and rarely did adults receive

the threat information (Fein, et al., 2002).

2. When youth are identified as “of concern,” mental health school staff

administer brief standardized risk and threat assessment screening tools to

identify problem areas and risk level.

3. When youth are identified as having emotional and behavioral problems

and needs through the initial brief screening, mental health staff administer

comprehensive, standardized assessment tools and approaches (e.g.,

structured interviews) to assess psychological and behavioral health needs

and violence risk in youth.

4. Mental health staff identify support strategies and interventions to target

the youth’s identified emotional, social, and behaviors of concern and

closely monitor youth receiving those supports/interventions to measure

progress and assess violence risk in an ongoing manner.

There are psychometrically strong, well-validated structured tools for

assessing violence and trauma history, violence risk, and mental and behavioral

health problems in youth that can be incorporated in such a process (e.g.,

Borum, Bartel, & Forth, 2003; Singh, Grann, & Fazel, 2011; Pynoos & Steinberg,

2013; Kelleher, Harley, Murtagh, & Cannon, 2011; Goodman et al., 1998; Gardner,

Lucas, Kolko, & Campo, 2007).

Supporting Web Sites: www.state.ago.co.us; safe and drug free schools (Attorney General’s Office) / www.state.cde.co.us; “prevention”, “safe and drug free

schools”, (Colorado Department of Education);/ www.csdsip.net (Colorado School Districts Self Insurance Pool);/ www.colorado.edu/cspv/safeschools (CU Center

for the Study and Prevention of Violence) / www.casb.org (Colorado Association of School Boards);/ www.ed.gov/offices/OESE/SDFS (U.S. Dept. of Education,

Office of Safe and Drug Free Schools)/ smph.psych.ucla.edu (UCLA Mental Health and Schools).

11/30/2005 Created by the Interagency Social Support Team working group, Jeanne Smith, Chair, Colorado Attorney General’s Office 303-866-5672

INTERAGENCY AGREEMENT AND SOCIAL SUPPORT TEAM SELF ASSESSMENT CHECK LIST

These questions are designed as an aid to create information sharing agreements among schools, law enforcement, prosecution, courts, mental health, social services and other stakeholder professionals. The goal is to assure a safe environment for students and staff, provide a basis from which communities can organize Interagency Social Support Teams (ISST) that are encouraged by the legislature and share information mandated by statute (CRS 22-32-109.1(3) & CRS 19.1.303 and 304). The questions should be answered from each agency’s perspective. Each stakeholder agency should complete the checklist independently, then share the results and resolve differences. It is helpful to create a set of answers for incidents occurring on school grounds and off-campus, and for differing behaviors such as 1) rule breaking, 2) threats, and 3) unusual behaviors that may signal a school/public safety concern.

A “No” or conflicting answers between stakeholders indicates more discussion/action required.

CHECKLIST YES NO

1. Does each ISST agency share sufficient information to address public safety concerns?

1-a Do you understand your confidentiality requirements?

1-b Do you understand that schools are criminal justice agencies and therefore have access to criminaljustice records?

1-c Do you understand there are exceptions to confidentiality requirements for public safety purposes?

1-c Do you have a written policy and/or procedures that indicate how information is shared betweenagencies and other providers?

1-d Do you have a form for release of information?

2. Does each ISST team include the following recommended members to manage threatand/or other public safety concerns involving students:

• School representatives (administrator, special ed, psychologist, social worker, counselor)

• Law enforcement and prosecution representatives

• Juvenile justice representatives (probation, parole, diversion, DA)

• Human services or social services

• Mental health agency

3. Are ISST agencies and staff trained to identify and respond to warning signs and/orthreatening behavior?

3-a. Does this training for new and returning staff occur at least on an annual basis?

3-b Have you adopted a threat assessment protocol? (Used for actual threats/violence)

3-c Have you adopted a risk assessment protocol? (Used to identify risk and protective factors)

4. When a student exhibits an early warning sign or threatening behavior, are other agenciesnotified?

4-a. Do you have a written policy and/or procedures that indicate who is responsible for notification?

4-b. Do you have a written policy and/or procedure that indicate who is notified and how they will benotified?

5. Is there an automatic review of the situation by more than the agency first collecting theinformation?

5-a. Do you have a written policy and/or procedure that indicate how a review will be conducted, whoattends the review, and when parents are involved?

6. Are results of the review communicated to persons working directly with the student?

6-a. Do you have a written policy and/or procedure that indicate how a review is communicated?

6-b. If the review reveals no public safety concern, is that communicated?

APPENDIX 4: INTERAGENCY AGREEMENT AND SOCIAL SUPPORT TEAM


Events AHS knew

about

Events AHS did not

know about

KEY

September

17, 2013

started

diary and

planning

attack

September

10, 2013

boundary

probing

September

3, 2013

yelled

“going to

kill” Murphy

November 24, 2003

hit peers with lunchbox

November 16, 2011

told peer

“go gut yourself”

March 15, 2013

yelled “fuck” in

math class

August 11,

2013

totaled car

September 9, 2013

AHS threat

assessment

performed

September

26, 2013

AHS

conducted

threat

follow-up

October 3, 2013

set attack

date for

mid-November

October 15, 2013

lied in psych

meeting

October --,

2013

viewed guns in

cafeteria

November 1, 2013

tequila incident

November

24, 2013

19 days

December

12, 2013

purchased

ammunition

December

12, 2013

showed

pictures of

gun

December 11,

2013

banged on

classroom door

December

6, 2013

bought gun

December

13, 2013

shot

Claire and

killed self

September

30, 2013

feel like a

bomb

September

--, 2013

wrote

KMFDM on

testDecember 18, 2003

kicked and hit peers

November 28,

2011

“make him my

bitch”

Date Unknown

“penis” line in

speech

August 21, 2013

bullied peers

verbally

September

5, 2013

threat noted

in police

report

September 9,

2013

private

mental health

assessment

performed

September 22, 2013

described self as

sociopath

October 11, 2013

wasted psych

meeting

October 1, 2013

planned to

shoot up AHSOctober 26,

2013

set attack

date for

December 13th

November 6,

2013

38 days

November 26,

2013

no AHS friends

December

12, 2013

last diary

entry

December

12, 2013

peer

reported

gun

purchase

to

counselor

December --,

2013

viewed guns &

shootings in

cafeteria –

reported to AP

December 12,

2013

acted

suspiciously

........2003 2011 ......... August

2013

September

2013

October

2013

November

2013

December

2013

APPENDIX 5: TIMELINE OF KP’S CONCERNING BEHAVIORS

University of Colorado-Boulder & University of Northern Colorado

APPENDIX 6: TRAINING


APPENDIX 7: THREAT ASSESSMENT OF KP








Briefing on the Virginia Student Threat Assessment Guidelines http://curry.virginia.edu/research/projects/threat-assessment

Developed and field-tested in 2002, based on FBI and Secret Service/Dept. of Education reports

• Threat assessment conducted when a student has made a threat or engaged in threatening behavior

• Step-by-step process in manual, Guidelines for Responding to Student Threats of Violence

• Goal is to prevent violence and return student to school by understanding why student made threat and

resolving the conflict or problem that stimulated the threat

• 2013 listed as evidence-based program in the National Registry of Evidence-Based Programs and

Practices (NREPP)

Each school establishes a multidisciplinary team based on its existing staff of school administrators, mental

health, and law enforcement professionals (Schools may adapt team composition to fit their staffing, draw upon

law enforcement officers from other schools or community)

• Follows a 7-step decision tree and triage approach, so that most threats are resolved quickly with only a

few team members; only the most serious threats require law enforcement and full team involvement (see

Figure 1 on next page)

• Teams trained in one-day workshop (additional review of manual needed)

School systems trained:

• 47 Virginia school divisions encompassing 1,000+ schools

• Schools in Arizona, California, Colorado, Florida, Indiana, Kansas, Maryland, Massachusetts, Minnesota,

New York, Ohio, Pennsylvania, South Carolina, South Dakota, Tennessee, Vermont, Wisconsin

• Canada, Germany

Published research findings from 2 field tests, 3 controlled studies, and 1 state implementation study

• School staff have decreased anxiety, increased knowledge in responding to threats

• Students do not carry out their threats

• Reductions of 50% in long-term suspensions

• Reductions in bullying infractions

• Increased use of school counseling, increased parent involvement

• Students report greater willingness to seek help for threats of violence, more positive views of school

personnel

Cornell, D., Sheras, P. Kaplan, S., McConville, D., Douglass, J., Elkon, A., Knight, L., Branson, C., & Cole, J. (2004).

Guidelines for student threat assessment: Field-test findings. School Psychology Review, 33, 527-546.

Kaplan, S., & Cornell, D. (2005). Threats of violence by students in special education. Behavioral Disorders, 31, 107-119.

Strong, K., & Cornell, D. (2008). Student threat assessment in Memphis City Schools: A descriptive report. Behavioral

Disorders, 34, 42-54.

Allen, K., Cornell, D., Lorek, E., & Sheras, P. (2008). Response of school personnel to student threat assessment training.

School Effectiveness and School Improvement, 19, 319-332.

Cornell, D., Sheras, P., Gregory, A., & Fan, X. (2009). A retrospective study of school safety conditions in high schools

using the Virginia Threat Assessment Guidelines versus alternative approaches. School Psychology Quarterly, 24,

119-129.

Cornell, D., Gregory, A., & Fan, X. (2011). Reductions in long-term suspensions following adoption of the Virginia Student

Threat Assessment Guidelines. Bulletin of the Nat Assoc of Secondary School Principals, 95, 175-194.

Cornell, D., Allen, K., & Fan, X. (2012). A randomized controlled study of the Virginia Student Threat Assessment

Guidelines in grades K-12. School Psychology Review, 41, 100-115.

Lovegrove, P., & Cornell, D. (2013). Large-scale implementation of the Virginia Student Threat Assessment Guidelines: A

quasi-experimental examination of effects on school suspensions. Chapter prepared for Race and Gender Disparities

in School Discipline. Center for Civil Rights Remedies, University of California, Los Angeles.

APPENDIX 8: BRIEFING ON VIRGINIA STUDENT THREAT ASSESSMENT GUIDELINES

Repo on the Arapahoe High School Shooting

http://curry.virginia.edu/research/projects/threat-assessment

Potential Violence Prevented by Threat Assessment

The following cases were reported by school authorities using our threat assessment guidelines (these are brief

summaries, not complete accounts of all factors considered):

1. A high school student posted on Facebook that he was considering killing himself and individuals on a

list. The threat assessment process revealed that the student was depressed, facing juvenile charges, and

was fantasizing about a way out of his troubles. Mental health services were provided and the family was

involved in a resolution.

2. A high school student threatened to blow up the school. The threat was investigated and could not be

resolved as transient, raising it to the level of a very serious substantive threat. Law enforcement

conducted an investigation which determined that the student had constructed a bomb that was found at

his home. The student was arrested.

3. A student was reported by friends to be contemplating a shooting at school. Interviews indicated that the

threat was imminent and law enforcement was alerted. The student was identified at the time he entered

the school and found to have a loaded firearm in his possession. He was arrested and charged with a

felony.

4. A student showed some classmates a knife at school. The information was shared with an adult and the

threat assessment team began an investigation. The student was called to the office and a search of his

book bag revealed a large knife and a loaded revolver. A threat assessment revealed a perception of being

bullied and various family issues. Mental health services and a bullying intervention were provided.

5. A high school student wrote a play that was about shooting students at school due to bullying. The

parents found the written play and brought it to the police, who notified school authorities. A threat

assessment revealed that the student was depressed and felt that he was being bullied at school. While he

did not have access to weapons, appropriate mental health services and referrals were made.

6. Parents took their daughter to an emergency room due to suicidal threats contained in letters found in her

room. The threat assessment revealed a plan to commit a mass homicide at school with her boyfriend,

and then they would then kill themselves. The girl was afraid that she was pregnant and both students

thought that the school environment was hostile. They had attempted to locate firearms, but were

unsuccessful. Both students received extensive mental health services.

7. A student made threats to carry out an ethnic cleansing at his school. A threat assessment was conducted

that included a search of his home. An unsecured loaded semi-auto pistol was found and confiscated. The

child was detained for a mental evaluation. The investigation revealed that he was communicating with

an online friend in another state who was considering a similar act. The police in that state were

contacted and the individual was arrested.

8. A high school student was disciplined by school administrators for writing a defamatory remark on his

ex-girlfriend's locker. Following the discipline meeting, the student posted on Facebook that he was going

to kill the principal and assistant principal. This information was brought by students to the attention of

the principal who immediately convened a threat assessment. The team judged the threat to be very

serious substantive, resulting in the requirement of a mental health evaluation. The evaluation revealed

urgent mental health concerns and significant evidence that he planned to carry out acts of homicide. As

a result, mental health intervention was court-ordered and a safety plan involving law enforcement was

implemented.



Step 1. Evaluate threat. • Obtain a specific account of the threat by interviewing the student who made threat, the

recipient of the threat, and other witnesses.

• Write down the exact content of the threat and statements by each party.

• Consider the circumstances in which the threat was made and the student’s intentions.

Step 2. Decide whether threat is clearly transient or substantive. • Consider criteria for transient versus substantive threats.

• Consider student’s age, credibility, and previous discipline history.

©All rights reserved

Step 3. Respond to transient threat.Typical responses may include reprimand, parental notification, or other disciplinary action. Student may be required to make amends and attend mediation or counseling.

Step 4. Decide whether the substantive threat is serious or very serious. A serious

threat might involve a threat to assault someone (“I’m gonna beat that kid up”). A very serious threat involves use of a weapon or is a threat to kill, rape, or inflict severe injury.

Step 5. Respond to serious substantive threat.

• Take immediate precautions to protect potentialvictims, including notifying intended victim andvictim’s parents.

• Notify student’s parents.

• Consider contacting law enforcement.

• Refer student for counseling, dispute mediation,or other appropriate intervention.

• Discipline student as appropriate to severity andchronicity of situation.

Step 6. Conduct safety evaluation. • Take immediate precautions to protect potential

victims, including notifying the victim and victim’sparents.

• Consult with law enforcement.

• Notify student’s parents.

• Begin a mental health evaluation of the student.

• Discipline student as appropriate.

Threat is serious.

Threat is clearly transient. Threat is substantive

or threat meaning not clear.

Threat is very serious.

Step 7. Implement a safety plan. • Complete a written plan.

• Maintain contact with the student.

• Revise plan as needed.

Threat Reported to Principal



APPENDIX 9: CSPV’S SAFE COMMUNITIES SAFE SCHOOLS MODEL


One example of a comprehensive approach to school safety is CSPV’s Safe

Communities-Safe Schools (SCSS) model. The SCSS Model consists of three

interdependent components, and each component addresses a critical and

distinct need for school safety and violence prevention. Taken together, the

components provide what Costa (2012) calls “parallel incrementalism” (i.e.,

several useful strategies implemented in tandem to address a persistent and

complex social problem).

The three components include: (1) implementation of an effective intelligence

gathering system (EIGS) to collect and interpret data at the school (e.g.,

school climate, problem behavior, discipline reports) and student levels (e.g.,

screening and risk/needs assessment for mental health, suicide, violence, drug

use, trauma); (2) engagement of an interagency social support team at the

school and the development of key community partnerships committed to

data-based decision making, culturally responsive interventions, and

embedding behavior change processes through the entire school’s structure;

and (3) development of a multi-level system of supports to build up schools

staff’s capacity to use evidence-based programs and strategies to address

data-identified student needs at three levels of need (all student, some

students, and specific students in crisis). Together, the three components

represent a three-legged stool; the stool falters if anyone is weak or missing.

Training, technical assistance, and one-on-one coaching help support the work

of teachers, counselors, community members, and administrators with all three

components. Starting in 2016, the SCSS model will be tested in 32 Colorado

communities over four years through a multi-million dollar National Institute of

Justice funded study granted to CSPV.