THROUGH W. EDWARDS DEMING’S PROVEN DATA-DRIVEN CONTINUOUS IMPROVEMENT MODEL

IMPROVING STUDENT

ACHIEVEMENT

Dr. Jerry Jenkins, NWESD SuperintendentBuck Evans, NWESD Assistant Superintendent for Operations

“Quality comes not from inspection, but from improvement of

the process.”W. Edwards Deming

Variation

Theory

Human Interaction

Systems

Profound Knowledge

An ongoing appreciation of the interdependence of these elements is prerequisite to continuously improving • classrooms• schools• school districts

School Level

Systems

• Provide for meaningful constituent (staff, parents, students) participation and readiness (able and willing/secure)

• Establish a culture of data-driven decisions• Prioritize — school-wide focus on specific

strategies to improve student learning • Remove barriers for those working on and in

the system • Celebrate successes

Classroom Level

Systems

• Prioritize/Focus — identify and share key learner outcomes

• Chart formative classroom learning data to inform teacher student decisions

• Provide for each student’s tracking of learning to inform his/her decisions

• Ask for and respond to student feedback related to improving performance

• Celebrate learning successes

Psychology in a System• People are born motivated• Systems extinguish motivation• Once discouraged, most people stay that way• Extrinsic rewards demotivate• Ranking destroys joy and motivation• Western society is based on extrinsic motivation

and scarcity mentality

Human Interaction

School Level• Meaningful participation within parameters

(SIPTAP Process Tools)• Determine why intrinsic motivation is not

present — remove factors from school• Keep intrinsic motivation elements prevalent

Autonomy Belonging CompetenceDelight Engagement

• Drive fear out of the school — adults and students

Human Interaction

Classroom Level• Ask for and respond to student feedback related

to improving performance• Recognize students are most like adults in

feelings, rather than thinking• Track and share learning results so learning

community can make adjustments• Coach until last two weeks; then referee• Determine why intrinsic motivation is not

present — remove factors from classroom• Celebrate class successes

Human Interaction

Theory in a System• Information is about the past• Knowledge is about creating a better future• Experience teaches you nothing without theory• One unexplained example invalidates the theory• Systems need to support and create leadership

Theory

School Level• Dig deeper — look for reasons behind practices,

successes, and failures Five Whys – Deming

• In God we trust — everyone else bring data• Learning is the constant — instructional

strategies the variable• Challenges — 96% systems / 4% people• Leadership — barrier removal

Theory

Classroom Level

Theory

• Use random sampling (√ of n) to coach, until last couple weeks when referee

• Learning is the constant — instructional strategies the variable

• Eliminate permission to forget• Improvement strategies trials — data• Research: Two practices increase student

achievement — peer tutoring and regular feedback (Gersten, U of O)

• Increase student success — decrease student failure

Classroom Level ExamplesData to Make Teaching Practical

• Develop rubric/scoring guide with students• Teach rubric — quality examples• Assign practice• Select √ of n (5-6 per class) to score• Share results, plan with class, teach/re-teach

based upon sample results• Repeat with next assignment• Grade final example at end of term

Theory

Classroom Level ExamplesSample Data Scoring Results

— 15 missing clear topic sentence— 13 unclear flow from intro to body— 9 subject-verb tense mismatch— 8 conclusion fails to summarize key points— 5 not formatting properly (margins, title size, name placement)

Theory

Classroom Level ExamplesEliminate Permission to Forget

• Provide list of key facts• Test each week

20% from previous years70% from current year10% from future year

• Students graph individual results• Graph class results

Theory

Classroom Level ExamplesIndividual Run Chart

Theory

Classroom Level ExamplesClass Run Chart

Theory

Classroom Level ExamplesClass Run Chart

Theory

Variation in a System• Can’t improve what’s not measured and

graphed/charted• Variation will always exist• The goal — reduce variation• Decision makers reduce variation through

statistics• Control charts permit “common cause” or

“special cause” identification• Statistics permit future predictions• Only reason to test —

determine what next Variation

-------------------Test # ------------- SUM

1 2 3 4 5 6 7 8 9 10Pat 0 0 0 1 0 0 0 0 0 0 1Eva 0 0 0 0 0 0 0 0 1 0 1Ted 0 0 0 0 0 0 0 0 0 0 0Jim 0 1 0 0 0 0 0 0 0 0 1Flo 0 0 0 0 0 1 0 0 0 0 1Ned 0 0 0 0 0 0 0 1 0 0 1Hal 0 0 0 0 0 0 0 0 0 0 0Sam 1 0 1 1 0 1 0 0 1 1 6Sue 0 0 0 0 0 0 0 0 0 0 0Rod 0 0 0 0 0 0 0 0 1 0 1

Student Test Results

Variation

Improvement and Variation

Variation

Num

ber of Students

Performance

Desired Statistics

Lowest Improves Highest

Improves

VariationNarrows

School Level• Extrapolate data by subgroups — teacher,

demographics, mobility, etc.• Determine whether special or common cause —

make appropriate adjustments• Track summative grade-level/school-wide

learning data to inform program, teacher, and student decisions

• Focus on systems improvements — 96% of time / people 4%

Variation

Classroom Level• √ of n is sample size for accurate data — if

collected weekly or bi-weekly• Select random samples — GameSciences• Determine whether special or common cause —

react appropriately• Track summative class learning data to inform

teacher and student decisions• Learning is the constant — instructional

strategies the variable

Variation

Classroom Level Example

Variation

Key Concepts1) A statistically valid way of sampling is using the

square root of n2) There are two kinds of causes of variation:

special and common cause3) A practice that does much more harm than

good is ranking4) The most common problem understanding data

is failure to understand variation5) To improve results, work on the process6) A visual tool that can help you plot results on

either a scoring guide/rubric or criterion-referenced assessment for both individual and class performance is a scatter diagram

Key Concepts7) Dr. Deming states that the only reason a test

should ever be given is to find out what to do next

8) A statistical graph that separates special from common cause of variation is a control chart

9) Two primary aims of a school system are to increase student success and reduce failure

10)The number of data points needed to rule out good or bad luck is seven

11)Three fundamental beliefs about people in a continuous improvement system are — people want to do a good job, they can learn, and they want to contribute and be involved

Key Concepts12)Planned change should be based on data and

research13)The system is at fault 96% of the time, people

4% of the time14)The purpose of analysis is insight15)To learn we need feedback; this is what data

provides us16)To tell how the whole class is doing, graph the

group total right on their assignments