Embed Size (px)
Citation preview
Revisiting Target Setting: A Complex Task
Judy M. Parr University of Auckland
EHSAS Seminars October 2008
Target Setting
• Target setting should be multi-focussed and student achievement targets differentiated
• Targets should relate to the elements in the cycle for enquiry
• Targets should relate to your theory for improvement
• Targets should help you to see the nature of the relationships that influence achievement (diagnose where the links need to be strengthened )
The Linked Cycle
• Inquiry and knowledge-building cycles for professional learning of leaders and teachers (Timperley, 2008)– What educational outcomes are valued for our
students and how are our students doing in relation to those?
– What knowledge and skills do we as teachers need to enable our students to bridge the gap between current understandings and valued outcomes?
The Linked Cycle (cont’d)
– How can we as leaders promote the learning of our teachers to bridge the gap for our students
– Engagement of students in new learning & teaching practices
– Engagement of teachers in further learning to deepen professional knowledge and refine skills
– Impact of changed actions on our students
Theory for Improvement
• As a group, we (hopefully) have some shared theory of improvement
• May draw on research findings like fact student achievement related to:– Clear emphasis on academic & learning goals– Goals embedded in school routines– Systematic monitoring of student progress– Teachers using evidence of learning formatively to
adjust practice– (Leadership BES, Black, Wiliam et al),
Theory for Improvement
• Targets need to allow you to examine elements of your theory– if your theory says that teacher’s ability to work
with data impacts student achievement, then have goals, criteria and monitor relating to this
– If your theory includes teacher practice (specific features of it) impacting then have goals, criteria and monitor practice
– If your theory says leadership is important, then have goals, targets and monitor
Overview: Theme 1
• Goals of a different order: Cycle of enquiry goals- goals for factors related to or impacting on longer term student achievement goal
• Group discussion
Overview: Theme 2
• Differentiating goals for student achievement– Recap: Student achievement target setting – Setting differentiated targets
• Group discussion• Conclusion: Further Issues
Goals of a Different Order
Goals of a Different Order
• These concern variables shown to have causal link to ultimate goal; evidence about them helps provide explanations– e.g. student attendance, amount instructional
time; specific teacher practices; amount reading; teacher PCK; leadership practices
Examples of Factors Predictive of Student Achievement
– Teacher knowledge– Teaching practice– Organisational/
leadership– Student attendance/
attitudes etc
Using Evidence: Student Achievement
• Analysing and using evidence of student achievement requires knowledge (including PCK) and skill– Theory of task (knowledge of domain) in order to
select/ develop tool
– Knowledge of tool (appropriateness & limitations)
– Knowledge of basic statistics -interpreting
– PCK to apply findings
• (Parr & Timperley, 2008)
Teacher Knowledge Goals
• These could relate to content knowledge in writing– E.g. Teacher understanding of asTTle writing
criterion statements
• Could relate to knowledge of comprehension strategies– E.g. Teacher ability to recognise
comprehension skill or strategy in different questions in tests from asTTle: Reading
Teacher Knowledge Goals
• Teacher ability to analyse data– Understand key concepts (e.g. mean, SD,
range, distribution etc) – To extrapolate major trends from data
• Teacher ability to ‘use’ data- articulate & demonstrate application to practice – In grouping, identifying & planning specific
foci, individual & small group teaching
Teacher Knowledge Goal Operationalised: Example
• Content knowledge of writing: Demonstrated in terms of consistency and accuracy in scoring scripts using asTTle– Target = In moderation, obtain 70%
agreement with ‘expert’ (agreement is exact or adjacent sub-level but not across levels) in all dimensions
Teacher Knowledge Goal Operationalised: Example
• Ability to interpret data: Demonstrated though teachers/ leaders writing a summary of assessment results from their class/ year level/ school. Target = all summaries meet following criteria
• Accurate
• Include major points
• specific, not generalised
• Reference a point to specific data
• Use terminology appropriately
Teaching Practice
• Generic indicators of effective practice (e.g. instructional time in curriculum area; differentiation)
• Specific indicators– achievement related feedback in curriculum
area– Explicit teaching of self-regulatory strategies
in writing
Setting Targets to Find Out About Teacher Practice
• Targets need to relate to specific aspects of practice that research shows are effective in raising achievement (and relate to needs of current students)
• “Increased collaboration to strengthen professional community”– Research shows PLC per se does not relate
to enhanced teacher learning or student achievement. Specific features may.
Setting Targets to Find Out About Teacher Practice
• Practices need to be able to be described/ specified in a way that can be “observed”. e.g. what might “improve writing teaching in more depth” look like?
• Systematic observation necessary (self-report inaccurate)
Examples of Specific Practices
• Enhanced practice in <writing> in terms of– Achievement related feedback during literacy
learning– Communicating expectations of what counts
as successful achievement of learning goal– Activating links between learning goals,
expectations and prior knowledge
• Observe with an instrument that has ordinal categories with descriptors
Example of Categorical Descriptors: Feedback
• 1. Teacher’s feedback is not directly related to achievement - rather it is approving, rewarding, and / or disapproving of behaviour
• 2. Teacher’s feedback refers in a general manner to:
• - learning goal / expectations regarding successful achievement
• - generic aspects of literacy learning
Categories re Feedback
• 3. Teacher tells the learner about whether and / or how their work has met / not met:
• - learning goal / expectations regarding successful achievement
• - generic aspects of literacy learning
Categories re Feedback
• 4. Learner and teacher discuss (with learner ‘taking the lead’) whether and / or how their work has met / not met:
• - learning goal / expectations regarding successful achievement
• - generic aspects of literacy learning • Could set targets with respect to number
of teachers demonstrating levels 3 & 4
Indicators of Student Learning
• Research also suggests certain student behaviours & attitudes that relate to achievement e.g.– self efficacy (e.g. asTTle writing study)– ability to self regulate (LitPD example)– attending (body and mind!)– Motivation (Guthrie in reading)– Time spent- reading (Allington)
• Be aware that amount variance accounted for may be quite small
Targets for Proximal Indicators
• Reading mileage: Time target differentiated by year – (logs, organisation, materials, incentives)
• Self regulatory behaviours: Proportion of students target– X% of students will be able to articulate (gather
data by asking a sample of students questions) e.g. what learning; how will know if learnt; how are going in relation to learning goal; what need to get better at; how will go about this
Issues in Operationalisation
• Practicality: e.g. 100% of teachers will be using high levels of descriptive formative practice (C & D category – most D- in typology of Tunstall and Gibbs).
• Finding an appropriate measurement tool
• Having a reference point for target
• Attributing influence
Issues in Operationalising Targets
• Method of obtaining data. Who provides data? Self=report? Cross checks– “Survey of principals to report degree to which
believe cluster operating as network of PLC’s”– “Survey of teachers on degree to which have
improved understanding and repertoire”
• Not necessarily a relationship between reported skill/ confidence and actual ability to do it
Group Discussion 20 mins
• Do you have goals/ targets that are NOT student achievement ones?
• Would there be an advantage in working out some “different order” goals?
• What might these concern (given your theory for improvement?)
Student Achievement Targets
• How to decide what is focus (area for improvement)?– contributes to strategic vision, wider goal,
priorities– evidence for focus
• What / who to focus on?
• Considerations– Significance, specificity, pragmatics,
acceptance, challenging but achievable
Student Achievement Targets
• How to express target: In terms of:– standards (criteria) e.g. all or X% will meet
specified criteria (criterion statements)– standard (level) e.g. X% will be at or above
national norms (in asTTle mean total score or curriculum level)
– progress rate (against expected) e.g. All year levels will make average progress beyond expected gain (total scores in asTTle: Writing)
– improvement (beyond chance/ error)
Student Achievement Targets
• To decide if target appropriate consider– Baseline data– Levels achieved or rate of improvement
nationally (perhaps initially in clusters/ schools like yours)
– Intensity of focus including PD– Current staffing
Differentiated Targets for Student Achievement
• Like success criteria, targets should be differentiated
• Generalised targets like 60% above mean ignore sizeable numbers students
• Generalised targets for progress ignore fact that potential for progress may be different at different parts of distribution
• Targets that increase growth rate each year may ignore the exponential factor (or limits of certain tools)
Setting Differentiated Targets
• Ensure these best meet your goals. If goal to raise tail following OK– “90% will be reading at CA. 10% tail group will
move a minimum of 1 chronological year”
• Consider setting by class or year level• To make it differentiated, need to stipulate
– e.g. bottom 20% will progress at twice national average rate
– Of 40% currently in stanines 1-3, 30% will move out of them.
Issues in Operationalising Targets
• Stipulating/ having a definition of both variable and target: – Critical thinking ability will improve
• Having a reference point for goal– E.g. Where did % come from in “Yr 3, 80% at
stanine 5 by Oct; 20% at 8 and 9”?
• Tool to measure outcomes – does it relate to focus?– Is it a “sound” tool?
Group Discussion: 20 mins
• Discuss potential for differentiated targets for student achievement in relation to your current focus
• What might these targets look like?
• Be clear what reference point and rationale is for each sub goal
Conclusion: Further Issues
• Where national norms based on appropriately obtained sample of what students can do, do not accord with theoretically derived curriculum levels for years
• Where focus concerns an area with no developed tools- expertise required
• Intensive nature of examining classroom practice goals
