Advancing Assessment Literacy Continuing the Conversation

Advancing Assessment Literacy

Continuing the Conversation

Advancing Assessment Literacy Modules: Continuing the Conversation (February 2008) 2

Sustainability

“A difficulty with every school improvement project I have encountered is that the schools spend too large a proportion of their professional development resources on training before implementing a new innovation.”

Holcomb, E. L. (2001). Asking the right questions: Techniques for collaboration and school change. Thousand Oaks, CA: Corwin Press.


Sustainability

“The implication is that we “front-end load” a process that takes years and requires individual movement through several stages of acceptance and use. The assumption is that adult learners can acquire new skills all at once, in isolation from practice, and can retain and retrieve them for use once the system has removed barriers and set them free to proceed.”

Holcomb, E. L. (2001). Asking the right questions: Techniques for collaboration and school change. Thousand Oaks, CA: Corwin Press.


Sustainability

Find a partner, and discuss the following:

• What in Holcomb’s statements ring true in your professional work?

• What things have you seen that impact the sustainability of initiatives? Provide both positive and negative examples.


Form vs. Identity

• Garmston & Wellman (1999) suggest that adaptive schools must address the issue of form vs. identity.

• They assert that schools have hung on to form to the detriment of their identity. For example, many schools operate with the same class divisions, schedules, and basic curriculum as they did 100 years ago. These schools are frustrated with their inability to connect with the needs of their students, but seem unable to change for fear of losing the form of schooling to which they’ve become accustomed.

Garmston, R. & Wellman, B. (1999). The adaptive school: A sourcebook for developing collaborative groups. Norwood, MA: Christopher-Gordon Publishers.


Form vs. Identity

• Garmston & Wellman (1999) suggest that adaptive schools are able to respond to changing needs and demands. They contend that this is possible in schools that have a clear identity and a lack of attachment to form.



Form vs. Identity

Garmston & Wellman (1999) provide the following examples:

Example Form Identity

Eastman Kodak Film manufacturer Image technology company

Reference Librarian Quality filter Teacher of quality filtering

Foreign Language Teacher

Specialist in one language

Specialist in second-language learning



Form vs. Identity

To examine this concept, Garmston & Wellman (1999) provide the following three focusing questions:

• Who are we?

• Why are we doing this?

• Why are we doing this, this way?

These three questions should be revisited often as your school or learning community works towards set goals.



Who Are We?

• Using the supplied template, provide details of your school’s identity.

• What data might need to be collected in order to confirm the identifying statements you have made?

• How might this information inform the improvement strategies you are working on?


Data to Support Answering“Who Are We?”

• One way of getting a picture of your school’s progress over time is to look at longitudinal data.

• Longitudinal data can reveal trends and patterns that allow the system to be more responsive to the needs of students.


Longitudinal Data

• Longitudinal data can be represented in a variety of ways. Some of the more common are bar graphs and radar charts.– Bar graphs provide an efficient means of

displaying progression of single items over time.

– Radar charts are useful for displaying the relationship between items as they change over time.

Grade 5 Math, 2004/2006

0

10

20

30

40

50

60

70

80

90

100

Num & Op -Whole

Num & Op -Fractions

Geometry Measurement DataManagement

CPR Questions APS Questions

Strand

Perc

en

tag

e C

orr

ect

2004

2006

• What statements can you make based on this display?

• What relationships do you see?

• How does displaying the data in this way affect how you read the data?

• What statements can you make based on this chart?

• This chart uses the same data as the bar graph. How does displaying the data in this way affect how you read the data?

• What questions might the teachers in this school be asking at this time?

Radar Chart 2004 vs 2006Percentage Correct in Each Strand

0

10

20

30

40

50

60

70

80

90

100

Num & Op - Whole

Num & Op - Fractions

Geometry

MeasurementData Management

CPR Questions

APS Questions

2004

2006


Longitudinal Data

What other sources of longitudinal data might inform the planning and work you are doing?


Standards andLongitudinal Data

• Saskatchewan Learning has engaged teachers in a standards setting exercise, carried out following each subsequent AFL assessment.

• These standards will allow for comparison between assessments through a compensation for differences in the level of difficulty between different years of assessment instruments.

• This is done because those using the data may not know how difficult the questions on the assessments were – the standard provides another lens through which to view data.

From 2007 Assessment for Learning Standard-Setting booklet.


Why Are We Doing This?

• The improvement goals set by schools often answer this question, but we need to dig deeper to ensure that we are approaching our work from a strong set of shared beliefs.

• For some, identification of gaps provides an opportunity to examine what aspects of our work should be most important and can reveal if we are putting our energy into the right places.

1 = not very important 5 = very important Rate from 1 – 5

In our school . . . ImportanceCurrent Status

Gap Size

We know where we’re going.

We use data to tell us how far we’ve progressed.

We use data to tell us if our vision is being achieved.

We have a strong vision that is pointing us forward.

Our change efforts are purposeful.

Essential Learnings guide our classroom efforts.

Our school leaders serve as “Keepers of the Vision”.

Importance: How important should this item be to us.

Current Status: The level of importance an item is currently receiving.

Gap Size: Subtract the Current Status score from the Importance Score.

The larger the score the larger the gap between what should be and what is.

Holly, P. J. (2003). Conceptualizing a new path. Princeton, NJ: Educational Testing Service.


Other Questions to Consider

To answer “Why Are We Doing This?”, consideration might be given to the goals thathave been set.

• Do they align with the mission and vision of the school?

• Do they clearly identify teaching and learning strategies that will be implemented as a means to improvement?

• How often might you need to revisit these types of questions?

• What questions would you ask as you work through an improvement plan?


Why Are We Doing This, This Way?

• Critical inquiry is an important component of any improvement plan.

• A sustained ability to inquire about and reflect upon our practice is essential for sustaining the longevity of any improvement plan.

• Action research is a powerful tool for facilitating inquiry and reflection.


Why Are We Doing This, This Way?

Action research can be defined as “an investigation conducted by the person or the people empowered to take action concerning their own actions, for the purpose of improving their future actions”. (Sagor, 2005, p. 4)

Sagor, R.(2005). The action research guidebook: A four-step process for educators and school teams. Thousand Oaks, CA: Corwin Press.


Three Questions

To determine if your study is action research, Sagor (2005) suggests that the answer must be “yes” to the following questions:

• Is the focus on your professional action?

• Are you empowered to adjust future action based on the results?

• Is improvement possible?



Four Stages ofAction Research

1. Clarifying Vision and TargetsKey Question: What do I want to accomplish?

2. Articulating TheoryKey Question: What do I believe is the approach with the greatest potential for achieving my goals?

3. Implementing action and collecting dataKey Question: What data will I need to collect to understand the efficacy and working of my theory in action?

4. Reflecting on the data and planning informed actionKey Question: Based on this data, how should I adjust my future actions (teaching)?



Focused Reading

• Using the provided passage, read to find “yes” answers to the three questions suggested by Sagor and find examples of the four stages of action research.

• What other important points caught your attention?

• When everyone is done, discuss your findings.


Action Research Planning

• Think about the improvement plans you have through your PLC, school, or division.

• Using the supplied template, do some initial exploration to see if your plans meet the three requirements of action research and if so, begin answering the questions for the four stages of action research.

• What benefits do you see in sustaining your initiatives through action research?


Opportunities forAction Research

The McDowell Foundation funds research into teaching and learning in publicly funded elementary and secondary schools. It supports a wide range of activities aimed at the improvement of education:

•research projects•publication of research results•conferences and workshops on research•awards for educational research•networking and mentoring opportunities for teachers and researchers

http://www.mcdowellfoundation.ca/main_mcdowell/about_us/index.htm



Groups of teachers or entire school staffs can apply for research funding through the McDowell Foundation. These funds can create release time for teachers in order to manage their study, collaborate and plan for the effective implementation of their findings.

How might applying for a McDowell project enable your staff to sustain the work you have begun?

Application forms and details can be found at:

http://www.mcdowellfoundation.ca/main_mcdowell/index.htm



Other opportunities for action research could include:

– University classes– Focused action research as the work of

a PLC– Participating in a study– Team teaching– Collaboration among colleagues


Actions

Reflecting on the material presented in this module, what three actions might you take that will change your work, thinking, or inquiry as you begin actualizing your improvement plans?

Documents

Advancing Assessment Literacy Continuing the Conversation