Organic School Info-Systems:Decision Making at the School Level

Christopher A. ThornUniversity of Wisconsin-Madison

Wisconsin Center for Education Research

The School Improvement Planning Process and Decision Support

Much my work has been split between district technology and research staff and schools working on SIP projects.

School Improvement Plans have been the justification for our efforts, but rarely seem to play an important role in improvement.

Is it possible to get inside the improvement process and see why this is failing?

What do teachers say about the utility of more data?

“ We need to have our minds opened to what else could be done with data.”

“To know what kind of data would be more useful. What’s out there, what’s meaningless?”

“I think data gives a small part of the picture. Some of it is very accurate, but I don’t think that it necessarily tells the entire tale, certainly not if you’re just looking at numbers.”

“It’s not so much getting a hold of the data, but applying it that’s the problem.”

How do teachers feel about using data?

“I tend to be somewhat cynical about data.”

“Information is power, but it's also really scary. It takes our staff out of their element.”

Teachers have concerns about data in regards to: Accuracy Quality of data sources and assessments Timeliness Usefulness in the classroom Use for accountability and evaluation

What teachers want from data?

“The problem I see is kids not learning, and if data could zero in on where we can do the most good, in time…”

“We need data that we can manipulate in the school and that can be accessed.”

School-level data Data with an instructional focus Disaggregated data, individual student data Longitudinal data, historical records Behavior data, attendance data Student home and background data Quick access to timely data

What skills do teachers (and building administrators) need?

Assessment literacy and alignment Technology (computers, software, databases) Data management, analysis, and application How to use data to:

Improve learning in our classroom and building Identify strengths and weaknesses in our curriculum and

instruction, and identify strategies for change Target student needs Ask better questions and get better answers To create visual representations Communicate with students, parents and other staff

Supporting information and decision making models

Recognize the role of leaders as sensemaker and that it is at least as important as the role of decision maker

Support PD that encourages building of analytical skills and local data analysis

Make technology investments that support the instructional mission

Make clear degree of alignment between curriculum, PD, and assessments

Data at finer levels of temporal resolution are key to understanding and addressing instructional problems. Local documentation of practice may help.

Needs assessment is a skill that seems to be lacking in most educational training. Rubric-type surveys could serve to education and provide feedback

Information/Flow Inventory:Where does your district fit?

Role Enter Data

Get Data Out

Provide data for lower level use

Provide data for upper level use

Local data for local use

Students

Parents

Teachers

Instruction Support

Special Services

School Administrators

District Administrators

Community Members

Decision support problem space

Existing IT Infrastructure

ProfessionalDevelopment

AccountabilityFramework

ChangingClassroomPractices

Capacity Assessment Tools

Capacity Type

Inquiry Process Technical Organizational

Decision Making /Data Selection

Data Acquisition

Data Management

Analysis and Reporting

Application

Evaluation

Using rubrics to guide the process

1)How well supported are the organizational and personnel issues (meeting time/space provided, work release time provided, leadership guidance, etc.) associated with managing and maintaining data?

Check the statement that best describes your school.

Education Plan Scenario

Assessing Classroom Learning

School Wide Problem

Little or no current support exists.

Current support exists, but is inadequate for current data needs.

Current support is adequate for current data needs, but not for future needs.

Current support is adequate for current and future data needs.

Tracking local assessments: Traditional and Standards-based

Linking to Standards to classroom work and assessments

Dimensions of Organizational Complexity

0 4

Sponsor

Objectives

Unresolved Issues

District Policies

Govt. Regulations

Leadership Team

Time to completion

Geography

Business rules

One

SMART*

Few

Established

Few/Simple

Experienced

Loose/flexible

One location

Established

None/many

Vague

Multiple

Non-existent

Many/complex

Inexperienced

Tight

System-wide

Non-existent

*Specific, Measurable, Achievable, Relevant to strategy, Time specific

Dimensions of Technical Complexity

0 4

Architecture

Technical Novelty

Comm. system

Decision making team

Team location

System platforms

Level of integration

Transaction volume

Fault tolerance

In place

Proven

Proven/stable

Experienced

One

One

Stand-alone

Low

High

Not in place

New

New

Inexperienced

Many

Many

Fully integrated

High

Low

School-level areas of intervention & innovation

School Level Ongoing dialogue about what constitutes relevant evidence

about student and teacher performance towards larger goals

Strategic collection of classroom-level data that supports core goals, not data for data’s sake

Flexibility to combine locally relevant data about practice with external data (district, cross-school, etc.)

Improvements to data access and manipulation

District Level Program identifiers for tracking program impact across

schools Embedding longitudinal models into centrally-held

information Providing ongoing training for teachers and administrators

on data collection and analysis Measuring what we care about rather than the stuff that is

easy to measure