MassCUE Evaluators

Day 1 – January 11, 2012

Greetings!

Logistics and Locations WiFi SSID and Password Bathrooms, etc.

Teams Who are you? Team members? Brief overview of your project

Your Facilitators

Sun Associates Jeff Sun Jeanne Clark www.sun-associates.com/masscue

Damian Bebell - Boston CollegeShelley Chamberlain - MassCUE

Agenda and Expectations

Review the agendaExpectations

All team members will attend all 4 days Some work will occur outside of these

sessions Collaborate! Produce a concrete evaluation plan Provide feedback to the process

Your Evaluation Plan Will…

Be organized around the goal(s) of your project

Define success through the creation of indicators keyed to your project goal(s)

Produce data specifically targeted at measuring success relative to your project goal(s)

Provide feedback and recommendations for improvement

Why Evaluate?

Determine if your investment in instructional technology is “paying off”

Measure progress toward meeting your project goals

Support action planning with dataIncrease eligibility for funding

Schematic View of the Evaluation Process

Goal

What is your project aiming to accomplish?The goal should be big-picture and over-arching…encompassing the spirit of what you want to accomplish.

Further detail on what it takes to meet the goal is covered in your indicators.

Indicators

What does it look like when your goal is met? Indicators are the organizing principle of your evaluation Indicators should reflect your project’s unique goals and

aspirations Indicators should clearly describe what it looks like

when/as you meet your goal(s)Indicators need to be highly descriptive and can include both qualitative and quantitative measuresSome indicators are more “measurable” than others Indicators guide your data collection

Data

What do you need to examine in order to find out if you’re meeting your indicators? CurriculumReview of curriculum units/lesson plansClassroom observations (of student impact)Student skills assessmentsReview of student work Professional DevelopmentTeacher interviewsProfessional development plansReview of developed lesson plans, etc.Classroom observation (of pedagogy) InfrastructureClassroom observationsSurveysCost dataHelp desk data

Coordinating with Existing Data

What data is already collected and available for you to use?

Outcomes data on student achievement?Existing student assessments?You may choose to use these other data

sources in your analysis to address aspects of your indicators.

Sometimes, this additional data will be one of the sources that you use to “triangulate” your findings.

Showing Impact and Change

How do you determine that what you see happening in your program is different from the status quo?Is there control/comparison group available for comparison?Are pre-measures available for comparison?

Analysis and Findings

Data analysis – comparing data to indicators -- will show the degree to which actions (PD, pedagogy, student work, infrastructure implementation) come together to produce the intended result.

Findings are the results of this analysis.

Recommendations

Findings (analysis) lead to recommendations and reporting.

Recommendations lead to new action items. What are you going to do to implement the

recommendations?

Developing Indicators

Indicators are the organizing principle of your evaluation Indicators should reflect your project’s unique goals and

aspirations Your goals need to clearly state the purpose of your

project What is it that you intend to create with this project?

Indicators should clearly describe what it looks like when/as you meet your goal(s) Indicators need to be highly descriptive and can include both

qualitative and quantitative measures Some indicators are more “measurable” than others

Indicators guide your data collection

Goal Statement:

Project / Plan Evaluation

What will your project be doing?

What will be the desired outcome of

these actions?

What do these

desired outcomes look like in the context

of your project?

What do you need to find out in order to assess how effectively your actions are

producing the desired outcomes and fulfilling the

indicators?

How closely does your

data match your

indicators?

ActionsDesired

Outcomes Indicator

Data Avenues/S

ourcesQuestions/

ProbesData

Analysis

Curriculum

PD

Infrastructure

Etc….

Day 1 Wrap-Up

Questions?Daily EvaluationSee you tomorrow!