V 2 Region 5 Evaluation Session 11 16 11

8/3/2019 V 2 Region 5 Evaluation Session 11 16 11

1/26

Region 5 RPTAC

Region 5 Evaluation

Work GroupNovember 16, 2011

Region 5 RPTAC1


2/26

Region 5 RPTAC

Barbara Buswell: Region 5 RPTAC Director

Nora Thompson: Region 6 RPTAC Director

Presented by:

2


3/26

Region 5 RPTAC

1. Explore evaluation sections of RFA

2. Increase understanding of evaluation approaches and

terms

3. Consider alignment of evaluation methods with logic

model, goals & activities, and OSEP Performance

Measures

4. Increase confidence in tackling evaluation dimensionsin project implementation and proposal development

3


4/26

Region 5 RPTAC

The Accountability Era

4


5/26

Region 5 RPTAC

What Does An Evaluation Do?

Answers one or more of four main types of questions?

What occurred ; what did we do?

How well did it go; how well did we do it?

So what? What difference did our efforts make? What

changed?

What should we change in future projects?

5


6/26

Region 5 RPTAC

Are we doing what we promised in our proposal in the

timeline established?

In what ways and to what extent is our project as a wholeeffective?

How effective are our specific approaches and activities?

How can we be sure that we are meeting our outcomes?

6


7/26

Region 5 RPTAC

Selected Sections from 2011 RFA

Sample Evaluation Table

7


8/26

Region 5 RPTAC

Key Sections directly related to evaluation in 2011

Request for Application

Management Section (section total of 20 points) (e) (ii)

Quality of Project Evaluation Sections (section total of 15points)

(f) (i)

(f) (ii)

(f) (iii)

8


9/26


10/26

Region 5 RPTAC

What evaluation methods do you use in your PTI or

CPRC project?

10


11/26

Region 5 RPTAC

Methods of Data CollectionSources of Information Existing data

- Program records,

attendance logs, etc- Pictures, charts, maps,

pictorial records

Program participants

Others: key informants,

nonparticipants, proponents,

critics, staff, collaborators,

funders, etc.

Data Collection Methods

Survey

Interview

Test Observation

Group techniques

Case study

Photography Document review

Expert or peer review

11


12/26

Region 5 RPTAC

(f) Quality of the Project Evaluation

(f) (i). The extent to which the methods of

evaluation are thorough, feasible, and

appropriate to the goals, objectives, and

outcomes of the proposed project

12


13/26

Region 5 RPTAC

(f) Quality of the Project Evaluation

(f) (ii). The extent to which the methods ofevaluation are appropriate to the context

within which the project operates

13


14/26


15/26

Region 5 RPTAC

Some projects include an evaluation chart

that articulates for each goal:

The Goal

Evaluation Question

Methodology to Be Used

Criteria

Timeline

15


16/26

Region 5 RPTAC16


17/26

Region 5 RPTAC

Use a variety of approaches to ensure that you

are using both formative and summative

approaches.

Consider the appropriateness of the method to

the activity.

Consider the yield of various approaches.

17


18/26

Region 5 RPTAC

Remember to be sure that your evaluation methods andplan align with

Your grant goals

Your management plan

Your logic model

The parameters in the RFA

OSEP Performance Measures (high quality, relevant, useful)

18


19/26

Region 5 RPTAC

Formative

Summative

Qualitative

Quantitative

High Quality

Relevant Useful

Others?

19


20/26

Region 5 RPTAC

Formative = Process/Implementation

Conducted during the development and implementation

of a program

Primary purpose is providing information for program

improvement.

20


21/26

Region 5 RPTAC

Summative = Outcome/Impact

Evaluation conducted after completion of a program (or

a phase of the program) to determine program

effectiveness and worth

21


22/26

Region 5 RPTAC

Qualitative

Effective, relevant, useful

Depth and detail

Generally narrative

Quantitative

How many, how much

Numbers of workshops, numbers of families trained

22


23/26

Region 5 RPTAC

Take Turkey Match quiz within 1 week

Begin outlining evaluation section

Check out Lingo to Go to sound sophisticated

Attend second workgroup session on Wednesday, November

30 at 12:00 pm MDT

Call Individual TA for specific questions or clarity

23


24/26

Region 5 RPTAC

Question: What about having a consultant or friend write

the grant (or evaluation section) for me? Pitfalls & benefits?

Can someone from TA write a section for me?

Can TA review and critique my proposal?

Do we really have to do all this work?

24


25/26

Region 5 RPTAC

What Have We Learned?

25


26/26

Region 5 RPTAC

Thank you!

26

Documents

V 2 Region 5 Evaluation Session 11 16 11