!

!

!

!

!! " # $ % & ! " % ' ' ( ) ! * $ + & , + - # ' ! . / & 0 1 ' 2 # & 2 !

3 4 # ' 1 # 2 + / & ! 5 6 + , 2 + 2 + / 1 0 7 ! * $ / - / 0 # ' ! !

r e g a r d i n g

8 9 % ! : % 2 % $; + & + & < ! = & 0 2 $ 1 , 2 + / & # ' ! * 1 $ - / 0 % 0 !

5 : = * 7 ! 8 $ # + & + & < ! * $ / < $ #; !

!s u b m i t t e d t o

6#$!>%02!?#@/$#2/$(!A/$!3B1,#2+/&#'!C%0%#$,9!#&B!

:%4%'/-;%&2!

Ed Tech 505

July 16, 2010

! !

!

3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! L!

!

IntroductionIntroduction

This document has been created and submitted on behalf of Bear Mountain Consulting, in response to the Request for Proposals (RFP), recently issued, by Far West Laboratory (FWL) for Educational Research and Development. The document is a proposal to provide the evaluation of a newly released training program called Determining Instructional Purposes (DIP). It will be conducted to provide information and recommendations 1) regarding distribution of DIP learning units, and 2) regarding purchasing and using DIP learning units in the future to administrators and grad students.

Proposed Program Description Proposed Program Description

FWL developed The DIP training package with a series of three units to train school administrators and graduate students. The training is intended to provide the professionals with planning and administering skills in effectively delivering educational programs. The DIP training program (print form) is comprised of the following:

NAME COST

Coordinator’s Handbook $4.50 each Training Units (Self-contained, Step-by-Step) (155 – 259 pages) (Use individually or in combinations.)

1. Setting Goals

2. Analyzing Problems

3. Deriving objectives.

$8.95 each or $24.95 for set of 3

The Coordinator’s Handbook provides details on the procedures including directions for the coordination of the three units. Each of the three units has 4-6 smaller modules within it, providing a limited number of instructional objectives regarding training. Typically each module has reading material on the skills taught, and includes practice activities related to the skills, provided either as individual or small group activities with trainee role-play and feedback. The estimated training time ranges from 10-18 hours for each unit, with unit 2 requiring the longest training time. A stipulation is that the training be both guided and facilitated by a program coordinator. It is necessary for the successful program coordinator to have worked through the training units, before commencing the facilitation. It is not necessary for the program coordinator to have any outside training or prior knowledge.

MethodMethod of Evaluation of Evaluation

Goal Goal

To provide feedback on whether or not FWL would benefit, from the capital it invests, to create and market the DIP training program package.

Objectives:Objectives:

1. Effectiveness: - To measure and identify module effectiveness.

2. Quality: - To assess module quality.

3. Implementation: - To measure implementation of program materials by school

districts.

! !

!

3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! M!

!

AudienceAudience & Evaluation Model & Evaluation Model

The FWL stakeholder committee is comprised of the following members: board of

directors, public shareholders, administration, and staff. All of the stakeholders will continue to

be included within the evaluation process. (To ensure that the needs of everyone involved are

addressed.) The decision-based evaluation model has been selected as the most effective design

model to use for this particular evaluation. Dr. Driver and stakeholders from the FWL committee

(as mentioned above), along with the Bear Mountain Consulting team have selected this model,

based on the desired outcomes of the evaluation.

Information RequiredInformation Required , , Sources Sources & Recommendations& Recommendations

Looking at each of the three objectives individually: effectiveness, quality, and implementation, the tables (below) detail the evaluation plan.

E f f e c t i v e n e s sE f f e c t i v e n e s s

EVALUATION

QUESTION

Do the modules provide effective training?

RECOMMENDED

METHOD

IN THE

FORM OF

TO

DETERMINE

MEASURING DETAILS

Quantitative Pre tests &

Post tests

- Entry level knowledge/skills

- Gained

knowledge/skills from taking DIP

training.

-The outcomes of each unit

-The outcomes of each unit

-100 % of clients/participants

tested.

-100 % of clients/ participants tested.

Qualitative Video or

audio recorded interviews

(before the pre

tests) &

Video or

audio recorded

interviews (after all the

tests)

- The attitude of

clients/participants before training.

- The attitude of

clients/participants

after training.

- The attitude of

clients/participants before training.

- The attitude of

clients/participants

after training.

- 100 % of

clients/participants formally

interviewed.

- 10% of administrators,

coordinators, and

grad students randomly sampled.

- 100 % of

clients/participants

formally interviewed.

- 10% of

administrators, coordinators, and

grad students

randomly sampled.

! !

!

3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! N!

!

The evaluation team of Bear Mountain Consulting will create, conduct, and administer the measurements. They will be conducted via computer. The evaluation team will compare and contrast the results of the pre and post test data. This will provide the team with data indicating the knowledge/skills gained by the client participants in taking each of the training units. The results will be provided to the FWL stakeholders. All interviews will be recorded either face-to-face or via teleconferencing. The recordings will be saved, and provided to FWL if requested, to ensure quality of evaluation is upheld and maintained. The interviews will be conducted either face-to-face or via teleconference. The information once collected, will provide the evaluation team with an attitude assessment, before and after training, based on the trainings effectiveness.

Q u a l i t yQ u a l i t y

EVALUATION

QUESTION

Do the modules provide quality training?

RECOMMENDED METHOD

IN THE FORM OF MEASURING/ DETERMINING

DETAILS

Qualitative - On- site observations

- On-site

Survey (short Likert- 5

point)

- Recorded

teleconferencing interviews (video or

audio) (short)

-The quality of the training procedures.

-The attitude at the

end of the unit.

-The impact of the training.

-The quality of the

training procedures. -The attitude at the

end of the unit.

-The impact of the training.

- 1 evaluator attends min of 1 training session.

-Random sample. 10% of

all including participants,

admin. & program coordinators.

- Participant/clients

To determine the quality of the DIP program, the evaluation team of Bear Mountain Consulting will create, conduct, and administer the quality measurements. The evaluator’s will have read the Coordinator’s Handbook, particularly focusing on the procedures, previous to attending the on-site observation training session. This will ensure that he/she is qualified to conduct the evaluation. All of the qualitative quality data will be collected, cataloged and graphed. This information will also be presented (and recorded interviews saved) to the stakeholders of FWL.

! !

!

3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! O!

!

I m p l e m e n t a t i o nI m p l e m e n t a t i o n

EVALUATION

QUESTION

Are the modules ready to be disseminated?

RECOMMENDED METHOD

IN THE FORM OF

MEASURING/ DETERMINING

DETAILS

Qualitative - Pre learning computer-

survey (Likert 5

point scale)

- Post learning computer-

survey (Likert 5

point scale) - Interviews

- Observations

video recorded.

- Perceptions and attitudes of what

may be gained from

this program.

- The impact the program had on the

participants.

-As above.

-How to properly

implement the program.

-100% of individuals directly involved with the program.

-100% of individuals directly involved with the program

- 100% of indiv. directly involved with the program

-Evaluator will observe a min of 1-

unit.

Quantitative -Checklist

(Looks at

whether obj. were met)

-Findings -Evaluator will tabulate all findings for

each unit within the training session.

The evaluation team will create, conduct, and administer the implementation measurements. The results of the surveys will indicate how willing people are to participate in the program, as well as what the impact of the program was. This will then provide information in the decision making process of whether or not to proceed with the DIP program, or choosing a better alternative, and how to implement the program in the future. The collected quantitative data will provide information as to usability of the program as well as the confidence levels in using the coordinator handbook. This checklist will provide feedback to the team as to the proper implementation of material. The evaluators will video record the observation sessions to ensure validity. They will also make all of the implementation data available to the FWL stakeholders.

TaskTask ScheduleSchedule BY COMPLETED BY

1. Meeting - stakeholders + evaluators, discuss & change. BMC Start: Sept 7, 2010

2. Present everything in plan for review. BMC Oct 4, 2010 3. Provide feedback on #2. FWL Oct 18, 2010

4. Revisions of #2 - Final copy submitted. BMC Oct 29, 2010 5. Delivery of evaluation tools to participants. BMC Nov 19, 2010

6. Evaluators have attended min. 1 training session. BMC Dec 3, 2010

7. Evaluators observe unit in real time. BMC Dec 17, 2010 8. Administration of surveys, assessments, and interviews. BMC Dec 17, 2110

9. Collect data. BMC Jan 31, 2011 10. Summarized data submitted to shareholders. BMC March 31, 2011

11. Feedback from initial findings of complied data. FWL April 30, 2011 12. Completed/ Final report to FWL. BMC End: May 31, 2011

Note: FWL = Far West Laboratory, BMC = Bear Mountain Consulting

! !

!

3DE?FE8=GH!*CG*GIE?!G6!:=*!8CE=H=HJ!*CGJCEK! P!

!

Project PersonnelProject Personnel

Bear Mountain Consulting provides the services of a highly skilled team of experts (http://edtech2.boisestate.edu/kellyk/505/our_team_of _experts.docx) with 30+ years of experience, and a rich history of diversified success in the field of program evaluation. The Bear Mountain Consulting team provides full service program evaluation, rigorous analysis of existing qualitative data, and training in qualitative methods and analysis. Each member of the team is passionate about his/her profession, focused on innovation, and dedicated to providing high value solutions to his/her customers. All associates hold either a doctorate or masters degree, and have a minimum of five years experience relevant in the project area to which they may be assigned. In addition, Bear Mountain Consulting can facilitate the assistance of native speakers of French, Spanish, Hindi and Thai. To learn more about our company’s philosophy, services, our commitment to you, the credential of each of our team members, and what our clients have to say about us, check out our web page: http://www.bearmountainconsulting.ca.

FeeFee s s (Computers will be used to cut down on costs and save time.)

TOPICS DETAILS FEE CHARGED

Personnel Salaries

Karen Kelly, Principal Consultant Alejandro Velasco, Associate Habiba Chopras, Associate

Mano Srisai, Associate

Secretarial Staff TOTAL PERSONNEL

50 days at $300/day 30 days at $200/day 30 days at $200/day 50 days at $150/day 25 days at $100/day

$ 15,000.00 $ 6,000.00 $ 6,000.00 $ 7,500.00 $ 2,500.00 $ 37,000.00

Travel

• 2- round trip airfares • Car rental and fuel • Food and Lodging

TOTAL TRAVEL AND / DIEM

Wildlife – Vancouver

$ 600.00 $ 500.00 $ 600.00 $ 1700.00

Supplies

• Materials & Supplies • Printing

TOTAL SUPPLIES

$ 500.00 $ 450.00 $ 950.00

Communication

• Telephone & Videoconferencing

• Courier Services TOTAL COMMUNICATION

$ 800.00 $ 120.00 $ 920.00

ESTIMATED TOTAL FEES $ 40, 570.00 Thank you! Our team looks forward to working with you!