Phillips Associates
Developing ValidDeveloping Valid
Level 1Level 1
EvaluationEvaluation
FormsForms
Developing ValidDeveloping Valid
Level 1Level 1
EvaluationEvaluation
FormsFormsPresented by:Ken PhillipsPhillips AssociatesMay 22, 2008
Phillips Associates 2
OBJECTIVESOBJECTIVESOBJECTIVESOBJECTIVES
Examine core purpose of Level 1 evaluations
Analyze strengths and shortcomings of 3 sample Level 1 evaluation forms
Apply 6 tips to design more valid Level 1 evaluations
Phillips Associates 3
QUESTIONSQUESTIONSQUESTIONSQUESTIONS
What is the core purpose of Level 1 evaluations?
How valid are the results?
Phillips Associates 4
ANSWERSANSWERSANSWERSANSWERS
Improve a
learning program
not prove
something
Not
very
valid
&
Phillips Associates 5
SAMPLESAMPLE
EVALUATIONEVALUATION
FORMSFORMS
SAMPLESAMPLE
EVALUATIONEVALUATION
FORMSFORMS
Phillips Associates 6
6 TIPS6 TIPS6 TIPS6 TIPS
Phillips Associates 7
TIP 1TIP 1TIP 1TIP 1
Where appropriate, match up
qualitative questions with
quantitative measures
Phillips Associates 8
EXAMPLEEXAMPLEEXAMPLEEXAMPLE
In a word, how would you describe thissession? ____________________________
Using a number, how would you describe this session?
No Value Great Value
1 2 3 4 5
Phillips Associates 9
TIP 2TIP 2TIP 2TIP 2
When collecting quantitative data
using a Likert scale, create a
response scale with numbers at
regularly spaced intervals and words
only at each end*
* Palmer Morrel-Samuels, “Getting the Truth into Workplace Surveys”, HarvardBusiness Review, 2002.
Phillips Associates 10
EXAMPLEEXAMPLEEXAMPLEEXAMPLE
How would you rate the overall effectiveness ofthe learning activities used in this session?
Not Very
Effective Effective
1 2 3 45
Phillips Associates 11
TIP 3TIP 3TIP 3TIP 3
Use only one response scale with an
odd number of points (e.g. 5, 7,9, 11)*
* Palmer Morrel-Samuels, “Getting the Truth into Workplace Surveys”
Phillips Associates 12
ODD vs EVEN SCALEODD vs EVEN SCALEODD vs EVEN SCALEODD vs EVEN SCALE
Phillips Associates 13
TIP 4TIP 4TIP 4TIP 4
Use small numbers at the low or left
end of the scale and larger numbers
at the right or high end of the scale
Phillips Associates 14
TIP 5TIP 5TIP 5TIP 5
Write items either as a continuum or
as a statement
Phillips Associates 15
The facilitator effectively used AV materials.
Strongly StronglyDisagree Agree
1 2 3 45
How effectively did the facilitator use AVmaterials?
Not Very
Effectively Effectively
1 2 3 45
EXAMPLESEXAMPLESEXAMPLESEXAMPLES
Phillips Associates 16
TIP 6TIP 6TIP 6TIP 6
Include at least one item asking
participants how relevant the
learning event/material was to them
and their job
Phillips Associates 17
EXAMPLEEXAMPLEEXAMPLEEXAMPLE
How would you rate the overall relevance of thesession to you and your job?
Not at all Veryrelevant relevant
1 2 3 45
Phillips Associates 18
REFERENCESREFERENCESREFERENCESREFERENCES
Boehle, Sarah, “Remember that mean, crabby teacher inhigh school?”, Training, August 2006, pps. 17-22.
Phillips, Ken, “Six Tips on Developing Valid Level 1Evaluation Forms”, Training Today, Fall 2007,pps. 8 & 14.
Morrel-Samuels, Palmer, “Getting the Truth intoWorkplace Surveys”, Harvard Business Review,February 2002, pps. 111-118.
Miner, Nanette, “The Art of Test Creation”, T + D,July 2007, pps. 27-29.
Phillips Associates 19
Ken Phillips
Phillips Associates
34137 N. Wooded Glen Drive
Grayslake, Illinois 60030
(847) 231-6068
www.phillipsassociates.com
Recommended
