An AEA Mini Workshop presented by James W. Altschuld The Ohio State University Yi-Fang LeeNational...

An AEA Mini Workshop presented by

James W. Altschuld The Ohio State UniversityYi-Fang Lee National Chi Nan UniversityJeff White University of Louisiana at

LafayetteHsin-Ling Hung University of North Dakota

Annual Meeting of the American Evaluation Association

November , 2010San Antonio, Texas

ObjectivesUnderstand Needs Assessment Learn where versions of surveys fit the pictureQuick overview of survey parameters in NAGet a hands-on feel for the processHave the opportunity to discuss/comment on the experienceHopefully to have some fun on your brief journey into the world of NA

Also to provide you with a glimpse into the Needs Assessment Kit (Altschuld, Eastmond, King, Kumar, Stevahn, and White, 2010)

Introduction (1 minute)Questions (3-5 minutes)Basic terms and concepts (8-10 minutes)Survey overview (15 -18 minutes)

Types and where they fit the processSamples of questions (scaled or otherwise)General analysis strategiesWording problems

Scenario overview, hands-on work, discussion, etc. (rest of time allotted for the session)

Questions for youHow many have done NAs?Have you used surveys in the process?For what purposes?Briefly describe the survey?How did you analyze the data?Overall success of the effort?Did you use other methods as well – what were they?Any other comments about what you did?

Terms/conceptsNeed

-definition with attention to use of words (need & needs) -not mixing solutions with needs-definitional problem immediately for wording survey items-examples of needs (see partial table from book 1)-types of needs (book 1)-levels of needs-Wow, issues at the outset

Needs assessment-three phase model (see schematic)-types of surveys and where they might fit-why you still might have to include solutions in survey despite above admonition

Need: the measurable discrepancy between “what is” or the present state of affairs in regard to the group and situation of interest and the “what should be” or desired state of affairs (Witkin & Altschuld, 1995).

Issues: measurable discrepancy is the key-needs not solutions (premature closure on solutions)-verb vs. noun concept (misuse of the word)-‘desired’, ‘likely to occur’, ‘ought to occur’, etc.-wish and want lists-many types of needs

NA is a systematic set of procedures undertaken for the purpose of setting needs-based priorities and making decisions about organizational improvement and allocation of resources (Witkin & Altschuld, 1995).

Issues-context for the NA-readiness for an assessment-NA is an organizational activity-political aspects to the activity-systems concept and how to think about it

Lots of subtle aspects of need and NAExamplesTypesWhy do it and how does it relate to evaluation

Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D. Kumar , 2010, Thousand Oaks, CA: Sage.

Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D. Kumar , 2010, Thousand Oaks, CA: Sage.

Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D. Kumar , 2010, Thousand Oaks, CA: Sage.

SamplingLevel 1, 2, and/or 3Wording issues

Where in the processFor NAC?Phase 1, 2, 3

What type of needShort term vs long termMaintenanceCollaborativeOther

Double, triple scales or moreContent

BehaviorsAttitudes

Content (continued)Degree of achievement Satisfaction with servicesImportance/value of servicesMotivation to resolve problemsFrequency of useObservationsFeasibility of resolving problemsLiterature based contentImpediments to solutionsList goes on and on

FormattingScales (what type)Open-endedBars

Examples of Surveys but first

Table 3.1 Comparisons of Survey Usage for Organizing the NAC and for Conducting Needs Assessments

Table 3.1 (Continued)

Table 3.1 (Continued)

Source: from Needs Assessment Kit 2, by J. W. Altschuld and J. N. Eastmond, 2010, Thousand Oaks, CA: Sage.

Table 3.2 Four Basic Survey Formats for Kicking Off Initial NAC Discussions

Source: from Needs Assessment Kit 2, by J. W. Altschuld and J. N. Eastmond, 2010, Thousand Oaks, CA: Sage.

Table 3.5 Structure of the NAC Survey

Table 3.5 (continued)

Source: from Needs Assessment Kit 2, by J. W. Altschuld and J. N. Eastmond, 2010, Thousand Oaks, CA: Sage.

Humann (1997)

Example of First Student Survey

OSEA (2005)

Example of Follow-Up SurveyExample of Follow-Up Survey

Lee, Altschuld, & White (2007)

Scenarios/orientationPretend that you are the needs

assessorIndividually and then collectively

What can a NAC tell you?QuestionsContentDiscussion focusing

Sketch out areas where they could give you guidance/input

So you need a survey for other groupsReview item writing rulesReview examples

Considerations for NA surveys

What content Levels 1, 2, and/or

3Relation to other

methodsUse of multiple

scalesImpedimentsOpen-ended

questions