Transcript Introduction to Designing Needs Assessment Surveys

INTRODUCTION TO DESIGNING NEEDS
ASSESSMENT SURVEYS
An AEA Mini Workshop presented by
James W. Altschuld
Yi-Fang Lee
Jeff White
Hsin-Ling Hung
The Ohio State University
National Chi Nan University
University of Louisiana at Lafayette
University of North Dakota
Annual Meeting of the American Evaluation Association
November , 2010
San Antonio, Texas
OBJECTIVES/AGENDA
Objectives
Understand Needs Assessment
Learn where versions of surveys fit the picture
Quick overview of survey parameters in NA
Get a hands-on feel for the process
Have the opportunity to discuss/comment on the
experience
Hopefully to have some fun on your brief journey into
the world of NA
Also to provide you with a glimpse into the Needs
Assessment Kit (Altschuld, Eastmond, King, Kumar,
Stevahn, and White, 2010)
AGENDA
Introduction (1 minute)
Questions (3-5 minutes)
Basic terms and concepts (8-10 minutes)
Survey overview (15 -18 minutes)
Types and where they fit the process
Samples of questions (scaled or otherwise)
General analysis strategies
Wording problems
Scenario overview, hands-on work, discussion, etc.
(rest of time allotted for the session)
BEGINNINGS
Questions for you
How many have done NAs?
Have you used surveys in
the process?
For what purposes?
Briefly describe the survey?
How did you analyze the
data?
Overall success of the
effort?
Did you use other methods
as well – what were they?
Any other comments about
what you did?
Terms/concepts
Need
-definition with attention to use
of words (need & needs)
-not mixing solutions with
needs
-definitional problem
immediately for wording survey
items
-examples of needs (see partial
table from book 1)
-types of needs (book 1)
-levels of needs
-Wow, issues at the outset
Needs assessment
-three phase model (see
schematic)
-types of surveys and where
they might fit
-why you still might have to
include solutions in survey
despite above admonition
DEFINITIONS & ISSUES
Need: the measurable discrepancy between “what
is” or the present state of affairs in regard to the
group and situation of interest and the “what
should be” or desired state of affairs (Witkin &
Altschuld, 1995).
Issues: measurable discrepancy is the key
-needs not solutions (premature closure on
solutions)
-verb vs. noun concept (misuse of the word)
-‘desired’, ‘likely to occur’, ‘ought to occur’, etc.
-wish and want lists
-many types of needs
MORE TERMS AND CONCEPTS
NA is a systematic set of procedures
undertaken for the purpose of
setting needs-based priorities and
making decisions about
organizational improvement and
allocation of resources (Witkin &
Altschuld, 1995).
Issues
-context for the NA
-readiness for an assessment
-NA is an organizational activity
-political aspects to the activity
-systems concept and how to
think about it
Lots of subtle aspects of need and
NA
Examples
Types
Why do it and how does it relate
to evaluation
Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D. Kumar , 2010,
Thousand Oaks, CA: Sage.
Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D. Kumar , 2010,
Thousand Oaks, CA: Sage.
Source: from Needs Assessment Kit Ι, by J. W. Altschuld and D. D. Kumar , 2010,
Thousand Oaks, CA: Sage.
SURVEY CONSIDERATIONS
Sampling
Level 1, 2, and/or 3
Wording issues
Where in the process
For NAC?
Phase 1, 2, 3
What type of need
Short term vs long term
Maintenance
Collaborative
Other
Double, triple scales or more
Content
Behaviors
Attitudes
Content (continued)
Degree of achievement
Satisfaction with services
Importance/value of services
Motivation to resolve problems
Frequency of use
Observations
Feasibility of resolving
problems
Literature based content
Impediments to solutions
List goes on and on
Formatting
Scales (what type)
Open-ended
Bars
Examples of Surveys but first
Table 3.1 Comparisons of Survey Usage for Organizing the NAC and for Conducting
Needs Assessments
Table 3.1 (Continued)
Table 3.1 (Continued)
Source: from Needs Assessment Kit 2, by J. W. Altschuld and J. N. Eastmond, 2010, Thousand Oaks, CA: Sage.
Table 3.2 Four Basic Survey Formats for Kicking Off Initial NAC Discussions
Source: from Needs Assessment Kit 2, by J. W. Altschuld and J. N. Eastmond, 2010, Thousand Oaks, CA: Sage.
Table 3.5 Structure of the NAC Survey
Table 3.5 (continued)
Source: from Needs Assessment Kit 2, by J. W. Altschuld and J. N. Eastmond, 2010, Thousand Oaks, CA: Sage.
Humann (1997)
Example of First Student Survey
OSEA (2005)
Example of Follow-Up Survey
Lee, Altschuld, & White (2007)
NOW THE HANDS-ON EXERCISE
Scenarios/orientation
Pretend that you are the needs
assessor
Individually and then collectively
What can a NAC tell you?
Questions
Content
Discussion focusing
Sketch out areas where they
could give you guidance/input
So you need a survey for other
groups
Review item writing rules
Review examples
Considerations for NA
surveys
What content
Levels 1, 2, and/or 3
Relation to other
methods
Use of multiple
scales
Impediments
Open-ended questions