Transcript Developing Meaningful Learning Experiences:

Developing Meaningful
Learning Experiences:
Research & Assessment in
Leadership Education
© A. Vorreyer & D. Covey
Introductions!
 Dr. Anita L. Vorreyer
 University of North Florida
 Dr. Douglass F. Covey
 Georgia State University
Overview




Questionnaires
Definitions
Theoretical (Research) Bases
The Intersection of Student Development
and Leadership Education
 Assessment—Complete with STEPS!
 Examples at the micro and macro level
Pre-Session Questionnaire
 1. Student Learning Outcomes:
 A. Are measures of content knowledge (attainment)
 B. Are intentional learning objectives
 C. Are the stated objectives used by Student Affairs
units
 2. You can measure SLO’s by:
 A. Student satisfaction surveys
 B. Tracking student participation/usage in programs
and services
 C. Observing and documenting changes in behavior
 3. Institutional Effectiveness Indicators are
measurements of:
 A. Continuous improvement
 B. Student learning
 C. A measure of adherence to CAS Standards
 4. How would you rate your confidence in
your ability in designing a program using
Student Learning Outcomes:
 A. Unable
 B. Uneasy
 C. Somewhat confident
 D. Confident
 E. Extremely confident
 5. How familiar are you with CAS
Standards:
 A. Unfamiliar
 B. Somewhat familiar
 C. Familiar with Standards
 D. Familiar and have used to a limited degree
 E. Very familiar and have regularly used in program
assessment
Learning Objectives
 Participants will be able to
 Define the terms “assessment”, “evaluation”, and
“student learning outcomes (SLOs)”
 Understand the differences conceptually between
 Outcomes Assessment
 Institutional Effectiveness (continuous improvement)




Design programs driven by SLOs
Recognize principles of good SLO assessment
Understand the steps in outcomes assessment
Design assessment strategies re: SLO chosen
Definitions
 Research: a methodical investigation into a
subject; guides theory development and tests
concepts
 Assessment: Any effort to gather, analyze, and
interpret evidence which describes institutional,
divisional, or agency effectiveness. (Schuh, Upcraft, &
Associates, 2001)
 Evaluation: any effort to use assessment evidence
to improve institutional, departmental, divisional
effectiveness.
 Assessment describes effectiveness. It guides good
practice. Evaluation uses these descriptions in
order to improve effectiveness.
Two additional terms
 Institutional Effectiveness Plan: Systematic
efforts of the university to engage in ongoing
institution-wide review of all of its academic,
administrative, and educational support units and
programs.
 Student Learning Outcome: That which you
want the student in your course/program to learn
as a result of your time with them. It may be
cognitive and or behavioral.
Why should Student Affairs
focus be on SLOs?
 Student learning and development are inherent in
our missions as divisions of Student Affairs.
 The formal education of students consists of the
curriculum and the co-curriculum, and must
promote student learning and development that is
purposeful and holistic. (CAS Standards)
 Outcomes assessment helps us improve the
learning of students rather than simply
documenting participation or satisfaction.
CAS Standards
 What is CAS: The Council for the Advancement of Standards
in Higher Education.
 Since 1979 CAS has written and promulgated standards for
programs and services in higher education.
 Complying with national standards is one of the means by
which educators can assure high quality educational practices
and subsequently student learning.
 Provides learning domains and sample achievement indicators
Theoretical Base
 Student Development
 Leadership
Theoretical Families
 Psychosocial / Identity Theories
 Cognitive Development Theories
 Typology Theories
 Environmental Impact Theories
Psychosocial / Identity Theories
 Focus on definition of self and relationship
to others
 Erickson
 Chickering
Cognitive Development Theories
 Focus on development of critical thinking /
reasoning
 Perry
 Kohlberg
 Gilligan
Typology Theories
 Focus less on change process than upon
persistent characteristics (learning styles,
personality type)
 Meyers Briggs
 Holland
Environmental Impact Theories
 Focus on interaction of individual with
organization and culture of college life
 Astin
 Pace
 Tinto
Common Assertions




Development is a continuous process
Development is cumulative in nature
Development progresses along a continuum
Development is fairly orderly, stage
oriented
 Development is reflected in behavior
 Miller and Winston 1990
Developmental Tasks of
Particular Interest to Leadership
Educators
 The ability to appreciate multiple
viewpoints
 The ability to understand pluralistic values
systems
 The ability to resolve conflicts
 The ability to be genuine
 The capacity for empathy
 The ability to be an agent of change
Leadership Theoretical Base
 Stogdill: “There are as many definitions of
leadership as there are people who have
tried to define it!”
 4 common components of leadership
definitions (Northouse):
 Leadership is a process
 Leadership is relational
 Leadership is based on influence
 Leadership is goal oriented
We assist our students in
learning leadership in many ways
 Through the curriculum; in coursework
 Through Co-curricular efforts:
 Workshops
 Retreats
 Advising student groups
 Working with special student communities (Greeks,
Student Government, Residential students, etc.)
We may use different theories





Situational Leadership (Hersey & Blanchard)
Servant Leadership (Greenleaf)
Transformational Leadership (Burns)
Relational Leadership Model (Komives, et al)
Social Change Model (HERI)
From a curricular perspective…
 We usually discuss the particular theories of
leadership, and examine various
components of the leadership process:
 Change
 Ethics
 Teams
 Organizations
 Emotional Intelligence
Within Student Affairs…and
from a co-curricular perspective
 Team Building
 Other Key Components:
 Motivation
 Conflict Resolution/Management
 Communication
 Decision-Making
 Ethical Behavior
 Whether curricular or co-curricular, our aim
is to help them as individuals and as
positional leaders to:
 1) become aware of the different components of
leadership,
 2) engage in reflective practice, and
 3) develop key skills in leading themselves and
in leading others.
CAS Standards for Leadership
Education:
 Leadership is an inherently relational
process of working with others to
accomplish a goal or to promote change.
Most leadership programs seek to empower
students to:
 Enhance their self-efficacy as leaders
 Understand how they can make a difference whether
as positional leaders or active participants in a group
or community process.
 Research (theory) guides us in developing
meaningful SLO’s.
 We must make, and help our students to
make, the theory to practice connections.
We must build a “bridge”…………
The “Bridge”
 Leadership
development involves:





Self awareness
Understanding of others
Values
Diverse perspectives
Organizations
 Change
 Competence in
establishing purpose
 Working
collaboratively
 Managing conflict
Meaningful Leadership Learning
Experiences (SLO)
Student
Development
Leadership
Assessment
Examples from the CAS
Standards for Leadership
 Learning Domain: Intellectual Growth
 Achievement Indicator: Uses critical thinking in
problem solving
 Effective Communication
 Able to influence others through speaking/writing
 Leadership Development
 Comprehends the dynamics of a group
Assessment & Evaluation






Different Types of Assessment
Principles of Good SLO Assessment
Summative and Formative
Direct and Indirect
Qualitative or Quantitative
Barriers and Resources
Different Types of Assessment
(Upcraft & Schuh)








Tracking
Needs assessment
Satisfaction assessment
Campus environment/student culture
Cost effectiveness
Comparable institution
National standards
Outcomes assessment
Principles of Good SLO
Assessment
 1. Assessment should always be linked to
course/program objectives
 2. Assessment should be regular and
ongoing
 3. Assessment should be designed to
facilitate learning (yours and your students)
Summative & Formative
 Summative assessment: tests, papers,
assignments; grades over the length of the
course.
 Formative assessment: gives students and
leadership educators feedback on the nature
and progress of student learning; mastery,
not grades.
Direct & Indirect
 Direct: scores, writing samples, portfolio
artifacts, projects and presentations, oral
exams, internships
 Indirect: course grades, surveys,
questionnaires, focus groups, interviews
(dealing with students’ perception of their
own learning)
Qualitative & Quantitative
 Qualitative methodologies: detailed
descriptions of situations, events, people,
interactions and observed behaviors
 Quantitative methodologies: assign
numbers to objects, events, or observations
according to some rule. Instruments used to
collect data; statistical methods used to
analyze data.
Barriers & Resources
 Barriers:
 Lack of time
 Lack of money
 Fear of results
 Resources:
 Colleagues
 Institutional Research departments
 Graduate students
The Steps
 1. Define/Identify the SLO (cognitive,
behavioral, or affective
 2. Align the curriculum/content
 3. Identify/Determine the assessment
strategy
 4. Engage and Assess
 5. Evaluate
 1. Define the SLO
 What is it, exactly, that you want students to be able
to know or do at the end of your program/course?
 2. Align the curriculum/content of the
program
 Is what you are doing/covering in the
program/course going to lead to what you want your
students to know or do at the end?
 3. Determine the assessment strategy:
 What is it that you are going to measure?
 What techniques will you use to obtain this?







Interviews
Focus groups
Observations
Performance/demonstration
Portfolios
Surveys/questionnaires
Pre-test & post-test
 Confirm the “do-ability” of your chosen technique
 4. Engage and Assess
 5. Evaluate: use your evidence to improve
your program (IEP).
Define the
SLO
Evaluate
Engage &
Assess
Align the
Content
Determine
the
assessment
strategy
Examples
 From the micro-level
 From the macro-level
Macroview
 Georgia State’s Division Wide SLO
Domains
 Cognitive Maturity
 Effective Citizenship and Intercultural Maturity
 Mature Relationships
 Integrated Identity and Personal Maturity
Cognitive Maturity
Sub Domain: Problem Solving in Context
Student Judicial Board SLO:
Demonstrate ability to effectively evaluate
evidence
Effective Citizenship and
Intercultural Maturity
 Sub Domain: Acknowledging, respecting
and responding to difference
 Safe Zone Committee (LGBTQ) SLO:
Be able to identify two listening skills that
will assist in interacting with someone
expressing a different perspective from own
Mature Relationships
 Sub Domain: Conflict Resolution
 Recreational Services Student Employee
SLO
Will be able to describe and demonstrate two
methods to manage or resolve conflict with
participants in the Student Rec Center
Meaningful Leadership Learning
Experiences (SLO)
Student
Development
Leadership
Assessment
Designing SLOs






What group of students
Who participate in
What activity/event/course/program
Will be able to do, know, value what
At what level of accomplishment
As determined by what means
 Keeling, Wall, Underhile, & Dungy
Post-Session Questionnaire
 1. Which of the following is NOT an
indicator of Institutional Effectiveness
 A. Student usage/participation data for a facility or
program
 B. Student satisfaction surveys
 C. SLO assessment
 2. Meaningful use of SLO requires
 A. That learning must be measured
 B. The learning being measured must be the
learning that was intended
 C. That the results of assessment must be used to
refined programs/experiences
 D. All of the above
3. If your intended SLO is for participants to be able
to demonstrate acquisition of conflict resolution
skills, the most relevant assessment measure would
be:
 A. Questionnaire of understanding of conflict resolution
techniques
 B. Professional observation of student performance
 C. Peer perception of skills
 4. As a result of attending this session, my
confidence in my ability to apply SLOs to the
development of leadership programs and
experiences is:
 A. No better than before
 B. Somewhat better than before
 C. Significantly better than before
Explanation
 From the Student Learning Imperative:
 Student Affairs must model what we wish
for our students: an ever increasing capacity
for learning and self-reflection.
 THANK YOU FOR YOUR PARTICIPATION!!!
 Dr. Douglass F. Covey,
 [email protected]
 Dr. Anita L. Vorreyer,
 [email protected]