Developing Meaningful Learning Experiences:
Download
Report
Transcript Developing Meaningful Learning Experiences:
Developing Meaningful
Learning Experiences:
Research & Assessment in
Leadership Education
© A. Vorreyer & D. Covey
Introductions!
Dr. Anita L. Vorreyer
University of North Florida
Dr. Douglass F. Covey
Georgia State University
Overview
Questionnaires
Definitions
Theoretical (Research) Bases
The Intersection of Student Development
and Leadership Education
Assessment—Complete with STEPS!
Examples at the micro and macro level
Pre-Session Questionnaire
1. Student Learning Outcomes:
A. Are measures of content knowledge (attainment)
B. Are intentional learning objectives
C. Are the stated objectives used by Student Affairs
units
2. You can measure SLO’s by:
A. Student satisfaction surveys
B. Tracking student participation/usage in programs
and services
C. Observing and documenting changes in behavior
3. Institutional Effectiveness Indicators are
measurements of:
A. Continuous improvement
B. Student learning
C. A measure of adherence to CAS Standards
4. How would you rate your confidence in
your ability in designing a program using
Student Learning Outcomes:
A. Unable
B. Uneasy
C. Somewhat confident
D. Confident
E. Extremely confident
5. How familiar are you with CAS
Standards:
A. Unfamiliar
B. Somewhat familiar
C. Familiar with Standards
D. Familiar and have used to a limited degree
E. Very familiar and have regularly used in program
assessment
Learning Objectives
Participants will be able to
Define the terms “assessment”, “evaluation”, and
“student learning outcomes (SLOs)”
Understand the differences conceptually between
Outcomes Assessment
Institutional Effectiveness (continuous improvement)
Design programs driven by SLOs
Recognize principles of good SLO assessment
Understand the steps in outcomes assessment
Design assessment strategies re: SLO chosen
Definitions
Research: a methodical investigation into a
subject; guides theory development and tests
concepts
Assessment: Any effort to gather, analyze, and
interpret evidence which describes institutional,
divisional, or agency effectiveness. (Schuh, Upcraft, &
Associates, 2001)
Evaluation: any effort to use assessment evidence
to improve institutional, departmental, divisional
effectiveness.
Assessment describes effectiveness. It guides good
practice. Evaluation uses these descriptions in
order to improve effectiveness.
Two additional terms
Institutional Effectiveness Plan: Systematic
efforts of the university to engage in ongoing
institution-wide review of all of its academic,
administrative, and educational support units and
programs.
Student Learning Outcome: That which you
want the student in your course/program to learn
as a result of your time with them. It may be
cognitive and or behavioral.
Why should Student Affairs
focus be on SLOs?
Student learning and development are inherent in
our missions as divisions of Student Affairs.
The formal education of students consists of the
curriculum and the co-curriculum, and must
promote student learning and development that is
purposeful and holistic. (CAS Standards)
Outcomes assessment helps us improve the
learning of students rather than simply
documenting participation or satisfaction.
CAS Standards
What is CAS: The Council for the Advancement of Standards
in Higher Education.
Since 1979 CAS has written and promulgated standards for
programs and services in higher education.
Complying with national standards is one of the means by
which educators can assure high quality educational practices
and subsequently student learning.
Provides learning domains and sample achievement indicators
Theoretical Base
Student Development
Leadership
Theoretical Families
Psychosocial / Identity Theories
Cognitive Development Theories
Typology Theories
Environmental Impact Theories
Psychosocial / Identity Theories
Focus on definition of self and relationship
to others
Erickson
Chickering
Cognitive Development Theories
Focus on development of critical thinking /
reasoning
Perry
Kohlberg
Gilligan
Typology Theories
Focus less on change process than upon
persistent characteristics (learning styles,
personality type)
Meyers Briggs
Holland
Environmental Impact Theories
Focus on interaction of individual with
organization and culture of college life
Astin
Pace
Tinto
Common Assertions
Development is a continuous process
Development is cumulative in nature
Development progresses along a continuum
Development is fairly orderly, stage
oriented
Development is reflected in behavior
Miller and Winston 1990
Developmental Tasks of
Particular Interest to Leadership
Educators
The ability to appreciate multiple
viewpoints
The ability to understand pluralistic values
systems
The ability to resolve conflicts
The ability to be genuine
The capacity for empathy
The ability to be an agent of change
Leadership Theoretical Base
Stogdill: “There are as many definitions of
leadership as there are people who have
tried to define it!”
4 common components of leadership
definitions (Northouse):
Leadership is a process
Leadership is relational
Leadership is based on influence
Leadership is goal oriented
We assist our students in
learning leadership in many ways
Through the curriculum; in coursework
Through Co-curricular efforts:
Workshops
Retreats
Advising student groups
Working with special student communities (Greeks,
Student Government, Residential students, etc.)
We may use different theories
Situational Leadership (Hersey & Blanchard)
Servant Leadership (Greenleaf)
Transformational Leadership (Burns)
Relational Leadership Model (Komives, et al)
Social Change Model (HERI)
From a curricular perspective…
We usually discuss the particular theories of
leadership, and examine various
components of the leadership process:
Change
Ethics
Teams
Organizations
Emotional Intelligence
Within Student Affairs…and
from a co-curricular perspective
Team Building
Other Key Components:
Motivation
Conflict Resolution/Management
Communication
Decision-Making
Ethical Behavior
Whether curricular or co-curricular, our aim
is to help them as individuals and as
positional leaders to:
1) become aware of the different components of
leadership,
2) engage in reflective practice, and
3) develop key skills in leading themselves and
in leading others.
CAS Standards for Leadership
Education:
Leadership is an inherently relational
process of working with others to
accomplish a goal or to promote change.
Most leadership programs seek to empower
students to:
Enhance their self-efficacy as leaders
Understand how they can make a difference whether
as positional leaders or active participants in a group
or community process.
Research (theory) guides us in developing
meaningful SLO’s.
We must make, and help our students to
make, the theory to practice connections.
We must build a “bridge”…………
The “Bridge”
Leadership
development involves:
Self awareness
Understanding of others
Values
Diverse perspectives
Organizations
Change
Competence in
establishing purpose
Working
collaboratively
Managing conflict
Meaningful Leadership Learning
Experiences (SLO)
Student
Development
Leadership
Assessment
Examples from the CAS
Standards for Leadership
Learning Domain: Intellectual Growth
Achievement Indicator: Uses critical thinking in
problem solving
Effective Communication
Able to influence others through speaking/writing
Leadership Development
Comprehends the dynamics of a group
Assessment & Evaluation
Different Types of Assessment
Principles of Good SLO Assessment
Summative and Formative
Direct and Indirect
Qualitative or Quantitative
Barriers and Resources
Different Types of Assessment
(Upcraft & Schuh)
Tracking
Needs assessment
Satisfaction assessment
Campus environment/student culture
Cost effectiveness
Comparable institution
National standards
Outcomes assessment
Principles of Good SLO
Assessment
1. Assessment should always be linked to
course/program objectives
2. Assessment should be regular and
ongoing
3. Assessment should be designed to
facilitate learning (yours and your students)
Summative & Formative
Summative assessment: tests, papers,
assignments; grades over the length of the
course.
Formative assessment: gives students and
leadership educators feedback on the nature
and progress of student learning; mastery,
not grades.
Direct & Indirect
Direct: scores, writing samples, portfolio
artifacts, projects and presentations, oral
exams, internships
Indirect: course grades, surveys,
questionnaires, focus groups, interviews
(dealing with students’ perception of their
own learning)
Qualitative & Quantitative
Qualitative methodologies: detailed
descriptions of situations, events, people,
interactions and observed behaviors
Quantitative methodologies: assign
numbers to objects, events, or observations
according to some rule. Instruments used to
collect data; statistical methods used to
analyze data.
Barriers & Resources
Barriers:
Lack of time
Lack of money
Fear of results
Resources:
Colleagues
Institutional Research departments
Graduate students
The Steps
1. Define/Identify the SLO (cognitive,
behavioral, or affective
2. Align the curriculum/content
3. Identify/Determine the assessment
strategy
4. Engage and Assess
5. Evaluate
1. Define the SLO
What is it, exactly, that you want students to be able
to know or do at the end of your program/course?
2. Align the curriculum/content of the
program
Is what you are doing/covering in the
program/course going to lead to what you want your
students to know or do at the end?
3. Determine the assessment strategy:
What is it that you are going to measure?
What techniques will you use to obtain this?
Interviews
Focus groups
Observations
Performance/demonstration
Portfolios
Surveys/questionnaires
Pre-test & post-test
Confirm the “do-ability” of your chosen technique
4. Engage and Assess
5. Evaluate: use your evidence to improve
your program (IEP).
Define the
SLO
Evaluate
Engage &
Assess
Align the
Content
Determine
the
assessment
strategy
Examples
From the micro-level
From the macro-level
Macroview
Georgia State’s Division Wide SLO
Domains
Cognitive Maturity
Effective Citizenship and Intercultural Maturity
Mature Relationships
Integrated Identity and Personal Maturity
Cognitive Maturity
Sub Domain: Problem Solving in Context
Student Judicial Board SLO:
Demonstrate ability to effectively evaluate
evidence
Effective Citizenship and
Intercultural Maturity
Sub Domain: Acknowledging, respecting
and responding to difference
Safe Zone Committee (LGBTQ) SLO:
Be able to identify two listening skills that
will assist in interacting with someone
expressing a different perspective from own
Mature Relationships
Sub Domain: Conflict Resolution
Recreational Services Student Employee
SLO
Will be able to describe and demonstrate two
methods to manage or resolve conflict with
participants in the Student Rec Center
Meaningful Leadership Learning
Experiences (SLO)
Student
Development
Leadership
Assessment
Designing SLOs
What group of students
Who participate in
What activity/event/course/program
Will be able to do, know, value what
At what level of accomplishment
As determined by what means
Keeling, Wall, Underhile, & Dungy
Post-Session Questionnaire
1. Which of the following is NOT an
indicator of Institutional Effectiveness
A. Student usage/participation data for a facility or
program
B. Student satisfaction surveys
C. SLO assessment
2. Meaningful use of SLO requires
A. That learning must be measured
B. The learning being measured must be the
learning that was intended
C. That the results of assessment must be used to
refined programs/experiences
D. All of the above
3. If your intended SLO is for participants to be able
to demonstrate acquisition of conflict resolution
skills, the most relevant assessment measure would
be:
A. Questionnaire of understanding of conflict resolution
techniques
B. Professional observation of student performance
C. Peer perception of skills
4. As a result of attending this session, my
confidence in my ability to apply SLOs to the
development of leadership programs and
experiences is:
A. No better than before
B. Somewhat better than before
C. Significantly better than before
Explanation
From the Student Learning Imperative:
Student Affairs must model what we wish
for our students: an ever increasing capacity
for learning and self-reflection.
THANK YOU FOR YOUR PARTICIPATION!!!
Dr. Douglass F. Covey,
[email protected]
Dr. Anita L. Vorreyer,
[email protected]