Transcript NSSE: A Window into Student and Institutional Performance George Kuh University of Maine Trustees Farmington ME March 19, 2007

NSSE:
A Window into Student
and Institutional
Performance
George Kuh
University of Maine Trustees
Farmington ME
March 19, 2007
Commission on Future of Higher Education
1.
2.
3.
4.
5.
6.
Improve student academic preparation and
make more financial aid available
Simplify & restructure student financial aid
system using incentives to control costs
Cultivate a "robust culture of
accountability & transparency“: invent
new student learning measures, make
comparable information public, focus
accreditation on outcomes
Embrace continuous innovation and
quality improvement.
Target federal investments to critical areas such
as math, science, and foreign languages.
Develop a national strategy for lifelong learning
Association of American Colleges and Universities
We value what we measure
Wise decisions are
needed about what to
measure in the context
of campus mission,
values, and desired
outcomes.
Overview
1. Why student engagement
matters
2. What we’ve learned from NSSE
3. Unsolicited advice for trustees
4. Conversation
Student Success in College
Academic achievement,
engagement in
educationally purposeful
activities, satisfaction,
acquisition of desired
knowledge, skills and
competencies,
persistence, attainment
of educational
objectives, and postcollege performance
Pre-college Characteristics
Associated with Student Success
Academic preparation
Ability and college-level skills
Family education and support
Financial wherewithal
32.N162FG12
Postsecondary Education OPPORTUNITY
Illinois
Association for Institutional Research – May 16, 2006 – Chicago,
Early College Indicators of
Persistence and Success
 Goal realization
 Psycho-social fit
 Credit hours completed
 Academic and social support
 Involvement in the “right” kinds
of activities
Factors That Threaten Persistence and
Graduation from College
 academically underprepared for
college-level work
 first-generation college student
 gap between high school and college
 30+ hours working per week
 part-time enrollment
 single parent
 financially independent
 children at home
What Really Matters in College:
Student Engagement
Because individual effort and
involvement are the critical
determinants of impact,
institutions should focus on
the ways they can shape their
academic, interpersonal, and
extracurricular offerings to
encourage student
engagement.
Pascarella & Terenzini, How College Affects
Students, 2005, p. 602
Foundations of Student Engagement
Time on task (Tyler, 1930s)
Quality of effort (Pace, 1960-70s)
Student involvement (Astin,
1984)
Social, academic integration
(Tinto,1987, 1993)
Good practices in
undergraduate education
(Chickering & Gamson, 1987)
College impact (Pascarella, 1985)
Student engagement (Kuh, 1991,
2005)
Good Practices in
Undergraduate Education
(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005)







Student-faculty contact
Active learning
Prompt feedback
Time on task
High expectations
Experiences with diversity
Cooperation among students
The Student Engagement Trinity
What students do -- time and energy
devoted to educationally purposeful
activities
What institutions do -- using
effective educational practices to
induce students to do the right
things
Educationally effective institutions
channel student energy toward the
right activities
NSSE’s Survey Instrument
The College Student Report
Student Behaviors
Institutional Actions &
Requirements
Reactions to College
Student Background
Information
Student
Learning &
Development
National Survey of
Student Engagement
(pronounced “nessie”)
Community College
Survey of Student
Engagement
(pronounced “cessie”)
College student surveys that assess
the extent to which students engage
in educational practices associated
with high levels of learning and
development
Effective Educational Practices
Level of
Academic
Challenge
Student
Faculty
Interaction
Enriching
Educational
Experiences
Active &
Collaborative
Learning
Supportive
Campus
Environment
NSSE Project Scope
 1,000,000+ students from 1,100+
different schools
 80% of 4-yr U.S. undergraduate
FTE
 50 states, Puerto Rico, Canada
 70+ consortia
NSSE Core Purposes
Institutional
Improvement
Public
Advocacy
Documenting
Good Practice
Customized Institutional Report
Overview
Institutional data
Means and frequencies
1st year students and
seniors
Comparisons by peers,
Carnegie, national
National benchmarks
Data use tips
CD with raw data, etc.
And more!
What have we
learned so far?
Grades, persistence,
student satisfaction,
and engagement go
hand in hand
Does institutional size matter to
engagement?
Yes, size matters.
Smaller is generally
better.
Benchmark Scores for All Students by
Undergraduate Enrollment
Benchmark Scores for All Students by Undergraduate Enrollment Intervals
Level of
Academic
Challenge
65
Active and
Collaborative
Learning
55
Student
Interactions
with Faculty
Members
Enriching
Educational
Experiences
45
35
25
01000
1001 1500
1501 2000
2001 2500
2501 3000
3001 4000
4001 - 5001 5000 7500
Enrollment Intervals
7501 - 10001 - 15001 - 20001 - 25001 10000 15000 20000 25000 highest
Supportive
Campus
Environment
Academic Challenge, Active Learning,
Student-Faculty Interaction by Enrollment
Level of Academic Challenge
Active and Collaborative Learning
Student Interactions with Faculty Members
60
50
40
30
20
186
1238
1921
3060
Enrollment
6337
11343
29426
Student engagement varies
more within than between
institutions.
Supportive Campus Environment:
Seniors at Master's Institutions
Percentile 10
Percentile 50
Percentile 90
100
80
60
40
20
0
1
2
3
4
5
6
7
8
9
10
Master's Institutions
11
12
13
14
Academic Challenge by Institutional Type
75
70
75
First-Year Students
Seniors
70
65
60
60
55
55
50
50
45
45
40
40
35
35
30
30
25
Doc Doc
Ext Int
25
Doc Doc
Ext Int
Benchmark Scores
65
MA
Bac Bac Nation
LA Gen
MA
Bac Bac
LA Gen Nation
Behold the compensatory
effects of engagement
Association of American Colleges and Universities
Narrow Learning is Not Enough—
The Essential Learning Outcomes

Knowledge of Human Cultures
and the Physical & Natural World
 Intellectual and Practical Skills
 Personal and Social Responsibility
 Integrative Learning
Essential Learning Outcome:
Integrative Learning
NSSE DEEP LEARNING SCALE
 Integrating ideas or information
from various sources
 Synthesizing & organizing ideas,
info., or experiences
 Included diverse perspectives in
class discussions/writing
 Making judgments about the
value of information
 Put together ideas from different  Applying theories to practical
problems or in new situations
courses
 Examined the strengths and
 Discussed ideas with faculty
weaknesses of your own views
members outside of class
 Tried to better understand
 Discussed ideas with others
someone else's views
outside of class
 Analyzing the basic elements of  Learned something that
changed how you understand
an idea, experience, or theory
an issue
Essential Learning Outcome:
Intellectual and Practical Skills
NSSE SELF-REPORTED GAINS
 Acquiring job or work-related knowledge and skills
 Writing clearly and effectively
 Speaking clearly and effectively
 Thinking critically and analytically
 Analyzing quantitative problems
 Using computing and information technology
 Working effectively with others
 Solving complex real-world problems
Essential Learning Outcome:
Personal & Social Responsibilities
NSSE SELF-REPORTED GAINS
 Voting in local, state, or national elections
 Learning effectively on your own
 Understanding yourself
 Understanding people of other racial and ethnic backgrounds
 Developing a personal code of values and ethics
 Contributing to the welfare of your community
 Developing a deepened sense of spirituality
Gains in Intellectual and Practical Skills
by Deep Learning Quartiles
Very much
First-Year
Senior
Quite a bit
Some
Very little
lowest
second
third
Deep Learning Quartiles
highest
Gains in Personal and Social Responsibilities
by Deep Learning Quartiles
Very much
First-Year
Senior
Quite a bit
Some
Very little
lowest
second
third
Deep Learning Quartiles
highest
Effective Educational Practices
 First-Year Seminars and Experiences
 Common Intellectual Experiences
 Learning Communities
 Writing-Intensive Courses
 Collaborative Assignments and Projects
 “Science as Science Is Done”;
Undergraduate Research
 Diversity/Global Learning
 Service Learning, Community-Based
Learning
 Internships
 Capstone Courses and Projects
Effects of Learning Communities on Engagement
First-year
Senior
Standardized
YRegression
Standardized
Coefficient Sig. Effect size
Standardized
YRegression
Standardized
Coefficient Sig. Effect size
Engagement Activities
Academic Efforts
Higher Order Thinking
Academic Integration
Active and Collaborative Learning
Interactions with Faculty
Diversity Experiences
.16
.20
.19
.26
.30
.21
***
.12
.19
.13
***
.24
.22
.18
***
***
***
***
***
***
.32
.40
.39
.53
.60
.41
.12
.15
.16
.24
.22
.16
***
.23
.37
.25
.08
.14
.10
***
.48
.45
.36
.18
.16
.11
***
***
***
***
***
***
.28
.35
.38
.54
.51
.36
Perception of Campus Environment
Quality of Academic Advising
Supportive Campus Environment
Satisfaction
***
***
***
***
.17
.32
.23
Learning Outcomes
Gains in Personal and Social
Gains in Practical Competence
General Education Gains
***
***
***
***
.40
.36
.24
Dependent variable
First-year students
Seniors
Diversity
Diversity
Diversity
Diversity
Density Climate In Course- Diversity Density Climate In Course- Diversity
Index Diversity Work
Press
Index Diversity Work
Press
Student Engagement
Academic challenge
Higher order thinking
Active and Collaborative
Diversity-related activities
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Supportive Campus Env.
Supportive Campus Env.
Interpersonal
-
Support for learning
Satisfaction
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Gains-Interpers. Dev.
Gains - Personal/social
Gains-Social Awareness
Gains-Understanding div.
Gains -Cont. to community
Gains-Understand self
+
Probability of Undergraduate Research
Faculty Time Devoted to Research and
Probability of Students Doing Research
0.50
0.38
0.25
0.13
0
2.02
6.04
10.07
Faculty Research (hours)
14.09
18.11
Faculty View of Importance of Doing Research with
Students and Probability of Students Doing Research
Probability of Undergraduate Research
0.50
0.38
0.25
0.13
0
1.87
2.25
2.62
2.99
Importance of Undergraduate Research
3.37
Faculty Priorities and
Student Engagement
AVG STUDENT
Acad emic
challenge
Activecollab
Diversity
experiences
Academic chall enge
emph asis



Active-collab
practices



Emph asis on diversity
experiences



Emph asis on higher
order thinking



Importance enriching
educ experiences


AVG FACULTY
Student faculty


What to Make of This?
1. When faculty members
emphasize certain educational
practices, students engage in
them to a greater extent than
their peers elsewhere.
2. Good things go together
Effective Educational Practices
 First-Year Seminars and Experiences
 Common Intellectual Experiences
 Learning Communities
 Writing-Intensive Courses
 Collaborative Assignments and Projects
 “Science as Science Is Done”;
Undergraduate Research
 Diversity/Global Learning
 Service Learning, Community-Based
Learning
 Internships
 Capstone Courses and Projects
Effective Educational Practices Increase
Odds That Students Will:
 Invest time and effort
 Interact with faculty and peers
about substantive matters
 Experience diversity
 Get more frequent feedback
 Discover relevance of their
learning through real-world
applications
Institutional Reflection
Areas of
Effective
Educational
Practice
Areas of
Question or
Improvement
Possibilities
 Merge NSSE data
with school records
 Identify engagement
patterns by student
characteristics
 Predict retention,
degree attainment,
grades, other
outcomes
 Track student
engagement year to
year
 Compare vs. peer,
aspirational, sameregion schools
 Program evaluation
 Accreditation
 Consortium and
system data sharing
 Student outcomes
research
 Public reporting
 A national reporting
template?!?
NSSE Accreditation Toolkit
NSSE Questions
• Links Between
NSSE &
Accreditation
• Mapping NSSE
to Accreditation
Standards
• Vignettes of
Institutional
Usage
1.l.
l. Used an electronic medium (list-serv, chat
group, Internet, etc.) to discuss or complete an
assignment
1.m.
Used e-mail to communicate with an instructor
10.g.
Using computers in academic work
11.g.
Using computing and information technology
SACS Standard
3.4.14
The institution’s use of technology enhances
student learning, is appropriate for meeting the
objectives of its programs, and ensures that
students have access to and training in the
use of technology.
Questions Trustees Should Ask
• To what extent does the
institutional mission—espoused
and enacted—unmistakably
emphasize student success?
• How does the institution’s
spending plan enhance student
learning and success?
• What resources are available to
support trying out good ideas?
Put money where it will make a
difference in student engagement
“…in professional
baseball it still
matters less how
much you have
than how well you
spend it”
Endowments & Undergraduate Enrollments
of Selected Public DEEP Institutions
Evergreen
$2,395,656
4,171 students
UMF
$9,391,000
2,452 students
Fayetteville State
$8,465,541
5,029 students
George Mason
$40,812,720
18,091 students
Source: US News America’s Best
Colleges 2007
• What institutional and studentperformance indicators are used
to inform board decisions?
• Who is responsible for
maintaining an institutional focus
on student success and reporting
these key indicators to the
board?
• Does the institution participate in
NSSE and how are the results
being used (Bok, 2006)?
• Where do students and their
learning fall among the board’s
interests and priorities?
• Is a standing committee
dedicated to student learning
and campus life?
• Does the culture of the board
encourage or discourage open,
productive dialogue about
matters related to student
success?
Why stay the course?
The good-to-great-transformations
never happened in one fell swoop.
There was no single defining
action, no grand program, no one
killer innovation, no solitary lucky
break, no miracle moment.
Sustainable transformations
follow a predictable pattern of
buildup and breakthrough…
(Collins, 2001, p. 186)
Last Word
A college or university cannot change
the lineage of its students. Campus
cultures do not change easily or
willingly. But we can do far more to
shape the way students approach
college and what they do after they
arrive.
Do we have the will to use promising
policies and educational practices to
increase the odds that more students
“get ready,” “get in,” and “get
through?”
Questions &
Discussion