Transcript Slide 1

Using Student Engagement to
Stimulate Change on Campus
John Hayek, Ph.D.
Senior Associate Director
Advance Organizer
What kind of
information about
the student
experience is
compelling and
useful for stimulating
change and
improvement on
campus?
Overview
NSSE Primer
Ways to Stimulate
Change
Institutional Examples
Open Discussion
Foundations of Student Engagement
Assessing the Student Experience
1970s
Quality of Student Effort (Pace)
1980s
Student Involvement (Astin)
Social & Academic Integration
(Tinto)
Good Practices in Undergraduate
Education (Chickering & Gamson)
Learning and Development Model
(Pascarella)
1990s
Student Engagement (Kuh)
National Survey of
Student Engagement

Started in 1999 with 12 institutions – grown
to 530+ in NSSE 2005

Over a half million students (first-year
students and seniors) at 850 colleges and
universities (2000-2004)

Focuses on promoting effective educational
practice and institutional improvement

Web and paper versions; extensively tested
to ensure validity and reliability

Assesses the extent to which student are
engaged in educational practices related to
high levels of learning and development
Using Student Engagement To
Stimulate Change
1) Link to Mission &
Accreditation
2) Share Information Widely
3) Enhance Faculty
Development
4) Benchmark (External &
Internal)
5) Connect to Outcomes and
Other Campus Data
6) Emphasize Effective
Educational Practices
1. Link to Mission & Accreditation
AASCU Mission Statements
Key Words
Diversity
Teaching and Learning
General Education
Information Technology
Liberal Arts
Community
Productive Citizens
Scholarship
Research
Academic Excellence
High Quality Education
Critical Thinking
Health and Wellness
Student Development
Comprehensive
Global
NSSE
Areas of Focus
Academic & Social Experiences
Technology
Diversity
Higher Order Thinking
Reading and Writing
Time Usage
Enriching Educational Experiences
Quality of Relationships with
Students, Faculty, and Staff
Arts, Wellness, & Spirituality
Civic Engagement
Campus Environment
Advising and Mentoring
Satisfaction
Personal and Educational Growth
1. Link to Mission & Accreditation
NSSE Accreditation
Toolkit
 Links Between NSSE
& Accreditation
 Mapping NSSE to
Accreditation
Standards
 Vignettes of
Institutional Usage
2. Share Information Widely
“NSSE is a great way to stimulate reflection and
debate about what we do more and less well, and
why. For us it’s proving an exciting and enlivening
tool for self-reflection and self-improvement.”
Michael McPherson,
President of The Spencer Foundation
(former President of Macalaster College)
Internal
External
President
Governing Boards
Faculty / Committees / Deans / Chairs
Accreditation
Students / Groups / Organizations
Alumni
Service Learning
Prospective Students
Enrollment Management / Admissions
Media
Student Affairs / Student Services
Parents
First-year Experience
Fund Raising
Advising
State Policy Makers
Assessment & Institutional Research
Performance Indicators
2. Share Information Widely
3. Enhance Faculty Development
• Faculty Retreats &
Workshops
• Scholarship of Teaching
& Learning
• Course Evaluations
• Tip of the Week
• Incentive Programs
• FSSE / NSSE Gap
Analysis
Mini-Grant: Early Engagement of
First-Year Students

Vice President for Academic Affairs

Identified start-up resources (up to
$2,500 per year for two years)

Help academic departments
introduce students engagement
initiatives

Designed to improve department’s
engagement of its students during
their first semester at the university
3. Enhance Faculty Development
FSSE / NSSE Gap Matrix
Mass Comm 4307 Importance x Engagement Matrix
Faculty Say
“Important” or
“Very Important”
Miss
Faculty Say
“Important” or
“Very Important”
50% +
Students Say
Never
(14) Participate in a community-based project
(18) Talk about career plans with instructor
Faculty Say
“Not Important”
Hit
50% +
Students Say
Never
(13) Tutor/teach other students
50% +
Students Say
“More than 5 times”
(1) Ask questions during class
(9) Work with classmates on projects during class
(11) Put together ideas/concepts form different courses when
completing assignments
(12) Put together ideas/concepts from different courses
during class discussions
(24) Analyze basic elements of an idea, experience, or theory
(25) Synthesize/organize ideas, information, or experiences
(26) Make judgments about the value of information,
arguments, or methods
(27) Apply theories/concepts to practical problems
Faculty Say
“Not Important”
Hit
50% +
Students Say
“More than 5 times”
(10) Work with other students outside of class to
prepare assignments
(22) Discuss ideas from class with others outside of class
Miss
4. Benchmark - External
NSSE 2004 Means Comparison Report
National Survey of
Student Engagement
Nesseville State University
Nesseville State compared with:
Nesseville State
Variable
1.
•ADP
• Peer
Groups
• Aspirant
Groups
• Special
Analyses
b.
Class
Asked questions in class or contributed to class
discussions
CLQUEST
Made a class presentation
CLPRESEN
Prepared two or more drafts of a paper or
assignment before turning it in
REWROPAP
d.
Worked on a paper or project that required
integrating ideas or information from various
sources
INTEGRAT
e.
Included diverse perspectives (different races,
religions, genders, political beliefs, etc.) in class
discussions or writing assignments
DIVCLASS
f.
Come to class without completing readings or
assignments
CLUNPREP
g.
Worked with other students on projects during
class
CLASSGRP
Worked with classmates outside of class to prepare
class assignments
OCCGRP
Put together ideas or concepts from different
courses when completing assignments or during
class discussions
INTIDEAS
i.
Mean
Mean
Sig
a
Size
b
Effect
Mean
Sig
a
Size
b
In your experience at your institution during the current school year, about how often have you done each
of the following? 1=never, 2=sometimes, 3=often, 4=very often
c.
h.
NSSE 2004
Effect
Academic and Intellectual Experiences
a.
• Consortia
Benchmark
Master's
ACL
ACL
ACL
ACL
FY
1.96
2.50
2.69
SR
3.11
3.10
3.18
FY
2.38
2.23
SR
2.65
2.74
FY
2.96
2.49
**
.29
SR
2.53
2.66
**
.27
FY
3.20
3.04
3.05
SR
3.27
3.35
3.35
FY
2.82
2.73
2.73
***
.48
2.24
***
.50
2.44
**
.32
2.68
***
.32
2.45
SR
2.73
2.81
2.81
FY
1.77
1.89
1.99
*
-.24
SR
1.82
2.10
2.03
*
-.18
FY
2.45
2.37
SR
2.68
2.51
*
.20
2.44
**
.28
FY
2.05
2.32
**
-.33
2.39
***
-.41
SR
2.54
2.71
*
-.19
2.73
**
-.21
FY
2.57
2.44
SR
2.68
2.84
*
-.22
2.33
2.47
*
-.20
2.86
4. Benchmark - External
Level of Academic Challenge
Level of Academic Challenge Items:
75
Preparing for class (studying, reading, writing, rehearsing,
etc. related to academic program)
Number of assigned textbooks, books, or book-length
packs of course readings
65
Benchmark Scores
Challenging
intellectual and
creative work is
central to student
learning and
collegiate quality.
Colleges and
universities
promote high
levels of student
achievement by
emphasizing the
importance of
academic effort
and setting high
expectations for
student
performance
Number of written papers or reports of 20 pages or more;
number of written papers or reports of between 5 and 19
pages; and number of written papers or reports of fewer
than 5 pages
55
45
Nesseville
Coursework
emphasizing
synthesis and organizing of ideas,
Consortium
Carnegie
National
information, or experiences into new, more complex
interpretations and relationships
35First-Year
52.5
52.4
25Senior
56.3
55.6
Senior
Nesseville
52.5
56.3
Consortium
52.4
55.6
Carnegie
51.8
54.9
National
53.4
57.0
51.8
53.4
Coursework emphasizing the making of judgments about
the value of information, arguments, or methods
57.0
Coursework54.9
emphasizing application
of theories or
concepts to practical problems or in new situations
Working harder than you thought you could to meet an
instructor's standards or expectations
Campus environment emphasizing time studying and on
academic work
2003 IUB Benchmark Deciles
First-Year
Senior
0
10
20
30
40
50
60
70
80
90
100
0
10
20
30
40
50
60
70
80
90
100
Level of Academic
Challenge
45
49
50
51
52
53
55
56
57
60
68
40
53
54
55
56
57
58
59
60
63
74
Active and Collaborative
Learning
30
36
37
39
40
41
43
44
46
48
60
38
45
47
48
49
50
51
52
54
55
66
Student-Faculty
Interaction
23
31
32
34
35
36
38
40
41
45
74
28
36
38
40
42
43
45
47
50
54
70
Enriching Educational
Experiences
40
48
51
53
56
58
59
61
64
67
80
31
41
42
44
46
48
51
53
55
59
75
Supportive Campus
Environment
46
55
57
59
60
62
63
65
66
69
85
45
51
54
55
57
59
60
62
64
66
76
Res/Doc-Ext
Act.
Pred.
Residual
Standard
Residual
Academic Challenge
56.0
52.4
3.5
1.3
Active Learning
43.5
41.4
2.1
.6
Stu-Fac Interaction
42.7
35.8
6.9
1.7
Enriching Experience
50.6
50.4
.2
0.0
Supportive Environment
70.8
63.3
7.5
2.0
Coursework emphasizing analysis of the basic elements of
an idea, experience or theory
First-Year
National
Benchmark
0
10
20
30
40
50
60
70
80
90
100
0
10
20
30
40
50
60
70
80
90
100
Level of Academic
Challenge
47
48
49
50
51
51
53
54
55
57
62
50
52
53
54
55
55
55
57
57
59
61
Active and Collaborative
Learning
33
35
35
36
37
38
38
40
41
42
47
39
43
44
45
45
46
47
47
48
49
56
Student-Faculty
Interaction
29
31
32
32
33
34
34
35
36
38
44
31
36
37
38
38
39
41
42
43
44
50
Enriching Educational
Experiences
48
52
53
55
56
58
59
60
61
64
71
39
43
44
45
46
47
48
49
52
54
58
Supportive Campus
Environment
50
54
55
56
57
58
59
59
60
61
72
45
48
50
51
52
53
54
55
57
58
69
4. Benchmark – Internal
4. Benchmark – Internal
60
Biochemistry
Lowest Major
Chemistry
Average
55
Theater or
Drama
Highest Major
Political
Science
Elem./Middle
Education
50
Pre-Med
Kinesiology
Chemical
Engineering
45
Environmental
Science
International
Business
Math
Speech
40
Physical
Education
Mechanical
Engineering
35
Pharmacy
Criminal
Justice
Management
Business
30
Sociology
Engineering
Other
Education
Professional
Arts &
Humanities
Social
Sciences
Biological
Sciences
Math &
Physical
Sciences
5. Connect to Outcomes &
Other Campus Data
Retention Rates to Second Year by Engagement
100
91
84
84
93
91 92 91
91
88
90
80
95
93 92
84
92 93
89
87 88
85 86
82
89
85
82
79
70
60
50
40
30
20
10
0
Level of Academic
Challenge
Academic and
Collaborative Learning
Very Low
Low
Student Interaction
w ith Faculty
Average
High
Enriching Educational
Experiences
Very High
Supportive Campus
Environment
5. Connect to Outcomes &
Other Campus Data
Grad. Rate
Satisfaction
Satisfaction
SR
FY
SR
Academic Challenge
.46
.29
.28
Active & Collaborative Learning
.09
.25
.23
Student Faculty Interaction
.37
.25
.29
Enriching Educational Experiences
.48
.22
.23
Supportive Campus Environment
.26
.56
.60
NSSE Benchmarks
5. Connect to Outcomes &
Other Campus Data
 In-house surveys
 National surveys
 CIRP / CSS
 YFCY
 CSEQ / CSXQ
 EBI Benchmarking surveys
 Noel Levitz Student Satisfaction Inventory
 ETS Major Field Tests
 ACT Collegiate Assessment of Academic
Proficiency
 Institutional data such as GPA, financial aid,
transcripts, retention, certification tests, etc.
6. Emphasize Effective
Educational Practices
1)
A “living” mission and a “lived”
educational philosophy
2)
An unshakeable focus on student
learning
3)
Clearly marked pathways to student
success
4)
Environments adapted for educational
enrichment
5)
An improvement-oriented campus
culture
6)
Shared responsibility for educational
quality and student success
Based on higher
than predicted
graduation rates
and student
engagement
Institutional Examples
Many schools are stimulating change and
improvement on campus by using student
engagement data.
University of Missouri – St. Louis
Using NSSE Data to Stimulate Change
AASCU 2005 San Diego Meeting
Glen Hahn Cope
Provost and Vice Chancellor
for Academic Affairs
University of Missouri in St. Louis
 Urban public research university
 Recent leadership and vision
changes
 UM System Strategic Planning
 UMSL Action Planning
 NSSE Participation
(begins with 2000 pilot)
 From “administrative use only” in
2000 to
2003, 2004 campus-wide forums
 From “denial of results” to colleges
verifying, faculty discussions, and
Action Plan benchmarks
University and Campus Activities
 University System-wide Efforts
 UM New Faculty Teaching Scholars
 President’s Academic Leadership Institute
 Campus-wide Efforts
 Center for Teaching and Learning (est. 10/2000)
 Faculty and TA programs, orientations
 Kuh presentations with academic and student
affairs leaders, early career faculty (2/02)
 Engagement concepts routinely used in program
names
 UMSL at 40:Campus Conversation Series 2003-2004




November: Student engagement
February:
Engaged research
April:
Community Engagement
February ‘05:Creating an Engaged University
 Action Planning 2003-2004
 Office of the Provost created August 2004
 Reliance on faculty governance, input
 Benchmarks with NSSE data
College Activities
 College efforts – sampling
 Arts and Sciences appended with
permission 15 NSSE items to fall
semester 2002 course evaluations
 College of Business Administration
includes all majors in capstone course
in its sample
 Honors College oversamples its
freshmen and seniors
 College efforts – discussion and action
 Presentations about NSSE invited by
COE, CoBA
 A & S Dean’s charge to departments
 Embracing undergraduate research
Outcomes
 Increased campus-wide awareness (FSSE,
NSSE)
 Persistent conversations
 Increased communication
 Responsibility assumed at unit level
 Increased acceptance of methodology, data
 Interventions identified in colleges and
departments
 Increased response rates
2003
2004
NSSE
38.0%
47.5%
FSSE
26.4%
44.5%
Norfolk State University (NSU)
• University of opportunity
• Wide variety of programs for students seeking
access to an affordable high-quality education
• Founded in 1935
• Located in the downtown Norfolk, Virginia
• Virginia’s largest public historically black
university (HBCU)
• Seventh largest HBCU in the nation
• Approximately 6,000 culturally diverse students
Surveys of Student Engagement at NSU
• Spring 2002
– NSSE (paper mode)
• Spring 2003
– NSSE (web-based mode)
– FSSE (web-based mode)
• Spring 2004
– NSSE (local administration)
• Fall 2004
– BCSS (paper mode)
• Spring 2005
– NSSE (web+ mode)
Using NSSE for
Quality Enhancement
• NSSE Data
• NSSE Process
• NSSE Concept
Using NSSE for
Quality Enhancement: Data
• Set up internal and external benchmarks to
assess and monitor NSU performance on
NSSE benchmarks and individual items,
salient for NSU
• Provide information for internal decisionmaking and strategic planning
• Engage faculty, administrators, and students in
Using NSSE for
Quality Enhancement: Data (Cont’d)
• Advance campus initiatives
• Articulate and affirm effective institutional
practices and improve NSU self-image and
community perception
• Triangulate internal reports and research
projects
• Triangulate external reporting
Using NSSE for
Quality Enhancement: Process
• Provide information for internal decisionmaking
• Advance campus initiatives
• Identify effective methods to administer other
university-wide surveys.
Using NSSE for
Quality Enhancement: Concept
• Begin developing a more comprehensive
institutional concept of academic quality
• Attract faculty and administrators’ attention to
best practices in the undergraduate education
Using NSSE for
Quality Enhancement: Future Plans
• Assessment of new campus initiatives
– First-Year Experience
– American democracy project (ADP)
• Reaffirmation of Accreditation
– Southern Association of Colleges and Schools (SACS)
Compliance Certification Audit
– SACS Quality Enhancement Plan (QEP) Theme
Contact Information
Nuria M. Cuevas, Ph.D.
Associate Vice President for Academic Affairs
and
Director, Institutional Effectiveness and
Assessment
NORFOLK STATE UNIVERSITY
Phone: (757) 823-8408
E-Mail: [email protected]
Using NSSE Data
Sharon Hahs
Provost
February 6, 2005
◆ In Illinois, 20
minutes from the
St. Louis Arch
◆ Only public
university in
Southwestern
Illinois
◆ Most populous
region of
downstate
Illinois
A Metropolitan University in a Pastoral Setting
◆ St. Louis
metro area
includes 2.7
million people
With Programs in:
◆ Arts and Sciences
◆ Business
◆ Dental Medicine
◆ Education
◆ Engineering
◆ Nursing
◆ Pharmacy
◆ 13,493 Students
10,811 Undergraduate
2,485 Graduate
198 Professional
◆ On-Campus Housing for 2,900 Students
◆ Most students live and work in a 60 mile
radius and commute to classes.
◆ About 30 percent of undergraduates are
new each fall, two-fifths of those as transfer
students.
◆ More than half of students receiving
baccalaureate degrees started as transfer
students.
Long Term Goals
◆ Revisited Every 5 to 10 Years
◆ Measured Annually
◆ Lead to Short Term Goals
Short Term Goals
◆ One to Three Years Long
◆ Founded on evidence
◆ Measured and Evaluated
◆ Lead to Actionable Projects
SIUE’s Long-Term Goals
1. Engaged Student and Capable
Graduates
2. Innovative High Quality Programs
3. Committed Faculty and Staff
4. Harmonious Campus Climate
5. Active Community Engagement
6. Sound Physical and Financial Assets
7. Excellent Reputation
Measures for Long Term Goals
NSSE:
◆ Multiple Measures
◆ Longitudinal Data
◆ External Comparative Data
◆ Intra-Institutional Data
Has content parallel to existing
alumni and faculty surveys
Annual since 2000
Urban Consortium
Can be “cut” to
components
within SIUE
Uses of NSSE in Measuring
Long Term Goals
◆ Measures of Student Engagement
◆ Measures of Faculty and Staff
Commitment to Educational Opportunity
◆ Measures of Harmonious Climate
◆ Measures of Community Engagement
◆ Measures of SIUE Reputation
Freshman Seminar Proposal
April 2002
• Piloted four options:
– Honors Seminars
– University Experience Course
– Culture, Ideas, Values Course (CIV)
– Learning Communities in Academic
Development
Common goals for freshman seminars—
Freshman Seminar Proposal 2002
1--To assist new freshman in making the
transition to college level work and
expectations;
2--To orient students to the services and
culture of the University;
3--To engage students in an intellectual
community of students and faculty.
Review committee and use of
special study
• Ad hoc subcommittee of the Curriculum
Council, Faculty Senate
– Literature Search
– Focus Groups
– NSSE study
• Summary Report—NSSE 2003 Special
Course Oversample April 19, 2004
Of the questions that showed significance at the 0.1 level,
four either directly or indirectly related to the objectives of
the freshman seminar course as outlined in the proposal.
These included the following questions:
1 A--Asked questions class or contributed to class
discussion.
10 F--Attending campus events and activities (special
speakers, cultural performances, etc.)
11 K--Understanding yourself
13--How would you evaluate your entire educational
experience at this institution?
(Summary Report—NSSE 2003 Special Course Oversample April 19, 2004)
New Student Seminar Task Force Report and
Recommendations (June, 2004)
• Recommendation 1: Adopt a freshman
seminar requirement
• Recommendation 2: Include a freshman
seminar in general education reform.
• Recommendation 3: Create committee for
implementation and management.
 Question 7. Which of the following have you
done or do you plan to do before you graduate
from your institution?
 h. Culminating senior experience (comprehensive
exam, capstone course, thesis, project, etc.)
• Response to 7.h.:
– Yes—70%
– No—25%
– Undecided—5%
Senior Assignment is a graduation requirement
Activity--NSSE questions
related to Learning Objectives
• Compare the NSSE questions with the
SIUE Statement of Objectives
– 1. Identify a NSSE question that could make
a difference.
– 2. What objective does this question
measure and how does the question
measure it?
– 3. How is this question actionable? What
could be done to improve the score?
Committee on Assessment
AQIP Action Project Recommendations
• Student perception and understanding
of the Senior Assignment as a
culminating experience (NSSE questions
2c, 7g,h, 11j,m)
• Student perception of academic
advising (NSSE questions 1o, 12)
• Quantitative reasoning (NSSE questions
2b,d,e, 11f)
• Communication (speaking and writing)
(NSSE questions 1a,p,q, 4c,d,e, 11c,d)
• Expectations, relationships, and
diversity (NSSE questions 8, 9)
Open Discussion
www.nsse.iub.edu