Transcript Engaging Community Colleges A First Look

Maximizing Your
NSSE & CCSSE
Results
Jillian Kinzie
Associate Director
NSSE
Nathan Marti
Senior Research
Associate
CCSSE
Agenda
Why is assessment of student
engagement important?
What is student engagement?
NSSE & CCSSE fundamentals
Interpreting NSSE and CCSSE data
Using benchmarks
Q&A
SDSU Assessment Conference NSSE/CCSSE Workshop
Introductory Activity
 What questions do you have coming into this
session?
 Assessment at your institution:
 What do you want to know about your
students?
 Why do you want to know this?
 What is the purpose of your assessment
initiative(s)?
 To what extent have you used NSSE and
CCSSE data?
SDSU Assessment Conference NSSE/CCSSE Workshop
Why assessment of student
engagement?
“Colleges and universities, for all
the benefits they bring,
accomplish far less for their
students than they should.”
-- Derek Bok
“We can tell people almost
anything about education except
how well students are learning.”
-- Patrick M. Callan, President,
National Center for Public Policy and Higher
Education
SDSU Assessment Conference NSSE/CCSSE Workshop
What Do We Need to Know?
What constitutes quality
in undergraduate
education?
How might we measure
quality?
What types of evidence
of quality would be
helpful in guiding
improvement efforts?
SDSU Assessment Conference NSSE/CCSSE Workshop
Foundations of Student Engagement
Quality of effort (Pace)
Student involvement (Astin)
Social and academic
integration (Tinto)
Good practices in
undergraduate education
(Chickering & Gamson)
Learning and development
model (Pascarella)
SDSU Assessment Conference NSSE/CCSSE Workshop
Agreement within the Research
What matters most is
what students do, not
who they are.
Educationally effective
institutions channel
student energy toward the
right activities.
SDSU Assessment Conference NSSE/CCSSE Workshop
Principles of Good Practice in
Undergraduate Education
Student-faculty contact
Active learning
Prompt feedback
Time on task
High expectations
Experiences with diversity
Cooperation among students
(Chickering & Gamson, 1987)
SDSU Assessment Conference NSSE/CCSSE Workshop
Path Model for Assessing Change in
Student Learning (Pascarella)
A General Causal Model for Assessing the Effects of Differential College Environments on
Student Learning and Cognitive Development
Interactions
with Agents of
Socialization
Structural/
Organizational
Characteristics
of Institutions
Learning and
Cognitive
Development
Student
Background/
Precollege
Traits
Institutional
Environment
Quality of
Student Effort
SDSU Assessment Conference NSSE/CCSSE Workshop
What Really Matters in College:
Student Engagement
Because individual effort
and involvement are the
critical determinants of
college impact, institutions
should focus on the ways
they can shape their
academic, interpersonal,
and extracurricular
offerings to encourage
student engagement.
Pascarella & Terenzini, How College Affects Students,
2005, p. 602
SDSU Assessment Conference NSSE/CCSSE Workshop
Two Assertions
The time and energy students devote to
educationally purposeful activities is the
single best predictor of their learning and
personal development.
These things can be influenced and
measured by institutions.
SDSU Assessment Conference NSSE/CCSSE Workshop
Two Components of Student Engagement
1. What students do –
time and energy devoted
to educationally
purposeful activities
2. What institutions do –
using effective
educational practices to
induce students to do
the right things
SDSU Assessment Conference NSSE/CCSSE Workshop
Educational Process Indicators
National Survey of
Student Engagement
(pronounced “nessie”)
Community College Survey of
Student Engagement
(pronounced “sessie”)
College student surveys that assess the extent
to which students engage in educational
practices associated with high levels of
learning and development
SDSU Assessment Conference NSSE/CCSSE Workshop
NSSE – The College Student Report
Based on effective educational practices
Items have face and content validity
Designed & tested for validity and reliability
Relatively stable over time
Credibility of self-reported
data
Students will participate
Actionable data
SDSU Assessment Conference NSSE/CCSSE Workshop
NSSE Administration
Centrally administered by third party
Spring term
Random samples of first-year & senior students
Administration modes:



Paper
Web-only
Web +
40% response rate
Over 275,000 students at
600 institutions annually
40+ consortium, state systems
SDSU Assessment Conference NSSE/CCSSE Workshop
NSSE’s Institutional Report
NSSE Overview
Institutional data
Respondent characteristics
Means and frequency
comparisons by selected
peers, Carnegie, national
Using NSSE results
Accreditation Toolkits,
Facilitators Guide
PowerPoint
presentation
SDSU Assessment
Conference NSSE/CCSSE Workshop
Benchmark Reports
Effective Educational
Practices
Level of Academic
Challenge
Active and Collaborative
Learning
Student-Faculty Interaction
Enriching Educational
Experiences
Supportive Campus
Environment
SDSU Assessment Conference NSSE/CCSSE Workshop
What have we learned from NSSE so far?
The single best predictor of student satisfaction with
college is the degree to which they perceive the college
environment to be supportive of their academic and
social needs.
Effective educational practices measured by NSSE are
independent of institutional selectivity.
Schools that have a lower student-faculty ratio, more
full-time faculty, and more classes with fewer than 20
students generally score higher on all five NSSE
benchmarks.
Grades, persistence, student satisfaction, and
engagement go hand in hand.
SDSU Assessment Conference NSSE/CCSSE Workshop
Grades, persistence, and engagement go
hand in hand
Second Semester GPA by Engagement
4.0
3.5
Very Low
3.0
Low
Average
2.5
High
Very High
2.0
1.5
1.0
Level of
Academic
Challenge
Academic and
Student
Collaborative Interaction with
Learning
Faculty
Enriching
Supportive
Educational
Campus
SDSU
Assessment Conference
NSSE/CCSSE Workshop
Experiences
Environment
Benchmark Scores for All Students by Undergraduate
Enrollment
Benchmark Scores for All Students by Undergraduate Enrollment Intervals
Level of
Academic
Challenge
65
Active and
Collaborative
Learning
55
Student
Interactions
with Faculty
Members
Enriching
Educational
Experiences
45
35
25
01000
1001 1500
1501 2000
2001 2500
2501 3000
3001 4000
4001 - 5001 5000 7500
Enrollment Intervals
7501 - 10001 - 15001 - 20001 - 25001 10000 15000 20000 25000 highest
Supportive
Campus
Environment
SDSU Assessment Conference NSSE/CCSSE Workshop
Academic Challenge at Two Public
Universities
Student engagement varies more within than between
SDSU Assessment Conference NSSE/CCSSE Workshop
institutions.
Relationship Between
NSSE & Graduation Rates
Benchmarks of Effective
Educational Practice
First-year
students*
Seniors*
Academic Challenge
.60
.46
Active & Collaborative Learning
.23
.09
Student Faculty Interaction
.28
.37
Enriching Educational Experiences
.53
.48
Supportive Campus Environment
.38
.26
SDSU Assessment Conference NSSE/CCSSE Workshop
*All correlations are significant at p<.01
NSSE Promising Findings
More than 75% of “A” students say they are highly
motivated to succeed compared with only half of the “C”
students.
At institutions where faculty members use effective
educational practices more frequently in their classes,
students are more engaged over all and gain more from
college.
A majority of students (54% of first-year students and
63% of seniors) says they often discuss ideas from
readings or classes with others outside of class, and well
over 90% do this at least sometimes.
SDSU Assessment Conference NSSE/CCSSE Workshop
NSSE Results
Are diagnostic; to help institutions look
holistically at undergraduate experience
Help pinpoint aspects not in line with mission, or
what institution expects
Identify weaknesses and strengths in
educational program
Help institutions know what to focus on to
improve student learning and success
SDSU Assessment Conference NSSE/CCSSE Workshop
Questions to answer with NSSE results
How many hours per week do first-year
students spend studying? Do women study
more than men?
What % of seniors work with faculty members
on activities other than coursework
(activities, committees)? Does this differ by
major?
What % of FY and SR spend 0 hours in cocurricular involvements? Is this more than at
peer institutions?
Do FY students work more frequently with
classmates on assignments outside of class
than their counterparts at peer institutions?
SDSU Assessment Conference NSSE/CCSSE Workshop
Questions to answer with NSSE results cont’d
Do NSSE results match our mission and what
we say about a [INSTITUTION] experience?
Are we meeting our own expectations for
having a supportive campus environment?
Since implementing a new multicultural
education initiative and expanding diversity
programming, has our score on the diversity
scale changed?
Are FY who withdraw from the institution
different in terms of engagement than students
who are retained?
How are we performing compared to our select
peers (normative benchmarking) or to our
institutionally identified standards (criterion
benchmarking)?
SDSU Assessment Conference NSSE/CCSSE Workshop
NSSE suite of instruments
Beginning College National Survey
Survey of Student
of Student
Engagement
Engagement
Annual survey of
first-year students at
baccalaureate
degree-granting
institutions
2005-2006 pilot
administration at 70
institutions
Administered prior
to start of classes,
usually at orientation
or welcome week
Annual survey of
first-year and
senior students
2006
administration at
571 institutions
Administered
during the spring
semester
Faculty Survey
of Student
Engagement
Parallel survey
to measure
faculty
expectations for
student
engagement
2006
administration at
131 institutions
Administered
during the spring
semester
SDSU Assessment Conference NSSE/CCSSE Workshop
Faculty Survey of Student Engagement
(FSSE)
Parallels NSSE
content
Fosters campus-wide
discussions of
teaching and learning
To date more than
53,000 faculty
members at over 300
four-year institutions
SDSU Assessment Conference NSSE/CCSSE Workshop
New!
Beginning College
Survey of Student Engagement – BCSSE
 Designed for entering
first-year students as a
companion to NSSE
 Measures:
• pre-college academic
and co-curricular
experiences
• expectations for
educationally
purposeful activities
during college
SDSU Assessment Conference NSSE/CCSSE Workshop
BCSSE Instrument
Three pilots: ‘04, ‘05, and
‘06
 2004 pilot with 28 schools,
15,890 students
 2005-2006 pilots with 80
institutions, 39,986 students
Study effect of students’
background on NSSE
scores
Use to examine gap
between expectations
and engagement
Registration now open
SDSU Assessment Conference NSSE/CCSSE Workshop
NSSE Accreditation Toolkit
SDSU Assessment Conference NSSE/CCSSE Workshop
NSSE Accreditation Toolkit
SDSU Assessment Conference NSSE/CCSSE Workshop
Connecting NSSE Data to Accreditation Standards Example
Accreditation standard: “Demonstrate effectiveness of student
academic and social support services”
Evidence for institutional self study:
Information about availability and student use of
tutoring, writing support, peer study groups,
counseling services
NSSE indicates FY & SR believe institution
emphasizes spending time studying and support for
student success; 79% seniors tutored or taught peers;
positive correlation between peer collaboration
outside of class, satisfaction and first-year retention
Positive student satisfaction data about support
services
Area for improvement - seniors indicate low gains in
writing and completing drafts of papers; institution
responds with examination of writing requirement in
senior capstone and targets seniors for increased use
SDSU Assessment Conference NSSE/CCSSE Workshop
of writing center
Institutional Example: NSSE and Enrollment
Management
The enrollment management area at Meredith has
used NSSE results to help guide the enrollment
marketing strategies. They look closely at trends
and make adjustments to programs and campus
visitation days to ensure that students are more
cognizant of student involvement and engagement
opportunities.
An academic dean reports using NSSE information
when speaking to parents at an admissions event.
"Parents seemed impressed that there was data to
support the points that I was making about what we say
about the student/faculty relationships and educational
opportunities at Meredith."
SDSU Assessment Conference NSSE/CCSSE Workshop
SDSU Assessment Conference NSSE/CCSSE Workshop
Institutional Example: Hanover
College
A detailed summary of NSSE is sent to the faculty as
well as the Admission and Student Life staffs to
ensure the results, both good and bad, are
understood by key folks on campus. Last year,
Admission requested an additional presentation and
discussion of findings to help them better understand
the strengths of the Hanover experience and how
that impacts student fit.
SDSU Assessment Conference NSSE/CCSSE Workshop
Institutional Example: NSSE & Assessing General
Education goals
Used NSSE items in 11a-p to assess
institutional impact on college-level
competencies (a.k.a., indirect measures of
student learning outcomes)
Undergraduate seniors 2005 NSSE results
confirmed findings from 2004
Most seniors (75%+) reported that KSU
experience had “substantial impact” (VM+QAB)
in 9 or 16 college-level competencies
KSU rank ordered competencies, showing
connection to mission, and compared to other
master’s instit where KSU was sig. higher,
comparable, sig. lower on competencies
SDSU Assessment Conference NSSE/CCSSE Workshop
Institutional Example: Program Development and
Strategic Planning
NSSE results framed a “Sophomore
Experience”
2005 = Pace’s 5th year of participation
Concern regarding SP- JR persistence; FY results
offers context for understanding exp. as students
enter SP year
Established “SP Experience Working Group” to
investigate if FY exp. carried over in SP year.
Focused on low NSSE score items, conducted
focus groups, created sophomore survey. Led to
pilot of “Pace Plan” (mentoring), includes Career
Exploration Course, Sophomore Kick-Off Day
NSSE also used in strategic indicators, Accred,
NCATE, AACSB, Faculty Development/Colloquia,
items used by offices (Technology, Multicultural
Affairs), studies performed by Enrollment Mngmt.
SDSU Assessment Conference NSSE/CCSSE Workshop
CCSSE History
In 2001, received 3-year grant funding from Lumina
Foundation and Pew Charitable Trusts to design, pilot and
administer a community college survey of student
engagement
Pilot administration in the fall of 2001 - 12 institutions
participated
National Field Test in the spring of 2002 - 48 institutions
participated
Annual, national, administrations in the spring of each
year:
2003 - 93 institutions
2004 - 152 institutions
2005 - 257 institutions
2006-193 institutions
2007-279 institutions
Self-funding project beginning September 2004
SDSU Assessment Conference NSSE/CCSSE Workshop
CCSSE’s Survey Administration
Stratified random sample of credit
courses
Stratification – morning, afternoon, evening
In-class, paper and pencil administration
Average administration time = 35 minutes
Completed surveys returned to CCSSE
for scanning and analyses
SDSU Assessment Conference NSSE/CCSSE Workshop
Community College Faculty Survey of
Student Engagement (CCFSSE)
Pilot test in fall of 2004
First national administration in spring of 2005
Over 20,000 participants
SDSU Assessment Conference NSSE/CCSSE Workshop
CCSSE Institutional Report
CCSSE Overview
College Results: A Synopsis
Means Summary
Frequency Distributions
Benchmark Reports
Using CCSSE Results
CCFSSE
Codebook
Appendix
SDSU Assessment Conference NSSE/CCSSE Workshop
CCSSE Annual Report
Review of CCSSE
Benchmarks
Active and Collaborative
Learning
Student Effort
Academic Challenge
Student-Faculty
Interaction
Support for Learners
Focus on trends and
issues for community
and technical colleges
SDSU Assessment Conference NSSE/CCSSE Workshop
Overview of 2005 National
Survey Results
More than 133,000 community college students
from 257 community and technical colleges in 38
states responded to the 2005 CCSSE survey.
Since 2002, more than 320,000 students in 404
colleges across 43 states have responded to
CCSSE.
Consortia for 2005 include statewide participation
in Indiana, Louisiana, North Dakota, Virginia, and
West Virginia. Other state-based consortia include
groups of colleges in Georgia, Illinois, Minnesota,
North Carolina, and Northeast Minnesota.
SDSU Assessment Conference NSSE/CCSSE Workshop
Institutional Use: Example
Estrella Mountain (AZ)
Received CCSSE results in the summer of 2004
Implemented an intensive improvement program that included student
engagement in the college-wide assessment process
Student engagement is now an Estrella Mountain Core Indicator of
Effectiveness for Student Success
Conducted three improvement sessions with students, faculty and staff;
discussed the implications of CCSSE at the annual strategic planning
retreat, and identified five improvement strategies for increasing student
engagement.
Developing a freshmen institute (with learning communities),
Improving publicity of workshops and services available to students (through the use
of electronic kiosks),
Developing student life activities to increase contact with faculty in non-academic
settings,
Creating facilities to improve adjunct faculty interaction with students, and;
Integrating student support services into the course curriculum
René G. Willekens, Dean of Planning, Research and Effectiveness
SDSU Assessment Conference NSSE/CCSSE Workshop
Institutional Use: Example
Northwest Vista College (TX)
CCSSE results are used in the Quality Enhancement Plan (QEP)
at Northwest Vista College (NVC).
NVC’s QEP concentrates on assessing student learning
outcomes for the core curriculum by serving as a focused area
of NVC’s Attitude, Skills, and Knowledge (ASK) model.
CCSSE supports both NVC’s QEP and ASK by providing
information about NVC’s student learning outcomes, including
critical thinking, cooperative learning, and student collaboration;
CCSSE survey items directly and indirectly relate to many of the
12 student learning outcomes identified as important by NVC
faculty.
Christa Emig, Math Instructor
SDSU Assessment Conference NSSE/CCSSE Workshop
Institutional Use: Example
North Hennepin Community College (MN)
We designed an exercise on the web site in which faculty are asked
to guess how students answered CCSSE questions. After
guessing, they see the real answer. Instead of doing this on paper,
we used personal response system "clickers" and got immediate
feedback after guessing at each question. The data could be
displayed in several ways for discussion. We had staff, faculty,
and administration very engaged!
We used questions from each of the benchmark groupings, and
after getting everyone interested in the data, we divided them
randomly into benchmark groups and asked them to identify 2
priorities for change at the college in that area. These are now
worked into our assessment plan initiatives.
Mary Diedrich, Active Learning and Assessment Coordinator
SDSU Assessment Conference NSSE/CCSSE Workshop
Questions?