Transcript Slide 1

Indiana University
Center for Postsecondary Research:
Assessing Student Engagement
& Educational Effectiveness
The College Student
Experiences Assessment
Program
FYE Annual Conference
February 17-20, 2007
Agenda
 The IU Center for Postsecondary Research (CPR)
 The College Student Experiences
Questionnaire Assessment Program (CSEQ CSXQ)
 National Survey of Student Engagement (NSSE,
BCSSE)
 How Can CPR Help Your Assessment Efforts
What is the IU Center for
Postsecondary Research?
 Designated Research Center of Indiana University
& the IU School of Education
 CSEQ Assessment Program (CSEQ & CSXQ)
 National Survey of Student Engagement (NSSE)
 Beginning College Survey of Student Engagement (BCSSE)
 Faculty Survey of Student Engagement (FSSE)
 Law School Survey of Student Engagement (LSSSE)
 Also home to the Indiana Project for Academic Success (IPAS) and
various other faculty-sponsored and grant-funded projects
What Does the Center for
Postsecondary Research do?
 Promote student success and institutional
effectiveness by helping postsecondary institutions
and related agencies gather and use data for
decision making and institutional improvement
 Variety of survey tools
 Institutional data reports and custom analyses
 Campus consultations (merging the quantitative and qualitative)
 Foster national dialogue and stimulate further research on the
undergraduate experience, assessment, educational effectiveness, and
institutional improvement
The College Student
Experiences Questionnaire
Assessment Program
The CSEQ Assessment Program
 CSEQ: The College Student  CSXQ: College Student
Experiences Questionnaire
Expectations Questionnaire

 Measures student progress and the
 Gauges how students expect to
quality of students’ experiences
participate in educational activities,
inside and outside the classroom.
experience the campus
 Administered to undergraduate
students at any point following first
semester of enrollment.
environment, and realize specific
learning outcomes.
 Administered to first-year students
before or immediately following start
of classes
CSEQ Survey Instrument
 Developed in 1979 by Dr. C. Robert Pace at UCLA
 Now in 4th edition (5th edition in development)
 Over 180,000 4th edition cases
 More than 500 institutions
 Divided into four sections
 Background Characteristics
 College Activities Scales
 Measures of the College Environment
 Estimates of Gains
Activities Scales
 Reflect quality of effort students expend in
activities related to:
 Library Experiences
 Clubs and Organizations
 Computer and Information Technology  Personal Experiences
 Course Learning
 Student Acquaintances
 Writing Experiences
 Scientific and Quantitative
Experiences
 Experiences with Faculty
 Art, Music, Theater
 Campus Facilities
 Topics of Conversation
 Information in
Conversations
The College Environment
 Ten scales measuring student perceptions of
the campus environment:
 The extent to which the campus
emphasizes diverse aspects of
student learning and personal
development (7 items)
 Relationships with faculty,
administrators, and other
students (3 items)
Estimate of Gains
 Consist of student ratings of progress toward
important educational goals
 Goals are presented in five major clusters:
 General Education, Literature,
Arts, and Social Sciences
 Personal Development and
Social Competence
 Science and Technology
 Intellectual Skills
 Vocational Competence
CSXQ Survey Instrument
 Developed from the CSEQ by Dr. C Robert Pace
and Dr. George Kuh in 1996 with FIPSE support
 Revised in 1998 (2nd edition)
 Over 61,000 students at
more than 60 institutions
 Divided into three sections
 Background Characteristics
 College Activities Scales
 Measures of the College Environment
Activities Scales
 Quality of effort students expend on activities:
 Library and Information Technology
 Experiences with Faculty
 Course Learning
 Writing
 Campus Facilities
 Clubs, Organizations, Service
 Student Acquaintances
 Scientific and Quantitative Experiences
 Topics of Conversation
 Information in Conversations
The College Environment
 Ten scales measuring perceptions of the campus
environment with regard to:


The extent to which the campus
emphasizes diverse aspects of
student learning and personal
development (7 items)
Relationships with faculty,
administrators, and other
students (3 items)
Why Do Expectations Matter?
 Expectations affect the actual experiences of
college students by:
1) Serving as a filter through which students evaluate and select
various types of educational opportunities.
2) Predisposing students to seek specific learning activities.
 Student expectations and subsequent experience
are not always congruent.
 Ex: First-generation or low-income students may have less
tacit knowledge of what college is like.
 May influence students’ overall performance in college and
satisfaction with their experience.
Why measure expectations?
National focus
on setting
high expectations
Social-psychological
theories
Differences between
student and
institutional
expectations
CSEQ/CSXQ Administration
 Paper or online version
 CSEQ takes 20-30 minutes
 CSXQ takes 10-15 minutes
 Institution receives descriptive statistics,
codebook, and SPSS files
 Normative comparison data available for each
 Additional services: Student Advising Reports,
custom statistical analyses
Student Advising Report
 Displays student CSEQ or CSXQ responses
alongside average responses of peers
 Provides direct and personalized feedback
 Deepens the student/advisor relationship
 Allows early intervention with at-risk students
 Increases the value of assessments to students
 SAR can easily be adapted to the needs of your
school or program
The National Survey of
Student Engagement
What is NSSE?
 Annual snapshot of student participation in
programs and activities that institutions provide
for their learning and personal development.
 The results will provide an estimate of how undergraduates
spend their time and what they gain from attending college.
 National Survey of Student Engagement items represent
empirically confirmed ‘good practices’; they reflect behaviors
associated with desired outcomes of college.
 The time and energy students devote to educationally
purposeful activities is the single best predictor of their
learning and personal development.
 Survey based on the CSEQ, developed by C. Robert Pace
Two Components of
Student Engagement
What students do -- time and
energy devoted to
educationally purposeful
activities
What institutions do -- using
effective educational practices
to induce students to do the
right things
History of NSSE
600
 Began in 1998 with aid from
Pew Charitable Trust
 Survey is conceptually based on
the CSEQ, developed by Robert
Pace
Number of Schools
 Alternative way to look at
educational quality and foster
institutional improvement
500
400
300
200
100
0
1999
2000
2001
2002
2003
2004
Year
 NSSE 2006 Administration
 More than one million first-year and senior students were invited to
participate in the 2006 NSSE administration.
 557 institutions in the United States and Canada
 331,601 students responded.
2005
2006
NSSE Instrument
 Research based on
effective educational
practices
 Designed and tested for
validity and reliability
 Relatively stable over time
 High credibility of selfreported data
 Over 200,000 students at
500 institutions annually
NSSE Administration
 Administration Mode
 Paper: We need accurate
mailing addresses,
letterhead, signatures
 Web+: 4x the paper
sample, we need e-mail
and mailing addresses
 Web: 5x the paper
sample, we need e-mail
addresses
NSSE Administration
 Sample Size
 Numbers are based on mode and school size
 Oversampling can increase sample size or ensure adequate
representation of populations of interest
NSSE Administration
 Special Groups
 Consortia
 Schools like you
 Additional Questions
 Selected Peers
 Schools like you
 Special Populations
 Identified in Pop file
 Special Analysis
 Post Hoc Analysis
 Experimental Items
 Web schools only
















2006 Consortia
American Democracy Project
Associated New American Colleges
AAUDE
Association of Independent Colleges of Art and
Design
Association of Independent Technical
Universities
Bringing Theory to Practice
Catholic Colleges and Universities
Colleges that Change Lives
Council for Christian Colleges & Universities
Council of Public Liberal Arts Colleges
Jesuit Colleges and Universities
Private Liberal Arts Colleges and Universities
Teagle Grant Consortium
Urban Universities
Women's Colleges
State and University Systems





Ontario Universities
South Dakota Public Universities
Tennessee Publics
University of Texas
University of Wisconsin
NSSE Deliverables
 Institutional Report (August)
 Institutional data
 Respondent characteristics
(Demographic Information)
 Means and Frequencies (item
averages and response
percentages)
 Benchmarks of Effective
Educational Practice
 Additional Data (If Applicable)
 Experimental items
 FSSE Report
 BCSSE Combined Report
 NSSE Institute Information
 Using NSSE Data
 Accreditation Toolkit
 Data Facilitator’s Guide
Benchmark Report (August)
Level of
Academic
Challenge
Student
Faculty
Interaction
Enriching
Educational
Experiences
Active &
Collaborative
Learning
Supportive
Campus
Environment
NSSE Institute
 NSSE Institute is the Service Arm of NSSE
 The Institute draws upon NSSE data and related research to help
faculty, administrators, and governing board members discover
and implement effective mechanisms for linking information about
student experiences to efforts to improve academic programs and
support services.








Campus Audits
Consultations
Workshops
Ongoing Research and Evaluation
Accreditation Toolkits
Regional Workshops
Pocket Guides
Facilitator’s Guide to Using NSSE Data
 Additional Research Projects
 Building Engagement and Attainment in Minority Students (BEAMS)
 Documenting Effective Educational Practices (DEEP)
What is BCSSE?
 The Beginning College Survey of Student
Engagement (BCSSE, pronounced “bessie”)
 Measures entering first-year students’ pre-college
academic and co-curricular experiences, as well as
their interest in and expectations for participating
in educationally purposeful activities during
college.
 HS experiences; Importance of college activities
 Baseline self-assessment of knowledge and skills
 Demographics
BCSSE Instrument
 Two pilots: 2005 and 2006
 2004-2005 pilot with 28
schools, 15,890 students
 2005-2006 pilot with 70
institutions, 39,986 students
 Use to examine gap between
expectations and
engagement
 Study effect of students’
background on NSSE scores
 Designed as a companion to
the National Survey of
Student Engagement (NSSE).
BCSSE Registration
 BCSSE 2007-2008
Registration Open!
 BCSSE measures entering firstyear students’ pre-college
academic and co-curricular
experiences, as well as their
interest in and expectations for
participating in educationally
purposeful activities during
college.
 To learn more about BCSSE, go
to www.bcsse.iub.edu
Choosing the CPR Tool That’s
Right for Your Campus
Only one step in assessment
process
Step #1: Survey Data
Step #4: Follow-up
• Use results as
benchmarks to
monitor progress
• Faculty & student
focus groups
• Survey students
• Review results
• Develop preliminary
list of strengths and
opportunities for
improvement
Step #3: Action Plan
• Finalize plan
• Share plan with
appropriate groups
• Link to strategic plan
• Implement action
Step #2: Feedback
• Share results with
faculty,
administrators &
students
• Identify themes &
priorities
• Design action plan
CPR and Your Assessment Plan
 Which CPR survey should I use?
 CSEQ offers maximum flexibility in terms of administration
timeline; student population surveyed; question
customization; cost-effective
 NSSE provides additional national, sector, and other
comparative information; standard survey population
sampling protocol and structured timeline; comprehensive
institutional results report
 CSXQ focuses on students expectations for their college
experience
 BCSSE focuses on high school academic and co-curricular
engagement, with select student expectation questions
CPR and Your Assessment Plan
 How often should I administer a CPR survey?
 Every Year: Gives you a snapshot of each class
 Every Three Years: Gives you a picture of a cohort at the
beginning and the end of their college experiences
 Every Five Years: Works well with most accreditation cycles
(Accreditation and Interim Reports)
 Other factors to consider
 Establishing a baseline
 Costs
 Additional Surveys/Sources of Data
 Time to take absorb results, make changes
CPR and Your Assessment Plan
 How can I use the survey results?
 Program Assessment
 Performance Indicators
 Institutional Research
 Student Satisfaction
 Accountability
 Recruitment
 Accreditation Studies
 Retention Efforts
 Curricular reform
CPR and Your Assessment Plan
 How else can CPR help our efforts?
 Consulting
 Help with making sense of data
 Presentation assistance
 Technical questions
 Result reports and research resources
 NSSE- Regional users workshops
Regional NSSE Users
Wichita Drive-in Workshop
Co-sponsored by Wichita State University, Friends University, and Newman
University
When: April 19-20, 2007
Where: Metropolitan Complex
Wichita State Campus
 Please join us for the spring Drive-in Workshop to learn how colleges
and universities are using NSSE as a foundation for institutional
improvement.
 We invite session proposals from NSSE users and encourage you to
bring a team to work on an action plan for your campus.
 Full workshop details will be available in January 2007 on the NSSE
Institute Web site: www.nsse.iub.edu/institute/.
Discussion and Questions
Julie Williams
Research Analyst and CSEQ
Project Manager
Michelle Salinas Holmes
CSEQ Research Associate
Indiana University Center for
Postsecondary Research
1900 East 10th Street
Eigenmann Hall, Suite 419
Bloomington, IN 47406-7512
[email protected]
[email protected]
 Web sites:
 www.cseq.iub.edu
 www.nsse.iub.edu
 E-mail:
 [email protected]
 [email protected]
Ph: 812-856-5825
Fax: 812-856-5150