Transcript National Survey of Student Engagement, 2008

National Survey of Student
Engagement, 2008
Results for UBC-Vancouver
NSSE Overview







Supported as an assessment tool by UBC many other Canadian
Universities including all of G13
Over 1200 colleges and universities across Canada and the U.S.
have participated in NSSE since the first pilot in 1999; ~450,000 first
and senior year respondents in 2008
Supported by strong research; NSSE provides an estimate of how
undergraduates spend their time and what they gain from attending
college
Survey items represent empirically confirmed "good practices" in
undergraduate education. That is, they reflect behaviors by students
and institutions that are associated with desired outcomes of
college.
125+ questions (core, contextual, experimental, consortium) and 5
composite engagement benchmarks
Results provided for participating university and selected
comparator groups
Additional info at http://nsse.iub.edu
Engagement Benchmarks
NSSE provides five benchmarks of effective educational practices:





Level of Academic Challenge (LAC): amount of time studying, reading,
writing; academic effort; coursework emphasis on analysis, synthesis,
applying theories
Active and Collaborative Learning (ACL): asking questions, class
presentations, teamwork in class, discussions and learning activities outside
of class and involvement in community based projects
Student Faculty Interaction (SFI): discussions with faculty on grades,
coursework, careers, research involvement with faculty, worked on other
projects or committee with faculty
Enriching the Educational Experience (EEE): participation in cocurricular activities, volunteer work, field experience, co-op, community
service, serious conversations with students from different ethnic
backgrounds, political/religious beliefs, etc.
Supportive Campus Environment (SCE): student, faculty, staff
relationships; campus services to help students with both their academic
and non-academic responsibilities
Peer Groups

G13 Peer Group: Ontario and Quebec universities had their own
consortiums for participation in NSSE in 2008 so the G13 peer group
includes UBC, The University of Alberta, University of Calgary and
Dalhousie (eventually we will have comparisons for all G13 institutions)

“Carnegie” Cdn Peers consists of McGill, Concordia, McMaster, Toronto,
Alberta, Calgary, York (large undergraduate populations)

Selected Peers consists of participating large research intensive
universities in the U.S.: The University of Texas at Austin, University of
Illinois at Urbana-Champaign, University of Minnesota-Twin Cities,
University of North Carolina at Chapel Hill, University of Virginia, University
of Wisconsin-Madison
Interpreting the Data

Responses are reported in the form of means and frequencies. Items that
make up the five benchmarks of effective educational practices are
identified in the means report and are also aggregated in the benchmarks
report.

This year we report differences among peer groups AND differences for
UBC since 2006. Important differences are identified by:
a) statistical significance (is the change unlikely to be simple chance
variation?) and
b) effect sizes

Effect size indicates the “practical significance” of the mean difference. In
practice, an effect size of .2 is often considered small, .5 moderate, and .8
large.
Summary of Findings

UBC 2006 compared to UBC 2008: Areas of improvement and decline

UBC compared to Canadian Peers

UBC compared to U.S. Peers

Overall Satisfaction

Student Priorities on improving the learning environment

Next Steps
UBC-V NSSE First Year Benchmarks
60
50
40
30
2006
2008
20
10
0
Acad Challenge
Active
collaborative
learning
Stud/fac
interaction
Enriching/educ
experiences
All changes are statistically significant. All effect sizes are small.
Supportive
campus
environment
UBC-V NSSE Fourth Year
Benchmarks
60
50
40
30
20
10
0
2006
2008
Acad Challenge
Active
collaborative
learning
Stud/fac
interaction
All changes are statistically significant. All effect sizes are small.
Enriching/educ
experiences
Supportive
campus
environment
Areas of improvement, First Year, 2006 - 2008
Very small, statistically significant increases, especially in areas where we were
weak.















ACL, Worked with classmates outside of class
ACL, Tutored or taught other students
ACL, Participated in a community-based project as part of course
EEE, Had serious conversations with students of a different race or ethnicity
EEE, Community service or volunteer work
EEE, Encouraging contact among students from different economic, social, ethnic backgrounds
LAC, Worked harder to meet expectations
LAC, Making judgments about the value of information, arguments, or methods
SCE, Providing support to succeed academically
SCE, Helping you cope with your non-academic responsibilities
SFI, Talked about career plans with a faculty member or advisor
SFI, Discussed ideas from your readings or classes with faculty members outside of class
SFI, Received prompt written or oral feedback from faculty on your academic performance
SFI, Worked with faculty members on activities other than coursework (committees, orientation,
student life activities, etc.)
SFI, Work on a research project with a faculty member outside of course or program requirements
Areas of decline, First Year, 2006 - 2008
None of the decreases in benchmark items were
statistically significant.
(and very few items actually declined)
Areas of improvement, Fourth Year, 2006 - 2008
Very small, statistically significant increases, especially in areas where we were
weak.























ACL, Worked with other students on projects during class
ACL, Worked with classmates outside of class to prepare class assignments
ACL, Tutored or taught other students (paid or voluntary)
ACL, Participated in a community-based project as part of a regular course
EEE, Used an electronic medium to discuss or complete an assignment
EEE, Had serious conversations with students of a different race or ethnicity than your own
EEE, Practicum, internship, field experience, co-op experience, or clinical assignment
EEE, Participate in a learning community where groups of students take two or more classes together
EEE, Culminating senior experience (capstone course, senior project or thesis, comprehensive exam, etc.)
EEE, Encouraging contact among students from different economic, social, and racial or ethnic backgrounds
LAC, Worked harder to meet expectations
LAC, Making judgments about the value of information, arguments, or methods
LAC, Applying theories or concepts to practical problems or in new situations
LAC, Hours per 7-day week spent preparing for class
LAC, Spending significant amounts of time studying and on academic work
SCE, Relationships with faculty members
SCE, Providing the support you need to help you succeed academically
SCE, Helping you cope with your non-academic responsibilities (work, family, etc.)
SCE, Providing the support you need to thrive socially
SFI, Talked about career plans with a faculty member or advisor
SFI, Discussed ideas from your readings or classes with faculty members outside of class
SFI, Worked with faculty members on activities other than coursework (committees, orientation, etc.)
SFI, Work on a research project with a faculty member outside of course or program requirements
Areas of decline, Fourth Year, 2006 - 2008
None of the decreases in benchmark items were
statistically significant.
(and very few items actually declined)
UBC-V, Canadian Peers, U.S. Peers,
NSSE 2008, First Year
70
60
50
40
30
20
10
0
UBC
Cdn Peers
U.S. Peers
LAC
ACL
SFI
EEE
SCE
UBC-V, Canadian Peers, U.S. Peers,
NSSE 2008, Fourth Year
60
50
40
UBC
Cdn Peers
U.S. Peers
30
20
10
0
LAC ACL
SFI
EEE SCE
UBC compared to Canadian and U.S. Peers
Canadian Peers:

UBC’s benchmark scores are similar to those of other Canadian Peer Universities.

UBC does relatively better in the area of Enriching Educational Experiences (first and
fourth year) and Active and Collaborative Learning (first year)

UBC scores slightly lower in the area of Supportive Campus Environment (first year)
and Level of Academic Challenge (fourth year).
U.S. Peers:

UBC’s scores are significantly lower on all benchmarks compared to U.S. Peer
Institutions (first and fourth year, statistically significant, moderate effect sizes).

-
We can hypothesize several reasons for this:
Different demographics
Different Socio-Economic status of students
Lower funding, larger classes
Commuter campus
Overall Satisfaction

In addition to the questions which comprise the five educational benchmarks, NSSE
also asks questions which explore overall student satisfaction as well as other areas
related to both the academic and non-academic environment.
UBC
Carnegie
CDN Peers
Mean
Mean
FY
2.63
SR
Sig
Effect
Size
Mean
2.73
***
-0.12
2.45
2.52
**
FY
2.91
2.94
SR
2.87
2.92
FY
3.18
SR
3.02
Class
Overall, how would you evaluate the quality of
academic advising you have received at your
institution?
How would you evaluate your entire educational
experience at this institution?
If you could start over again, would you go to the
same institution you are now attending?
Selected
Peers
Sig
Effect
Size
2.98
***
-0.41
-0.08
2.78
***
-0.35
-0.04
3.35
***
-0.62
-0.06
3.36
***
-0.69
3.15
0.04
3.46
***
-0.38
2.99
0.03
3.43
***
-0.52
*
Student Priorities
NSSE allows for the inclusion of institutional specific questions for
consortium participants. UBC, together with Dalhousie, Alberta
and Calgary, provided customized questions which were asked of
all first and fourth year students.
Students were asked to identify two areas that UBC needs to address
in order to improve student learning in the classroom and outside
the classroom
Issues UBC needs to address IN the classroom (2 responses
per student)
Increasing opportunities to learn more about global issues
Changing the mix of lectures, seminars, tutorials and labs
Providing more current/relevant courses and curriculum
Improving student access to information technology
Improving the quality of labs
Reducing class sizes overall
Increasing the number or variety of course offerings OUTSIDE
YOUR MAJOR
Increasing the number or variety of course offerings IN YOUR
MAJOR
Ensuring a better fit between course content, assignments and
tests/exams
Improving the quality of teaching assistants
Improving the quality of course instruction by professors
Improving the quality of classrooms or lecture halls
4th Yr
1st Yr
Issues UBC needs to address OUTSIDE the classroom (2
responses per student)
Providing a better social environment for students
Increasing opportunities for international experiences
(e.g. exchanges, study abroad)
Improving the quality/availability of study spaces
Improving library services (e.g. circulation, staff
availability, internet/computer availability, etc.)
Improving the library collection
Providing students with more opportunities to undertake
research with faculty
Expanding and/or improving the quality of counseling
services
Expanding and/or improving the quality of academic
support services
Increasing contact with professors outside of class (e.g.
office hours)
4th Yr
1st Yr
Next Steps

Analyze student comments and compare to select peer institutions

Provide NSSE data to deans at faculty level

Drill down to specific majors for Arts, majors/honors for science,
and ILP programs (eg. CAP)

NSSE Deep learning scales and analysis by aboriginal, visible
minority, international

Use NSSE as the stimulus for discipline specific learning outcomes
(e.g.: what more do you need to know to guide learning in your
discipline?)

Focus NSSE discussion to align with institutional strategic goals
(Trek 2010)