ICAI_2014_Mihran_Aroian - Center for Academic Integrity
Download
Report
Transcript ICAI_2014_Mihran_Aroian - Center for Academic Integrity
The Whistleblower Effect:
A Quantitative Analysis Demonstrating that Reporting Students for
Academic Dishonesty Negatively Impacts Faculty Evaluations
Mihran Aroian
Presentation Outline
-
Student Judicial Services Overview
Faculty-in-Residence program
Faculty Perceptions
Why Conduct this Research
Results
Recommendations
Questions
SJS Overview
Student Judicial Services
As a part of The University of Texas at Austin’s Office of
the Dean of Students, Student Judicial Services (SJS)
fosters moral development on campus through the
resolution of academic integrity and conduct-related
matters as well as educational outreach to students,
faculty, and staff.
In addition to our educational component, SJS
investigates alleged violations of the Institutional Rules
(both academic and non-academic) and implement the
disciplinary process with a focus on student learning and
development.
Primary SJS Roles
• Outreach/Education
• Faculty Consultations
• Case Management
•
1,600-1,800 annual referrals (AI and conduct)
• Institutional Rule Revision
Some Statistics
3,300 faculty
2,500 TA’s
50,000 student body
140 academic departments
2011-2013 academic years
320 faculty/TA’s reported cases to SJS
1,155 AI cases over 2 years
50% of all submissions came from (can you
guess how many faculty)?
15 out of 320
SJS Data
Administrative disposition versus
Faculty Disposition – 55/45
Top 3 AI violations
Cheating, Plagiarism, Collusion
Classification
Freshman (10); Sophomore (20%); Junior (20%);
Senior (35%); Grad/Professional (15%)
GPA – even distribution
Gender (M/F) – 65/35
Establishing a
Faculty-inResidence Program
at UT Austin
Academic Integrity
As Steve Covey says “seek first to
understand, then to be understood”
Improving AI is simple
“just as plain as the nose on your face”
For example:
What is the shape and color of a Stop Sign?
What is the shape and color of a Yield Sign?
Benefits to SJS
Dedicated outreach time not available to typical
conduct staff
Cultivation of relationships outside of “crisis”
events typical in conduct offices
Increased exposure through meetings with
stakeholders previously unidentified by SJS
Ally outside of the office to speak with unaffiliated
individuals
Addition of a “faculty perspective” to conversations
within the conduct office
Discussions at the Macro-level (student conduct
as a field) rather than at the Micro-level (a
student’s conduct)
Benefits for Me!
Greater appreciation for SJS
Understanding the complexity of student conduct
Part of the team
Ability to cross boundaries
Support SJS with other stakeholders
Educate campus stakeholders regarding SJS
Involvement with student groups
Being part of the solution
Satisfying my own needs as an educator
Faculty Perceptions
Lack of understanding by faculty as to
what constitutes cheating.
Not all faculty engaged in high AI
standards.
Faculty unaware of common forms of
cheating.
Faculty unwilling to expend effort to
meet with students and file paperwork.
Faculty avoid student conflict/tension
AI is an afterthought in the class
Faculty Perceptions
Lack of consistent message to
students.
High level of indifference by many
campus stakeholders.
Some faculty do not care because they
are not rewarded for teaching.
Faculty need assistance on identifying
cheating and how to deal with students
who do cheat.
The Whistleblower Effect: A Quantitative
Analysis Demonstrating that Reporting
Students for Academic Dishonesty
Negatively Impacts Faculty Evaluations
Authors: Mihran Aroian
Raymond Brown
Scenario A
Three students submit essentially the
same writing assignment.
Student A provided paper to student B & C
Student B & C take responsibility
Student A believes he is innocent
Students upset at instructor
Students are found in violation
Students are members of the same student
organization as are other students in the class
Scenario B
A group presentation of six students
includes significant plagiarized material
Initially, nobody takes responsibility
One-on-one conversations
Students start throwing each other under the
bus
Case sent to SJS for investigation
Investigation completed the following
semester
Two of the six found in violation
Question
Will the end-of-semester evaluations
for these instructors increase,
decrease, or not be affected by this
situation?
Scenario A?
Scenario B?
Why This Study?
Interviewing stakeholders on campus
as the faculty-in-residence
Faculty were hesitant to report students
Faculty want to avoid conflict/confrontations
Belief that reporting students would impede
professional advancement
As someone who always reports, I was
shocked!
Original hypothesis
Perception by faculty was a false perception and
analysis would demonstrate no correlation
Introduction
Examine the impact of sanctioning
students for acts of academic
misconduct on student evaluations of
teaching (SETs).
use of student evaluations for faculty
advancement and promotions
faculty may be conflicted between enforcing
academic integrity and maximizing their
professional advancement with high SET
scores.
Method
A multi-level modeling design with
8,940 end-of-semester student
evaluations
32 faculty members from 17 academic
departments was employed to
determine if there was a whistleblower
effect.
Literature Search
Extensive qualitative articles
Lack of quantitative analysis
Only quantitative studies focused on
grade inflation and higher SETs
GPAs have been increasing while SAT
scores have been decreasing
Genesis of Research Design
Initial evaluation - faculty against peer
instructors
Compare faculty against all faculty
within department
Compare faculty against all faculty
within college
Final analysis – compare faculty
against themselves
Creation of Sample
Compare faculty against themselves
Analyze evaluations for course in a
semester when they reported cheating
Analyze evaluations for same course in
a semester when they did not report
cheating
Evaluation must be for the same class
Looked for the closest semester in
temporal proximity - summers excluded
Must have reported 3 or more cases
Challenges to Methodology
Highest reporting instructors reported
cheating every semester therefore were
not included
Cases had to be submitted to SJS prior
to 10 days before the end of semester
Course Instructor Survey’s are given up to 10
days before the end of the semester
The Course Instructor Survey
Five point scale ranging from “strongly
disagree” to “strongly agree”
Questions asked on survey
The course was well organized
The instructor communicated information
effectively
The instructor showed interest in the progress of
the student
The tests and assignments were usually graded
and returned promptly
The instructor made me feel free to ask
questions, disagree, and express my ideas
Survey
Overall rating for the instructor and
course ranging from “very
unsatisfactory” to “excellent” on a 5point scale
Overall, this instructor was
Overall, this course was
Key criteria for performance evaluation
is the overall instructor rating
Limitations of Analysis
Results based on data collected at one
university
Unable to link evaluations to students
Unable to test highest reporting faculty
Results
Statistically significant effect for
reporting students (p<.001)
Estimated mean rating for classes in
which students were not reported was
3.95
Estimated mean rating for classes in
which students were reported was 3.75
Hypothesis - instructors who report
students tend to be more demanding of
their students in general than those
who do not report
Discussion
Is it time for a policy review?
Instructors expected to enforce AI
Colleges want to instill ethics – beyond college
By not enforcing, instructors are implicitly
condoning cheating and undermining accurate
student assessment
Evaluations play a critical role for
advancement for both tenure-track and
non tenure-track faculty
Do you harm yourself when you enforce AI?
Student Responsibility
UT participates in the National
Assessment of Student Conduct
Adjudication Process (NASCAP)
Project
21 higher education institutions participate
Upon completion of adjudication, students
complete survey
When asked the question “What was the outcome
of your case” 6.9% of UT students answered “my
case was dismissed”
In reality, only 0.15% of the cases were
dismissed
Recommendations
Encourage all faculty to communicate
expectations of AI
Provide regular communication to
faculty regarding:
How students cheat
How to deal with cheaters
How to confront students
Best practices
How to reduce cheating
Require students to complete an online
AI tutorial each year
Recommendations
Encourage faculty to increase
frequency of message
Each academic department should
have an AI point person
Faculty should regularly update
assignments and exams
Recommendations
Include a question about faculty AI
expectations and practices on evals
AI should be part of new faculty
orientation
Administrators should include
consideration for faculty who report
cheating regarding merit review and
promotions