Transcript Using Aggregate IDEA Data Across the College: Assessing

Assessment Overview
Drake CPHS
Overview
•
•
•
•
Overview of IDEA Data
Assessing college-wide teaching goal
Advising Results
Q&A
Drake Background
• IDEA System
– Course evaluation system: Heavily researched
– Used by ~ 7 SOP/COP; 320 institutions
– Measures progress against faculty objectives
• IDEA at Drake
– Since 2004
– Paper and on-line (35-45 courses/semester)
IDEA Background
• Students' feedback on their own learning
progress, effort, and motivation,
• Student perceptions of the instructor's use
of teaching methods and strategies.
• Rather than teaching style or personality, IDEA
focuses on student learning and the methods
used to facilitate it.
Progress on Relevant Objectives vs. IDEA
Converted Scores
2011-12
2010-11
2009-10
2008-09
2007-08
National
Database
Much Higher
(10% of courses)
8.2%
5.5%
1.6%
1.4%
1.1%
Higher
(20% of courses)
34.3%
34.2%
26.6%
18.3%
24.2%
Similar
(40% of courses)
43.8%
42.5%
43.8%
57.8%
50.6%
Lower
(20% of courses)
5.5%
12.3%
15.6%
15.5%
15.4%
Much Lower
(10% of courses)
8.2%
5.5%
12.5%
7.0%
8.8%
86.3% ✓
82.2% ✓
72%
77.5%
75.9%
Goal Progress
Actions
• What actions have been taken?
– Faculty Development & meetings on:
•
•
•
•
•
•
•
Soul searching on what I’m trying to achieve
How to choose objectives (right ones, right number)
Should Teaching methods be adjusted?
Linking content and methods to the objective
More use of IDEA resources (POD notes, etc.)
Tie action to faculty annual goals
Initiate FIF’s early in semester
Good discussions and culture of assessment
Primary Instructional Approach (11-12)
Instructor Related Course Requirements
(Some or Much required)
Percent of CPHS classes selecting objective as either
Essential or Important (FIF)
Student ratings of progress on objectives chosen as
Essential or Important
1=no progress
2=slight progress
3=moderate progress
4=substantial progress
5=exceptional progress
Amount and Difficulty of Course work: Student Ratings
AY 2007-2008
AY 2008-2009
AY 2009-2010
AY 2010-2011
AY 2011-2012
Average
% of classes
below 3.0
% of classes 4.0
or above
CPHS
2.8
3.0
3.0
3.1
3.0
52%
41%
46%
42%
46%
8%
16%
22%
23%
19%
IDEA
3.2
33%
15%
CPHS
3.4
3.3
3.2
3.3
3.2
34%
32%
39%
28%
42%
28%
20%
8%
13%
11%
IDEA
3.4
21%
18%
CPHS
3.4
3.4
3.3
3.4
3.3
36%
33%
35%
29%
35%
33%
31%
26%
28%
25%
IDEA
3.4
20%
18%
Amount of Reading
Amount of work in
other (non-reading)
assignments
Difficulty of subject
matter
Values are similar if within 0.3
1=Much less than most courses, 2=less than most, 3=about average, 4=more than most, 5=much more
2012
• Trends towards more on-line evaluations
• Average response rate was down to 62% (57%
online rate)
• Average Class size was 77
• Average number of objectives by faculty 3.7
Advising: Interactions with Advisor
over the last AY
Meetings
Emails
Phone
calls
Other
Total
HSCI (n=72)
3.1
6.9
0.3
0.1
10.4*
Pre-Pharm (n=159)
2.4
5.0
0.2
0.1
7.7*
Pharmacy (n=249)
1.6
3.2
0.1
1.0
5.9*
Total:
2.1
4.3
0.2
0.6
7.1
* Significantly different between all majors at 0.05
Advising: Lowest agreement
HSCI
Pre-Pharm
Pharmacy
Total
knows where to refer me for campus-wide
support services (writing center, supplemental
instruction, counseling center, disability resource
center, academic achievement office).
63%
70%
55%
61%
has discussed professional involvement with me.
51
84
69
71%
knows where to refer me for College support
services (Student Services, Admissions,
Professional and Career Development, etc.)
74
82
71
75%
has helped me assess my career goals.
69
84
70
75%
is knowledgeable about the campus general
education program (Drake Curriculum).
72
84
71
75%
Advising: Highest agreement
HSCI
Pre-Pharm
Pharmacy
Total
85%
96%
94%
93%
is available for appointments.
87
96
90
92%
encourages me to make my own decisions.
83
93
91
90%
is willing to spend sufficient time with me to deal
with my questions and concerns.
82
95
88
89%
knows where to refer me for information on
college policies and programs.
81
92
83
86%
treats me as an individual.
General Advising Effectiveness
HSCI
Pre-Pharm
Pharmacy
Total
I am pleased with the overall nature of my
meetings with my faculty advisor.
69%
88%
80%
81%
Overall, I have a good faculty advisor.
69%
90%
82%
83%
Future/ongoing areas
• Identifying embedded assessments
• Re-evaluate our educational outcomes?
• Populate new AAMS system (AACP’s
Assessment and Accreditation Management
System)
• Re-write (condense) our Assessment Plan
• HSCI Program Review
• PharmD Accreditation (2014-2015)
Questions/Discussion