Transcript Summary of CPHS Course Evaluations: AY 2007-08 and 2008-09 Chuck Phillips August 2009 IDEA Evaluations: Overview Summary Statistics AY 07-08 AY 08-09 Number of Classes Evaluated Average Class size 75% 76% 64%

Summary of CPHS Course
Evaluations:
AY 2007-08 and 2008-09
Chuck Phillips
August 2009
IDEA Evaluations: Overview
Summary Statistics
AY 07-08
AY 08-09
Number of Classes Evaluated
99
81
Average Class size
75
80
75%
76%
64% (s.d. 21%)
65% (s.d. 19%)
Overall Response Rate
On-line Response Rate
Mean (s.d.)
Range
0-100%
33-100%
Old 192 and small classes
In-class Response Rate
Mean (s.d.)
91% (s.d. 8%)
93% (s.d. 6%)
Range
69-100%
76-100%
Average # of Objectives: E or I
4.3
4.3
Response Rate Issues
• Low response rates on-line were for either
the old therapeutics (192 with several
instructors) or classes with very small n (a
few electives).
• Expect average response rate to increase
in 09-10 for on-line evals
Class format
Increase in lecture probably due
to fewer individual lab sections
being evaluated.
Amount and Difficulty of Course work: Student Ratings
AY 2007-2008
AY 2008-2009
Amount of
Reading
Amount of work in
other (nonreading)
assignments
Difficulty of subject
matter
Average
% of classes
below 3.0
% of classes
4.0 or above
CPHS
2.8
3.0✓
52%
41%
8%
16%
IDEA
3.2
33%
15%
CPHS
3.4
3.3
34%
32%
28%
20%
IDEA
3.4
21%
18%
CPHS
3.4
3.4
36%
33%
33%
31%
IDEA
3.4
20%
18%
Values are similar if within 0.3
1=Much less than most courses, 2=less than most, 3=about average, 4=more than most, 5=much more
Instructor Related Course Requirements
(Some or Much required)
Reading and memorization
were new categories in 0809
Instructor Related Course Requirements
2008-09 Breakdown
Percent of CPHS classes selecting objective as either
Essential or Important (FIF)
Student ratings of progress on objectives chosen as
Essential or Important
1=no progress
2=slight progress
3=moderate progress
4=substantial progress
5=exceptional progress
Percent of Reliable CPHS Classes in each
Category vs. IDEA (Adjusted scores*)
Expected
Distribution
A.
Progress on
relevant
objectives
07-08
08-09
B.
Excellence of
teacher
C.
Excellence of
course
07-08
07-08
08-09
08-09
Summary
(Average of A,
B, C)
07-08
08-09
Much Higher
10%
1%
0%
0%
0%
4%
3%
2%
1%
Higher
20%
15%
13%
25%
27%
19%
20%
20%
18%
Similar
40%
53%
53%
✓
46%
39%
41%
35%
46%
51%
Lower
20%
13%
21%
17%
17%
27%
15%
23%
14%
Much Lower
10%
18%
13%
12%
17%
✓
9%
27%
✓
9%
16%
>30% in top 2 categories: teaching effectiveness appears to be superior to IDEA Database
* Results adjusted for 5 factors: Student motivation to take the class regardless of who taught it; Student work
habits; Class size; Student effort not attributable to the instructor; Course difficulty not attributable to the instructor
CPHS Adjusted Mean Scores
A.
Progress on
relevant
objectives
B.
Excellence of
teacher
C.
Excellence of
course
Summary
(Average of A,
B, C)
07-08
08-09
07-08
08-09
07-08
08-09
07-08
08-09
All CPHS Courses
3.8
3.8
4.1
4.0
3.8
3.7
3.9
3.8
IDEA System Courses
3.8
3.8
4.2
4.2
3.9
3.9
3.9
3.9
Overview
• On-line evaluations still have lower response
rate
 Improving with use of fewer ‘co-teachers’ but can
improve more
• CPHS similar to national database on:
 Amount of reading
 Amount of work in non-reading assignments
 Difficulty of subject matter
 Do our students have higher expectations?
Overview
• Ave. number of objectives still 4.3
 3-5 recommended as max.
 High on CT, Memorization, reading
• For computer application, math/quant, &
creative/artistic: Almost none require
“Much”
Overview
• Primary class format remains lecture
 Should more active learning be stressed?
• Faculty believe more of our courses
address “Learning to apply course
material” & “Developing specific skills,
competencies, and points of view”
compared to IDEA database
Overview
• Students rate us lower on:
 Learning to analyze & critically evaluate
 Developing clearer understanding and
commitment to personal values
 Developing skills in expressing myself orally
or in writing
 Gaining broader understanding & appreciation
of intellectual/cultural activity
Overview
• We do better in
 Excellence of Teacher
 Excellence of Course
 Vs. Progress on relevant objectives
• But all three are lower than expected
• Everyone needs development: almost no
course or instructor in 90th %ile
Questions to Consider
• Are we balancing teaching methods and course
requirements as intended
• Are we emphasizing correct objectives
• Are courses rigorous?
• What has been the affect of the new curriculum?
• Do we need to require computer application,
math/quantitative?
Questions to Consider
•
•
•
•
Too much lecturing?
Disconnect on what faculty say vs. students
Where should we improve??
Others?
Continued Assessment
‘When you dance with a bear, you can’t quit just
because you’re tired”
(Russian proverb)
• Need to continue assessment work
• Need to improve toward goals
• Better measure student outcomes