Transcript Wayne State University College of Education TEAC: Inquiry Brief Proposal Presentation Presented by the TEAC Advisory Committee April 24, 2012

Wayne State University
College of Education
TEAC: Inquiry Brief
Proposal Presentation
Presented by the TEAC Advisory Committee
April 24, 2012
Overview
 Program Overview
 Claims & Rationale
 Methods of Assessment
 Results, Discussion, Plan

Claim 1

Claim 2

Claim 3
 Auditor’s Comments
Program Overview
 Background and History
 Distinguishing Features of the Program
 Requirements for Admissions & Graduation
 Demographics of Initial Certification Candidates
 Description of Faculty
 Logistics of Program
Claims & Rationale
 Claim 1: Initial Certification Completers
understand subject matter knowledge they will teach
(Learning to Learn, Multicultural, Technology).
 Claim 2: Initial Certification Completers have
pedagogical knowledge (Learning to Learn,
Multicultural, Technology).
 Claim 3: Initial Certification Completers know
how to teach in a caring and effective matter
(Learning to Learn, Multicultural, Technology).
Methods of Assessment
 Sources of Measure
 Surveys (performance assessments, WSU surveys,
MDE surveys, observations)
 Course Grades (methods & total GPA)
 Ratings of Capstone Conversation
 Scores on MTTC
 Research Design
 Reliability and Validity Factors
Claim 1:
Subject Matter
Knowledge
Subcommittee: Janet Andrews, Mariane Fahlman,
Chavon Jackson, Jo-Ann Snyder, Cheryl Somers,
Initial Certification Completers
understand subject matter
knowledge they will teach.
 Learn and understand the subject matter they hope to
teach.
Competencies & Data Sources
 Competencies: 1, 2,
 Data Sources:
•Unit Plan
•MDE Supervisor Survey
•University Supervisor
Final Eval. Observation
•GPA
•MTTC
•Cooperating Teachers
Final Eval. Observation
•WSU Student Exit Survey
Performance Assessments and
Observation Data
Claim 1: Findings
 Finding 1 – based on unit plan
assessment, 95% of students
demonstrated knowledge of subject
matter at the proficient or exemplary
level
 Finding 2 – both cooperating teachers
and college supervisors rates 93%
proficient or exemplary in knowledge of
subject matter
Claim 1: Findings
 Finding 3 – all teacher candidates
receive a grade of C or better in all
subject matter courses.
 Finding 4 – all teacher candidates
successfully complete the MTTC.
Findings of CCT: LEARNING
TO LEARN
 Receiving passing grades in 100 % of
content area classes is evidence that
strategies for learning were at their
command
 Successful completion of the MTTC test
demonstrated their mastery of content
material again evidencing their ability to
learn
Discussion of Content
Knowledge (claim 1)
The data we examined suggests that our
candidates have learned and do understand
the subject matter when they leave us.
Our data sources for cross-cutting themes
appeared to be too narrow.
Additionally we have data we have not
examined for cross-cutting themes (e.g. mc
course, tech course).
Implications for Content
Knowledge (claim 1)
 Need better documentation of how students use
technology to develop concepts in their discipline.
 We need to include other sources that demonstrate
evidence of this such as:
DIVERSITY:
University Foreign
Culture requirement,
COE Multicultural
requirement,
COE inclusion
requirement,
TECHNOLOGY
Computer literacy
requirement,
Technology course
in every curriculum
E-Portfolios
Blackboard
LEARNING TO
LEARN:
What else can we
do?
Claim 2:
Knowledge of
Pedagogy
Subcommittee: Oscar Abbott, Abby Butler, Bev
Schneider, David Whitin, Phyllis Whitin,
Initial Certification Completers
have pedagogical knowledge:
 They organize for effective instruction by
establishing a safe and orderly environment that is
conducive to learning and creative and critical
thinking.
 They utilize school, community, and technological
resources and integrate content across the
curriculum.
 They use a range of assessment strategies to
inform ongoing instruction.
Competencies & Data Sources
Competencies: 2, 3, 4, 8, 9
Data Sources:
 Community
Observation
 District School
Curriculum Analysis
Final Eval Observ.
 Univ. Supervisor’s
Final Final Eval
Observ.
 Classroom
 WSU Student Entry
 Lesson Plan
 WSU Student Exit
Management Plan
 Unit Plan
 Case Study
 Cooperating Teacher’s
Survey
Survey
 MDE Supervisor Exit
Survey
Performance Assessments and Observation Data
Competency
Performance Assessments
CO* DSC* CMP*
2
37%
49%
3
48%
LP
UP
Observations
CS
CTF
USF
89% 95%
93%
96%
91% 97% 87%
95%
94%
4
42%
90% 95%
95%
96%
8
44%
85% 90% 76%
97%
97%
49%
89%
95%
97%
9
40%
Percentages are proportions of students rated proficient or
higher on the respective competency for each data source.
*Indicates assessments given early in the program where a
score of basic is considered acceptable.
Findings for Pedagogy (Claim 2)
•
Higher percentages of proficient or exemplary ratings are
found in assessments that take place later in the program.
•
Highest ratings (Lesson/Unit Plan) and Cooperating
Teacher and Supervisor assessments indicate that
candidates exit the program pedagogically well prepared.
•
There are lower ratings for CO, DSC, CMP, and CS. CO,
DSC, and CMP reflect introductory experiences. CS is
assessed later in the program.
Discussion for Pedagogy (Claim 2)
 CO, DSC, and CMP, with only 37 – 48% showing
proficient and above performance, are indicative of a
beginning level understanding when this content is
first introduced. A rating of basic is considered
satisfactory for these 3 assignments.
 Data from 2010-2011 reflect the work of many
candidates who were not exposed to the service
learning component that is designed to give candidates
experience working with children and community
organizations.
 The low Case Study scores (87% and 76%) are of
more concern since it is assessed later in the program.
Discussion for Pedagogy:
Cross-Cutting Themes (Claim 2)
 Learning to Learn is reflected in five performance
assessments and two observations: CO, DSC, CS, LP, UP,
and CTF, USF.
 Multicultural is reflected in three performance assessments
and two observations: DSC, LP, UP and CTF, USF.
 Evidence for these two themes is integrated throughout
students’ experiences.
 Technology is only reflected in two assessments: DSC, LP.
 Evidence for Technology theme is thin.
Implications for Pedagogy (claim 2)
 Closely monitor assessments with lower ratings, and
implement changes to strengthen the program.
 These results, particularly for CS, suggest modifications to
our program, e.g. to incorporate into methods classes
additional experiences involving formulating instructional
decisions based on ongoing observational assessments.
 Modules developed by SED may be useful to all initial
certification candidates in their work on CMP.
 Revise assessment rubrics that reflect technology in
candidates’ teaching.
 Initiate conversations to examine evidence of technology-
integrated pedagogy at the program level for each content
area.
Claim 3:
Caring and
Effective
Teaching
Subcommittee: Tom Edwards, Sharon Elliott,
Sharon Sellers-Clark, Bo Shen, Marshall Zumberg,
Initial Certification Candidates
are Caring and Effective
 Caring Teaching: Reflects interest in, and
concern for, students.
 Effective Teaching: Results in student
learning
Competencies & Data Sources
 Competencies: 3, 4, 5, 6, 7, 10
 Data Sources:
Community
Observation (CO),
Classroom
Management Plan
(CMP),
Reflective Journal (RJ),
Teaching and Learning
Statement (TLS),
Lesson Plan (LP),
Unit Plan (UP),
Case Study (CS).
Performance Assessment
Competency
CO*
3
CMP*
RJ
48%
73%
4
73%
5
47%
TLS
LP
UP
CS
91%
97%
87%
84%
90%
95%
87%
88%
95%
85%
86%
91%
82%
6
42%
44%
77%
81%
7
35%
44%
74%
86%
10
39%
82%
Percentages are proportions of students rated proficient or higher on the
respective competency for each data source.
*Indicates assessments given early in the program where a score of basic is
considered acceptable.
Note:
Community Observation (CO), Classroom Management Plan (CMP), Reflective
Journal (RJ), Teaching and Learning Statement (TLS), Lesson Plan (LP), Unit Plan
(UP), Case Study (CS)
Competencies & Data Sources
 Competencies: 3, 4, 5, 6, 7, 10
 Data Sources: (Con’t.)
 Capstone Conversation (CC),
 WSU Student Entry Survey (WSEn),
 WSU Student Exit Survey (WSEx)
 MDE Student Exit Survey-MDE (MSEx),
 MDE Supervisor Survey (MSS),
 University Supervisor Final Observation (USF),
 Cooperating Teacher Final Observation (CTF).
Survey and Observation Data
Percentages are proportions of students rated proficient or higher on the
respective competency for each data source.
*Indicates assessments given early in the program where a score of basic is
considered acceptable.
Note: CC=Capstone Conversation; WSEn=Student Entry Survey;
WSEx=Student Exit Survey; MSS=MDE Supervisor Survey; USF=University
Supervisor Final Evaluation Observation; CTF=Cooperating Teacher Final
Evaluation Observation.
Findings for Caring & Effective (Claim 3)
 Percentage of proficient and exemplary ratings tends to be
higher on assessments that occur later in the program.
 Highest percentages of students rated proficient and
exemplary on items related to caring and effective teaching
occurred in the MSS, USF, and CTF.
 There are only small differences in the percentages of
students rated proficient and exemplary on the WSEx and
MSEx compared to the USF and CTF.
 Lowest percentages of proficient and exemplary ratings
occurred on the CO and CMP, both of which occur early in
the program. (It is also noteworthy that a rating of “basic” is
considered satisfactory on those two assessments.)
Findings for Cross-Cutting Themes
in Caring & Effective Teaching
 Caring & Effective Teaching: Data drawn from 7
performance assessments, 4 surveys, and 2 observations
across 6 competencies.
 Learning to Learn data was drawn from
 Four of the performance assessments (CO, RJ, TLS, LP)
across Competencies 7 & 10.
 Three (CC, WSEn, WSEx) of the surveys in Competency 6.
 Both of the observations in Competency 6 (CTF, USF)
Findings for Cross-Cutting Themes
in Caring & Effective Teaching
 Caring & Effective Teaching: Data drawn from 7
performance assessments, 4 surveys, and 2 observations
across 6 competencies.
 Multicultural Perspective data was drawn from
 Six of the assessments (CO, RJ, TLS, LP, UP, CS) across
Competencies 5, 6, 7, & 10.
 All four of the of the surveys across Competencies 5 and 7.
 Both of the observations across Competencies 5 and 7
Findings for Cross-Cutting Themes
in Caring & Effective Teaching
 Caring & Effective Teaching: Data drawn from 7
performance assessments, 4 surveys, and 2 observations
across 6 competencies.
 Technology data was drawn from
 Three of the assessments (TLS, LP, UP) in Competency 4.
 Three of the surveys (WSEn, MSS, WSEx) across
Competencies 3, 4, and 6.
 There was no data addressing issues of caring & effective
teaching from either either observation that provided direct
evidence of technology.
Findings for Cross-Cutting Themes
(CCTs) in Caring & Effective Teaching
 There was data for each of the CCTs embedded in the data
for Caring & Effective.
 The data that was there supports each theme.
 Multi-Cultural Perspective had the most supporting data.
 Technology had the least supporting data.
 Technology is not addressed directly in either the CTF or
USF.
Discussion: Caring & Effective Teaching
 Our analysis of the data revealed a clear trend in the
growth of the proportion of candidates rated proficient or
higher on the performance assessments in areas related to
caring & effective teaching.
 This analysis suggests that our program completers grow
over time in their ability to provide caring and effective
teaching.
 The vast majority reach proficient or exemplary ratings.
 The evidence we have supports the claim that we prepare
teachers who are caring & effective.
Discussion: CCTs in Caring & Effective
 Regarding the cross-cutting themes, the evidence
 of a multi-cultural perspective is strongest, because it comes
from more data sources and addresses more of the related
competencies.
 of learning to learn is adequate ,
 of technology is weakest, coming from only a few data sources,
one of which is self-reported.
Implications for Caring and
Effective Teaching
 Revisit the rubrics of those performance assessments with
the lowest percentages of proficient and exemplary ratings
to ensure that the rubrics are providing the information we
seek.
 Consider revising the rubrics for all of the performance
assessments so they provide more information about our
candidates technological proficiency in the service of
pedagogy.
 Revisit the CMP rubric to learn why no data related to
caring & effective teaching from that source supports any
cross-cutting theme.
 Revise the observation reporting forms to provide
information on technology.
Small Group Discussion
1. Do we have the evidence to support these claims? Why/Why not?
2. What are the strengths of the evidence?
3. What might be missing?
4. Other assessments/tools we should consider to assure:

Subject Matter Knowledge

Pedagogical Knowledge

Caring & Effective Teaching
With the 3 cross cutting themes:

Learning to Learn

Multicultural Perspectives

Technology
Auidtor’s Comments