Transcript Unit Assessment System

Unit Assessment System
Teacher Education at Purdue
March 4, 2012
What is our
assessment
philosophy ?
CONTINUOUS IMPROVEMENT
All components of teacher education at
Purdue engage in continuous improvement
by gathering and analyzing relevant data
and making decisions based on data.
3
BALANCE
The unit balances central coordination of
the overall assessment system with
encouraging individual offices and programs
to develop flexible, creative, assessment
system components that fit their mission
and foster continuous improvement.
4
What have we done
since the last visit to
improve our UAS?
THE FIRST THREE YEARS
• CI Focus: Initial licensure program
candidate assessment system
– Reviewed the literature and state systems to
identify clear criteria for instructional performance
– Created the Gate C & D rubrics to measure those
performances using the identified criteria
– Worked with program faculty to align program key
assessments with the unit rubrics
6
THE GATE D RUBRIC
• FOUR SECTIONS PLUS A HOLISTIC SUMMATIVE
ASSESSMENT
I.
II.
III.
IV.
PLANNING INSTRUCTION (also used for Gate C)
TEACHING AND LEARNING EXPERIENCES
ASSESSING LEARNERS
REFLECTING ON LEARNING AND TEACHING
• EACH SECTION HAS…
– 3-5 unit items
– Items to assess impact on P-12 student learning
– Encouragement to programs to add discipline-specific programs
• FOUR POINT SCALE FOR PERFORMANCE RATINGS
– Clear descriptions of target behaviors in the cells
• PROGRAMS ALIGN TO THE UNIT RUBRIC IN VARIOUS
WAYS
7
THE GATE D RUBRIC
Student Communication
Opportunities (II.3)
Communicating Performance
Expectations and Assessment
Criteria (III.1)
#3
#4
0
3
10
1
0
0
0
6
6
8
8
8
6
0
0
0
Focus of Student Assessments
(I.5)
0
3
10
1
Summative Assessment of
Student Performance (III.1)
0
3
10
1
Expert
Sound Instructional Strategies
(I.4, II.1)
#2
Proficient
Subject Matter Focus (I.3)
#1
Developing
Rubric Criteria
Beginning
English Education
Engineering/Technology
Education
0
4
17
34
0
6
14
34
0
4
13
34
I.4 Strategies and Resources,
Including Technology, to Support
Student Learning NCTE Standards 2.0
(2.4, 2.5, and/or 2.6), 3.3 (3.3.1, 3.3.2,
and/or 3.3.3) 4.0 (4.2, 4.3, and/or 4.6)
0
3
19
34
I.5 Focus of Student Assessments
NCTE Standards 4.0 (4.10)
0
1
20
34
Rubric Criteria
I.1 Knowledge of Students’ Learning
Needs and Interests
I.2 Knowledge of Community, School,
and Classroom Factors NCTE
Standards 1 (1.3), 3.0 (3.1.3)
I.3 Subject Matter Focus of the
Unit/Lesson(s) NCTE Standards 3.0
(3.1.4, 3.1.5, and/or 3.1.7), 3.5 (3.5.1,
3.5.2, 3.5.3, and/or 3.5.4)
8
ACCURACY, FAIRNESS, & CONSISTENCY
• CI Focus: Initial licensure program
candidate assessment system
– Use the Gate C&D rubrics for all initial
programs
– Define and use key assessments for all
programs
– Audit transition points
– Coordination
• Programs– Program convener
• Multiple Section Courses – Course coordinator
9
THE PAST FOUR YEARS
• CI Focus: The operational assessment/decision
system
– Created to specify the decision system
– Created a UAS annual report template
– Implemented TEC change decisions
• CI Focus: Advanced programs for teachers
– Reviewed and categorized all graduate programs
(TEC)
– Created task force to design a unit framework,
providing clear criteria for performance (TEC)
– Currently, advanced programs for teachers are being
redesigned to the new unit framework (Programs)
10
CURRENT
• CI Focus: Program Assessment Systems
(Standard 2)
– March 2011: First SPA program reports in 20 years
• Results comparable to national averages
• Most concerns related to the assessment systems
– December 2011: First state program reports in 20
years
• Two approved, one approved with conditions
• Most concerns related to the assessment systems
– Fall 2011: TEC established Standard 2 at the program
level as the continuous improvement focus (2011-12)
• Assessment Coordinator met with all TE Program Conveners
• Program faculty are currently working to improve their
assessment systems
11
What are the features
of our current unit
assessment system?
OPERATIONAL DECISION SYSTEM
Purdue University Teacher Education Operational Assessment/Decision System
Office
& Program
Management
OFFICE
& PROGRAM
MANAGEMENT
Registrar
Graduate Office
Praxis (ETS)
Office of Professional
Preparation & Licensure
Program Completers
Employers
Cooperating Teachers
DATA COLLECTION
Applicants
Office of Field Experiences
Business Office
Program Conveners
Office of Advising &
Recruiting
Cooperating School Administrators
Purdue Student Education
Council
Data Types
Decision support data gathered include:
stakeholder perceptions, candidate
performances, demographic information,
operational process information, etc.
Data Gathering Methods
Methods used to collect and store this data
include: surveys, course evaluation forms,
portfolio assessments, operational performance
metrics, grades, etc.
Data Storage
This information is stored in office and
13
departmental data repositories, both electronic
and paper.
Decision
Makers
DECISION
MAKERS
COE Leadership
Team
Instructor Data
Candidate Information
Analysis
ANALYSIS
DATA COLLECTION & ANALYSIS
Data Sources
DATA SOURCES
UAS
Annual
Report
Office of
Strategic
Assessment
Key
Reporting and information sharing
Information/Consultation upon request
Post-decision feedback and influence
ETS:
Educational Testing Services
ADLE:
Associate Dean for Learning & Engagement
This chart depicts the way data is used to improve the
Purdue Teacher Education Program.
Teacher
Education
Administration
(ADLE,
NCATE Coordinator)
Teacher
Education
Deans
Provost
Teacher
Education
Council
Dean of
College of
Education
Program
Convener
Council
Dean’s
Advisory
Council
Decisions are made and enacted
by entities in green.
CANDIDATE ASSESSMENT SYSTEM
Unit and Program Levels
Unit Level includes the conceptual framework and related
standards, systems for assessing and managing critical
transitions, and aggregated performance data from unit
wide rubrics
Program Level (a) applicable program standards;
(b) program assessment systems based on program and
unit guidelines for transitions and key assessments;
(c) national or state program review
14
CANDIDATE ASSESSMENT SYSTEM
OnePurdue Transition to Banner
– Facilitated grade reports for content assessment
– Facilitated some aspects of Gate audits
– DegreeWorks… coming soon…
Performance-based Assessment Technology
– P3T3 grant (before 2005)
– TaskStream, an outsourced solution (2006-present)
– We are currently researching alternatives
• Inexpensive
• Integrated with Banner, DegreeWorks, and BlackBoard
• Excellent capabilities
18
Questions?