Transcript Assessment System - Widener University

Assessment System
Overview
Center for Education
Overview for the NCATE BOE Team
April 18-22, 2009
Assessment concerns
Does the assessment system provide information about
• Aggregated evidence about candidate knowledge, skills and
dispositions?
• Disaggregated evidence about areas of strength and weakness
in individual programs?
• Formative,on-going feedback to candidates, faculty,
administrators and PK-12 partners?
• Summative feedback about program outcomes?
• Benchmarked information about candidate proficiency and
progress linked to transition points?
Characteristics of the current
assessment system
Assessments are aligned with
•
the conceptual framework
•
professional standards of PDE
•
or specialized professional associations
Assessments are tied to transition points.
The content of instruction is aligned with assessments.
Multiple methods and multiple evaluators are used.
Challenges in AY 2006
• Integrating information that we already had in order to
have a “system”
• Refining data collection
• Acquiring new skills (rethinking rubrics, achieving
alignment)
• Creating new assessments for national program
recognition
• Implementing TaskStream at all levels
• Creating a collective assessment culture by through
collaboration and cross-disciplinary participation
• Understanding that assessment is everyone’s job and
everyone has to be involved
What are the broad elements
of the assessment system?
• Assessment tools and strategies used to evaluate
candidate proficiencies and faculty performance
• Evaluators, including faculty, administrators, school
partners, and candidates, as well as other members of
the professional educational community
• The proficiencies that are evaluated
• Procedures and time-tables for collecting evidence at
each level of the assessment system
Organization of the assessment
system
• See next slide
Databases
• ACCESS Database (2003 - 2006, transiting to a revised
database in 2007)
• ACCESS Database (2007- present)
• PRAXIS Database
• Enrollment/Completion Database
• Databooks (collections of required course assessments for
key courses in all programs)
• TaskStream DRF’s
• Mobile clinical evaluations using PDA’s & ACCESS
• Miscellaneous databases needed to answer specific
questions
ACCESS Databases
relevant evidence linked to transition points
2003 ACCESS database
(candidates for initial
teaching licenses only)
• Admission to Widener
• Admission to teacher
certification
• PPST scores
• Writing samples (yearly)
• Academic progress
• Student teacher placements
• PRAXIS scores
• FBI/Criminal history
compliance
2007 ACCESS database
(all
candidates including candidates for
graduate level certificates and
degrees)
•
•
•
•
•
•
Admission to Widener
Admission to certification
PPST scores
Writing samples
Academic progress
Student teaching/clinical
practice placements
• Proficiency in student
teaching/clinical practice
• PRAXIS scores
• FBI/Criminal history
compliance (as relevant)
PRAXIS database
• Pass rates by programs (Title II - HEA reports)
• Total scaled scores disaggregated by program and by
level
– Initial teaching license baccalaureate level
– Initial teaching license post baccalaureate level
– Specialist certificates
– Advanced certificates for other school personnel
(principals and supervisors)
• Category scores also disaggregated by program and
program level
Enrollment/Completer Database
•
•
•
•
•
Headcounts of candidates
Undergraduate degrees and certificates
5-year program candidates
Master degree candidates
Initial teaching, specialist and other school
personnel in graduate level certification
programs
• Doctoral degree programs
• School Psychology candidates
Databooks: Evidence and
indicators
• Assessment narratives
• Directions to students
• Rubrics and scoring guides for every student
This data is compiled for each course by the NCATE
office on EXCEL spreadsheets.
Data is then reported at semester assessment meetings
to all unit faculty as a comprehensive set of data
that indicates the percentages of candidates who
scored each level of proficiency in each key course.
TaskStream: Evidence and
indicators
• Candidate work samples
• Rubrics aligned with standards or the unit
conceptual framework
• Evidence of candidate proficiency
• Evaluator feedback
This evidence is assessed on a continuing basis by
students and faculty. Candidate performance is
aligned with professional standards, PDE standards
and the unit conceptual framework, according to
relation to proficiency levels. The evidence can be
aggregated to study program outcomes.
Miscellaneous Databases
• These are individual collections of data that
address specific questions.
Who are the evaluators?
•
•
•
•
•
•
•
Administrators
Advisory Boards
Cooperating faculty in PK-12 schools
Faculty committees
Center for Education Faculty
Arts and Sciences Faculty
Faculty who supervise candidates in student
teaching, practica and internships
• Candidates
Administrators assess
• Faculty proficiency in teaching, scholarship
and service to their professions
• Graduate/alumni satisfaction
• Employer perceptions
Advisory boards assess
•
•
•
•
•
Knowledge of content
Knowledge of pedagogy
Pedagogical skill
Dispositions
Employer perceptions
Cooperating faculty assess
•
•
•
•
Knowledge of content
Knowledge of pedagogy
Pedagogical skill
Dispositions
Faculty committees assess
•
•
•
•
•
•
•
•
Knowledge of content
Knowledge of pedagogy
Pedagogical skill
Dispositions
Readiness for next steps in licensure or degrees
Candidate work samples
Student learning
Faculty proficiency in teaching, scholarship and service to
their professions
Arts and Sciences faculty assess
• Knowledge of content
Center for Education faculty assess
•
•
•
•
•
•
•
•
Knowledge of content
Knowledge of pedagogy
Pedagogical skill
Dispositions
Readiness for next steps in licensure or degrees
Candidate work samples
Student learning
Faculty proficiency in teaching, scholarship and service to
their professions
• Graduate/alumni satisfaction
• Employer perceptions
Faculty who supervise
candidates assess
•
•
•
•
•
•
Knowledge of content
Knowledge of pedagogy
Pedagogical skill
Dispositions
Readiness for next steps in licensure or degrees
Candidate work samples
Candidates assess
• Student learning
• Faculty proficiency in teaching
• Graduate/alumni satisfaction
Has the unit benefited from
continuous assessment?
• Faculty as a unit are aware of candidate proficiencies and
program content.
• Assessment is now a unit-wide enterprise because all
faculty attend all assessment workshops and share
assessment information.
• Skills, assessment language, and enthusiasm for assessment
has increased.
• Time has been created to have cross-disciplinary discussions
for the first time.
• Faculty in Higher Education and Human Sexuality Education
are using the NCATE model to develop standards and
assessment for their programs.
What are the next steps?
• Embedding large parts of the system within the Office
of Certification
• Sustaining gains in collaboration throughout the unit
and the PEU
• Refining our tools
• Modifying assessments in line with recent changes in
PDE accountability mandates at all levels of
certification
• Continuing the integration of the assessment of nonlicensure programs
• Adapting our methods to the proposed Widener
assessment database