Transcript The Role of Assessment in the EdD – The USC Approach

The Role of Assessment in
the EdD – The USC Approach
To Whom is the EdD Accountable?



What are the “checks and balances?”
For the PhD degree, one of the major
“quality control” mechanisms is the
dissemination of work for eternal peer
review
For the EdD, if the primary role is not to
produce theoretical, generalizable
knowledge for academic outlets, what is
the equivalent?
The USC Context
• Large - 150 plus students per year
• One EdD - school wide
• Common mission – improve urban education
• Common themes that define core (learning,
accountability, diversity, leadership
• All faculty are EdD faculty
• Thematic dissertations - 1 faculty & team of 8-9
students
How We Are Approaching
Assessment
At USC, the mission of the School is “to strengthen urban education
locally, nationally, and globally. We define urban education broadly,
conceptualizing it as something that takes place within many
contexts including pre-kindergarten through high school, in human
services, higher education, and workplace settings. We seek to
transform urban education by:
•
Preparing and developing educational leaders who are change
agents with a commitment to focusing on urban education,
developing the competencies to address the complex
educational and social issues facing urban communities.
•
Leading the search for innovative, efficacious, and just solutions
to the challenges in urban education by engaging in
collaborative action research on educational practices and
policy.
•
Creating mutually beneficial partnerships to rethink curriculum,
improve educational environments, and develop sound policy.
External Evaluation
Our “new” program is now 10 years old – we are
conducting an external evaluation (West Ed) with
the following questions:

How have the programs been implemented?

To what extent have the programs achieved
their student learning, program quality, and
educator placement objectives?

What implementation and student factors are
predictive of student outcomes in the
programs and beyond in the students’
professional employment?
Approach
An adapted version of the Kirkpatrick Four
Level Evaluation model – measures are
organized by type of outcome:
Reactions to the Program,
 Classroom learning,
 Transfer/implementation of learning, and
 Program outcomes.

Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs (3rd ed.).
San Francisco: Berrett-Koehler Publishers.
A sampling of instruments and
collection methods - Formative
Reactions (Ensures that students are motivated and
interested in learning.)
 Student feedback forms to measure emotions, motivation,
and self-efficacy
 Student end-of-course evaluation
 Faculty end-of-course evaluations
 Administrators and student advisor feedback of students’
reactions
Learning (Measures knowledge acquired, skills learned and
improved, and/or attitudes changed)
 Student assignments and grading rubrics
 Student pre-assessment of knowledge and skills at the
beginning of a course compared to achievement of course
goals measured by course grading rubrics (pre- and postassessments)
Formative Assessment
Continued…
Transfer/Implementation/Behavior (Measures
learning transfer and the extent to which
behavior changes as a result of learning)






Student performance in subsequent courses
Early warning systems
Student self-report of using new knowledge and skills in
their professional work
Faculty perceptions of students’ applying knowledge and
skills learned in previous classes to learning opportunities
in current classes.
Capstone alignment with program goal, quality indicators,
and completion rates
Employer reports about student job performance
throughout the program
A sampling of instruments and
collection methods - Summative
Program Results (Measures the extent to which the
program goal has been met or exceeded.)
 Graduation
rates
 Placement statistics
 Graduate attainment of career goals
 Graduate perceptions of knowledge, experience, and selfefficacy in performing the program goal
 Faculty perceptions of the implementation of the program
 Employer perceptions of graduate performance
 Third-party and University evaluations
Establishing a Process
The results of this external evaluation will be
used by the Dean and the Governance
Committee to create a long term
strategy/structure/process for ongoing
assessment