Susan Malone Mercer University

 “The unit has taken effective steps to

eliminate bias

in assessments and is working to establish the

fairness, accuracy

, and

consistency

of its assessment procedures and unit operations.”

  Address contextual distractions (inappropriate noise, poor lighting, discomfort, lack of proper equipment) Address problems with assessment instruments (missing or vague instructions, poorly worded questions, poorly produced copies that make reading difficult)

  [Review candidate performance to determine if candidates perform differentially with respect to specific demographic characteristics] [ETS: Guidelines for Fairness Review of Assessments; Pearson: Fairness and Diversity in Tests]

    Assure candidates are exposed to the K, S, & D that are being evaluated Assure candidates know what is expected of them Instructions and timing of assessments are clearly stated and shared with candidates Candidates are given information on how the assessments are scored and how they count toward completion of programs

  Assessments are of appropriate type and content to measure what they purport to measure Aligned with the standards and/or learning proficiencies they are designed to measure [content validity]

 ◦ ◦ ◦ ◦ Produce dependable results or results that would remain constant on repeated trials Provide training for raters that promotes similar scoring patterns Use multiple raters Conduct simple studies of inter-rater reliability Compare results to other internal or external assessments that measure comparable K, S, & D [concurrent validity]

    Dispositions Assessment Portfolios Analysis of Student Learning Summative Evaluation

  ◦ ◦ ◦ ◦ Alignment with INTASC standards, program standards, and the Conceptual Framework (accuracy; content validity) Matrices LiveText standards mapping PRS Relationship to Standards section PRS alignment with program standards requirement in Evidence for Meeting Standards section ◦ ◦ Alignment with other assessments (accuracy; concurrent validity) Matrices Potential documentation within LiveText Standards Correlation Builder

   Rubrics/assessment expectations shared with candidates in courses; field experience orientations; and LiveText (fairness) Rubrics/assessment expectations shared with cooperating teachers by university supervisors (consistency) Statistical study (in process) examining correlations among candidate performances on multiple assessments (where those assessments address comparable K, S, & D) (consistency; concurrent validity)

   Multiple assessors (consistency) Exploration of faculty’s assumptions re: purpose of the assessment, expectations of behaviors, and meaning of rating scale (consistency; reliability) Revision of rating scale, addition of more specific indicators, development of two versions (courses/field experiences) (accuracy; content validity)

   Norming session with supervisors (consistency; reliability) Revision of instructions to align more closely with rubric expectations and expected process (fairness; avoidance of bias) Review of coursework and fieldwork to ensure candidates are prepared for assignment (fairness)

  Seeking feedback from experts (P12 partners) on whether assignment reqs and assessment criteria are authentic (accuracy; content validity) Annual review of data disaggregated by demographic factors (gender, race/ethnicity, site, degree program) (fairness, avoidance of bias)

   Recent revision of portfolios and rubrics to align with new INTASC (accuracy; content validity) Revision of rubrics to include more specific indicators related to the standards (change from generic rating descriptors) (accuracy; content validity) Cross-college workshop on artifact selections (accuracy; content validity)

  Annual review of artifact selections (accuracy; content validity) Inter-rater reliability study (consistency; reliability)

    Review of rubric expectations and all other PR and ST assignments to ensure opportunities to demonstrate all standards and indicators during experience (fairness) Workshops for supervisors on the rubric expectations (consistency) Feedback from cooperating teachers on relevance of the assessment (accuracy; content validity) Annual review of data disaggregated by demographic variables (fairness; avoidance of bias)

  Statistical study to identify correlations among entry reqs and successful program completion Statistical study to determine if key assessments and entry criteria are predictive of program success (as defined by success in student teaching)