CLEAR 2008 Annual Conference

“Crossing Borders, Exploring Boundaries: Providing Test Accommodations for High Stakes Examinations in Canada and the US.”

Elizabeth D. Azari, JD Janet D. Carson, JD Deborah Danoff, MD, FRCPC, FACP Catherine Farmer, PsyD Robert S. Lee, CA, MBA Anchorage, Alaska

Agenda

• Principles: What frameworks inform your approach to requests for test accommodations?

• Weaving the Principles: How is your approach affected by the frameworks?

• The Future: Where are the boundaries? Are the frameworks changing? What are the trends?

CLEAR 2008 Annual Conference Anchorage, Alaska 2

Audience Survey

By show of hands, do you: Handle requests for test accommodations?

Administer a licensing or certification exam?

Other exam?

Administer using paper and pencil? CBT? Physical/practical?

How often? Periodically? Continuously?

Where? In US? In Canada? Elsewhere?

CLEAR 2008 Annual Conference Anchorage, Alaska 3

Principles

• Measurement Framework • Legal Framework – Canada – United States • Societal Framework CLEAR 2008 Annual Conference Anchorage, Alaska 4

Measurement Framework Across the Borders

• High-stakes licensing and certifying programs rely on the use of standardized examinations in which each examinee is allowed to demonstrate his/her knowledge, skills, and abilities in the same, known, highly-controlled context.

• Accommodation on an examination refers to a change made in one or more aspects of that examination context, e.g., content, format, environment, or administrative procedures.

CLEAR 2008 Annual Conference Anchorage, Alaska 5

Measurement Framework Across the Borders

• Purpose of examination • Format – Multiple choice items (

single best answer, short menu)

– Short answer questions – Oral – Performance • Delivery method (written, computer based, performance, etc.) • Frequency of administration • Length CLEAR 2008 Annual Conference Anchorage, Alaska 6

Measurement Framework Across the Borders

• What knowledge and/or skills are being measured?

• Will the absence of an accommodation: – Interfere with or preclude the examinee’s demonstration of the knowledge and/or skills being tested? – Result in an inability to adequately/appropriately assess the examinee’s knowledge and/or skills?

CLEAR 2008 Annual Conference Anchorage, Alaska 7

Measurement Framework Across the Borders

• Will the provision of an accommodation: – Interfere with or preclude the measurement of the examinee’s knowledge and/or skills?

– Give the examinee an advantage not available to other test-takers?

– Will the scores obtained by the accommodated examinee have the same “meaning” as scores obtained by those testing under standard conditions?

– Should the accommodation be disclosed?

CLEAR 2008 Annual Conference Anchorage, Alaska 8

Measurement Framework Across the Borders

• Will the provision of a particular accommodation: – Raise doubts as to whether a valid assessment was made?

– Raise doubt about score comparability?

– Pose concerns for test security?

CLEAR 2008 Annual Conference Anchorage, Alaska 9

Legal Framework Canada

• There is a duty to accommodate: – Canadian Federal Charter of Rights and Freedoms (1982) and – Provincial/territorial legislation governing human rights CLEAR 2008 Annual Conference Anchorage, Alaska 10

Legal Framework Canada

• Interplay: – Education, health regulation and licensure is in the purview of the Provinces and Territories with some financial and political influence from the federal government.

– Where a provincial statute does not exist, the Federal Charter takes precedence.

– A provincial/territorial statute cannot override the Federal Charter unless such provision was agreed upon at the time agreement was struck (notwithstanding clause).

CLEAR 2008 Annual Conference Anchorage, Alaska 11

Legal Framework Canada

• Who is protected under Canadian law?

• Broad and purposive approach to the interpretation of anti-discrimination statutes (mental or physical disability, ailment, social construct, perceived limitation) CLEAR 2008 Annual Conference Anchorage, Alaska 12

Legal Framework USA

• Americans with Disabilities Act – Prohibits discrimination – Also imposes duty to provide accommodations – Applies to governmental and private entities CLEAR 2008 Annual Conference Anchorage, Alaska 13

Legal Framework USA

• ADA Title III requires: – Facilities that are accessible in equally convenient locations.

– Auxiliary aids and services that may include taped examinations, Braille or large print examinations, qualified readers, or transcribers.

– Modifications that may include changes in testing time or adaptation of the manner in which the examination is given.

CLEAR 2008 Annual Conference Anchorage, Alaska 14

Legal Framework USA

• Accommodation is not required under the ADA if providing it would fundamentally alter the measurement of the skill or knowledge the examination is intended to test.

CLEAR 2008 Annual Conference Anchorage, Alaska 15

Legal Framework USA

• Who is protected under the ADA?

– Generally, a person with a disability – Disability defined as a physical or mental impairment that substantially limits one or more of the major life activities of an individual – Impairment does not include temporary impairments, e.g., broken limb, normal pregnancy CLEAR 2008 Annual Conference Anchorage, Alaska 16

Legal Framework USA

• Substantial limitation under the ADA: – “The individual’s important life activities are restricted as to the condition, manner, or duration under which they can be performed in comparison to most people.” [Restricted = more than moderately] – Major life activities are basic activities that the average person in the general population can perform with little or no difficulty CLEAR 2008 Annual Conference Anchorage, Alaska 17

Legal Frameworks Summary

• ADA vs Charter of Rights • Who is covered: ADA rules narrowly define who is covered; Charter principles allow “everyone” to be covered • Appeal process: ADA appeals through legal system hierarchy; Charter appeals to Human Rights Commission (in Canada can also appeal through legal system) CLEAR 2008 Annual Conference Anchorage, Alaska 18

Societal Framework Across the Borders

• Candidates have access to and participation in accommodation and due process • Sense and perceptions of “fairness” • Expectations established in educational and other contexts • Activities involving protection of the public • Practice areas often more focused than certifying and licensing examinations • Avoidance of being a career “road block” CLEAR 2008 Annual Conference Anchorage, Alaska 19

General Procedures Across the Borders

• Notice: Candidates request in advance – Accident or unforeseen event—inform as soon as possible • Documentation: – Specific diagnosis – Description of limitations and context – Specific accommodation(s) for consideration – Current documentation, qualified professional • Decision: Case-by-case CLEAR 2008 Annual Conference Anchorage, Alaska 20

Weaving the Principles into Application

• Hypotheticals – Nature of the alleged impairment – Nature of the requested accommodation –

Panelists and Audience: How do the legal, societal, and measurement frameworks impact your decision-making

?

CLEAR 2008 Annual Conference Anchorage, Alaska 21

The Future?

• U.N. Convention on the Rights of Persons with Disabilities • Proposed changes to the ADA (ADAMA, DOJ regulations and comments) • New kinds of assessment tools • More assessments over one’s lifetime?

• Global / regional trends / societal shifts?

CLEAR 2008 Annual Conference Anchorage, Alaska 22

For More Information

• Americans with Disabilities Act – http://www.ada.gov

• ADA Amendments Act of 2008 – H.R. 3195 – S. 3406 • http://thomas.loc.gov/home/c110query.html

CLEAR 2008 Annual Conference Anchorage, Alaska 23

For More Information

• Canadian Charter of Rights and Freedoms – http://laws.justice.gc.ca/en/charter • UN Convention – www.un.org/disabilities CLEAR 2008 Annual Conference Anchorage, Alaska 24

For More Information

• Medical Council of Canada – http://www.mcc.ca/en/exams/ee/special_needs.shtml

– – http://www.mcc.ca/en/exams/qe1/special_needs.shtml

http://www.mcc.ca/en/exams/apply/special_needs.shtml

• National Board of Medical Examiners (USMLE information) – http://www.usmle.org/Test_Accommodations/test_accomm odations.html

• Royal College of Physicians and Surgeons of Canada – http://www.rcpsc.medical.org

CLEAR 2008 Annual Conference Anchorage, Alaska 25

Resources

• Standards for Educational and Psychological Testing (1999) – American Educational Research Association (AERA) – American Psychological Association (APA) – National Council on Measurement in Education (NCME) CLEAR 2008 Annual Conference Anchorage, Alaska 26

Speaker Contact Information

Janet D. Carson, JD Counsel, United States Medical Licensing Examination Program www.usmle.org

[email protected]

Consultant to Testing and Certifying Entities [email protected]

CLEAR 2008 Annual Conference Anchorage, Alaska 27

Speaker Contact Information

Deborah Danoff, MD, FRCPC Fellow Royal College of Physicians and Surgeons of Canada (RCPSC) [email protected]

CLEAR 2008 Annual Conference Anchorage, Alaska 28

Speaker Contact Information

Catherine Farmer, PsyD Manager Disability Services ADA Compliance Officer National Board of Medical Examiners 3750 Market Street Philadelphia, PA 19104 [email protected]

CLEAR 2008 Annual Conference Anchorage, Alaska 29

Speaker Contact Information

Robert S. Lee Director, Evaluation Bureau Medical Council of Canada www.mcc.ca

[email protected]

CLEAR 2008 Annual Conference Anchorage, Alaska 30