Transcript Protocol for the Conduct of Joint Visits WASC & ATS

Joint ATS-WASC Accreditation
Reviews
Jerry McCarthy, ATS
Teri Cannon, WASC
Value of ATS-WASC Reviews
• Enriches the experience for the institution
• Brings more points of view to the
evaluation of the institution
• Reduces duplicative work of preparing
reports for and conducting two reviews
2
General Responsibilities
• Sharing information, reports, and actions on
institutions
• Providing orientation to participating institutions
and teams
• Coordinating actions
• Staffing teams collaboratively
• Planning visits collaboratively
• Conducting visits and preparing reports together
3
Protocol for the Conduct
of Joint Visits
WASC & ATS
General Principles of the Joint
Protocol
• To facilitate the process of accreditation for
the member institutions
• To guide the process of joint visits
involving both accrediting agencies
5
Responsibilities of the
WASC and ATS Staff
• Coordination of WASC Institutional Protocols
• Orientation of ATS/WASC Chairs
• Coordination of team recommendations and respective
Commission actions
• Staffing: Usually 3-4 WASC visitors and 2 ATS visitors/
possibility of joint chair
• Two visits: Capacity and Preparatory Review (CPR) and
Educational Effectiveness Review (EER)
• Normally most CPR members also serve, if possible, on
EER team
6
Responsibilities of the
WASC and ATS Staff
• Sharing of Documents:
 No restrictions on sharing of accreditation-related information
• Logistics of Visit Planning:
 WASC/ATS staffs consult on team composition 12 months
prior to visits
 WASC/ATS staffs consult 20-24 weeks prior to coordinate
planning
 WASC staff consult with school on team dates and
coordinate with ATS staff
 Materials for visits sent to each agency 12 weeks
prior to team members
7
Responsibilities of the
Institutions
• Distribution of Institutional Presentations:
 Submitted to WASC and ATS (Proposals, CPR Review, EER
Review)
• Distribution of WASC Institutional Proposal to ATS after approval by
WASC
• CPR/EER Reports: 1 paper copy and 1 electronic copy to
ATS/WASC and 1 copy to each team member 12 weeks prior to visit
• Reports for Special/Focused Visits: 4 copies to each agency + 1 to
each team member 8 weeks prior
• Interim Reports and Progress Reports: 4 copies to agency requiring
report + 1 copy to partner agency
8
Responsibilities of
Team Evaluators
• Team Co-Chairs (one from each agency)
• Role of ATS Co-Chair: As prescribed in ATS Handbook of
Accreditation
• Role of WASC Co-Chair: Responsible for oversight of
visit, drafting WASC team report with WASC Assistant
Chair, making team recommendation to Commission
9
Development and Construction
of the Team Report
• General Report Coverage: Address standards/criteria of both
agencies
• Report Format: Each agency’s standards are addressed and
appended to the partner agency report
• Report Content: Follows the guidelines of each agency
• Team Recommendations:
 Coordinated as much as possible for both agencies
 Consensus desired with respect to recommendations and
monitoring actions
 Alignment desired as much as possible, but differences may
emerge
10
Development and Construction
of the Team Report
•
Finalizing Team Report
 Commission staffs, team chairs/assistant chairs confer as soon as
possible
 Drafts to be available ordinarily within two weeks of visit
•
Exit Meeting and Confidentiality of Team Recommendations:
 Each agency has different practices and will report findings
appropriately at the exit interview
 ATS and WASC teams are not required to make the same
recommendation
 The Commissions of each agency make the final recommendations that
may or may not follow the recommendation of the visiting team
11
More on the WASC Three-Stage
Review Process
1. Proposal: Identifies priorities, areas of emphasis, and
outcomes. Aligns work with institutional plans and needs.
2. Capacity/Preparatory Review: Focuses on capacity
and readiness for educational effectiveness.
3. Educational Effectiveness: Focuses on results of
assessment of student learning/quality improvement.
12
More on the Scope of the Two Reviews
Capacity and Preparatory
Educational Effectiveness
• Preparatory as: readiness for
• Demonstrating student learning
the Educational Effectiveness
• Demonstrating institutional
Review
learning
• Capacity as: purposes, integrity, • Demonstrating evidence-based
stability, resources, structures,
decision-making
policies, processes
13
Timeline For Review
(CPR/EER)
12 weeks
Institution
mails report
to team and
WASC
Team holds
conference
call; chairs
talk with CEO
2 months
Site visit held
and team report
written
Institution
responds to
errors of fact in
team report
Institution
responds to
final team
report
WASC
Commission
acts at
February or
June meeting
14
More on the Team Pre-Visit Preparation and
Coordination
• Materials review
• Call with the CEO and chairs
• Pre-visit conference call
– To meet team members
– To identify issues and strategy
– To agree on team assignments
– To refine visit schedule
– To identify needed documents
– To plan visit logistics and report
15
WASC Standards and CFRs
• Core Commitments: Capacity and EE
• Standards: Broad, holistic, encompassing
• Criteria for Review: Provide specificity and
meaning
• Guidelines: Ways to demonstrate compliance
with CFRs
16
Team Use of the Standards and CFRs
• Team judgments must be linked to specific
Standards and CFRs
• CFRs must be cited in reports
• Standards and CFRs form the basis for
Commission decisions
• Standards and CFRs provide a context for
continuous quality improvement
17
WASC Standards
at a Glance
STANDARD 1:
Defining Institutional Purposes and
Ensuring Educational Objectives
Institutional Purposes
Integrity
STANDARD 2:
Achieving Educational Objectives
Through Core Functions
Teaching and Learning
Scholarship and Creativity
Support for Student Learning
STANDARD 3:
Developing and Applying Resources and
Organizational Structures to Ensure
Sustainability
Faculty and Staff
Fiscal, Physical, and Information Resources
Organizational Structures and
Decision-Making Processes
STANDARD 4:
Creating an Organization Committed
to Learning and Improvement
Strategic Thinking and Planning
Commitment to Learning and
Improvement
Example: CFR 2.6
SLOs used to evaluate students, graduates achieve
stated levels of attainment
Capacity and Preparatory
 Has the institution defined
expected levels of
attainment for SL?
 Are they embedded in the
standards and measures for
student work?
 What data are collected &
how analyzed?
 How are they measured?
Educational Effectiveness
 What do data show?
 Are data disaggregated and
analyzed?
 Has the institution used data
analysis to make changes
and/or improvements?
 How well are graduates
achieving SLO?
23
What’s New on WASC Visits
in 2008-09
WASC Visit Advisory for 2008-09
• Systematic review and reporting of off-campus sites
and distance education programs
• Analysis of retention and graduation rates
• Team rating of institution on Framework for Evaluating
Educational Effectiveness (pilot)
• Compliance audits for new and sanctioned institutions
25
New WASC Tools for Teams: Rubrics for
Assessment of Student Learning
•
•
•
•
Academic Program Learning Outcomes
Use of Portfolios in Assessing Program
Outcomes
Use of Capstones in Assessing Program
Outcomes
Integration of Student Learning Assessment
into Program Review
26
WASC Expectations about Assessment
of Student Learning
CPR
• Student learning outcomes
set at program and course
level
• SLOs are in syllabi
• Faculty has developed
assessment plans
• Faculty has set
expectations for student
achievement
• Faculty has tools to
measure learning (direct
and indirect; multiple)
EER
• Results of assessment
show extent to which
graduates are meeting
expected levels of
achievement
• Results used to improve
student learning
• Results used to improve
assessment strategies
27
Expected Examples of Evidence of
Assessment
• Retention and graduation data/disaggregated
and analyzed
• Standardized test results/licensing exams
• Faculty assessments
– Grades
– Portfolios, capstones and work samples
• Surveys and standardized interviews
28
Good Practice in Evidence of
Educational Effectiveness
• Direct evidence of student learning and selfreported or indirect evidence
• Evidence related to the intended student
learning outcomes -- validity
• Replicable evidence, representative of the
student population -- reliability
• Multiple measures of student learning
29
Using the WASC EE Framework
• Identify where the institution fits on the
framework for each line
• Use results to determine where the institution is
in its evolution toward being a highly developed
learning organization
• Use language of the framework in the report to
guide the Commission and the institution
30
Using Evidence in Team Reports
•
•
•
•
•
•
Use qualitative and quantitative evidence
Select evidence carefully and purposefully
Connect evidence to an assertion or question
Analyze information; do not just put forth data
Let evidence suggest improvements
Use evidence that speaks to the institution’s themes and
the team's questions
• Address results/findings of assessment, not just process
31