Transcript Commission’s Expectations for Chairs and Evaluators

The Accreditation Team Experience:
Ten Steps for Team Success
Middle States Workshop for New Team Chairs and Evaluators
October 4, 2006
Patricia A. McGuire
President, Trinity (Washington) University
1
WHAT’S IT LIKE
TO SERVE ON A MIDDLE STATES ACCREDITATION TEAM?
…IN FIVE WORDS OR LESS!
• INTENSE
• TEAMWORK
• STIMULATING
• INTELLECTUAL
• COLLEGIAL
2
TEN STEPS TO TEAM SUCCESS
1. Know “Accreditation”
2. Know the Process
3. Be Insightful About
Institutional Needs and
Expectations
4. Know Your
Characteristics!
5. Know Your Role and
Play It Well
6. Analyze Self-Study and
Prepare Field Work
Based on Analysis
7. Interview, Observe,
Review
8. Write a Great Report
9. Know Your MSA
Resources
10. Good Home Training
3
STEP ONE:
KNOW WHAT
“ACCREDITATION”
MEANS
4
“Accreditation”
• Characteristics of Excellence
“Accreditation is the means of self-regulation and peer
review adopted by the educational community.
The accrediting process is intended to strengthen and
sustain the quality and integrity of higher education,
making it worthy of public confidence and minimizing the
scope of external control.
The extent to which each educational institution accepts
and fulfills the responsibilities inherent in the process is a
measure of its concern for freedom and quality in higher
education and its commitment to striving for and
achieving excellence in its endeavors.” (p. iv)
5
STEP TWO:
KNOW HOW
ACCREDITATION
IS DONE
6
PRIOR YEAR:
ACCREDITATION AND PLANNING CONCEPTUAL TIMETABLE
TWO-YEAR PROCESS
PLANNING
ASSESSMENT
DATA COLLECTION
ANALYSIS OF
CHARACTERISTICS
WINTER YR
TWO
AUGUST YEAR ONE
FALL YEAR TWO
SUMMER
YEAR TWO
OUTLINE
STEERING CMTE
DATA COLLECTION
DOCUMENTATION
SELF STUDY
DRAFTED
SELF STUDY
CIRCULATED
FOR COMMENT
AND REVISED
SPRING YR ONE
SPRING
YEAR TWO
SPRING 05
COMMITTEES
DEVELOP
REPORTS
SEPT YR ONE
SELF-STUDY
SUBMITTED
TO MSA
TEAM
VISIT
TEAM
REPORT TO
MSA
STAFF VISIT
FALL YR TWO
PRELIM VISIT
TEAM CHAIR
OCTOBER YR ONE
END YR ONE
DESIGN FOR SS
SUBMITTED
COMMISSION
ACTION
ON
ACCREDITATION
BY END OF YR
ALL COMMITTEE
REPORTS DUE
COMMITTEES
UNDERWAY
AC YR ONE___________________________________________________________AC YR TWO________________________
AUG
SEP
OCT
NOV
DEC
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
_____________________
OCT NOV DEC JAN FEB MAR APR MAY JUNE
7
STEP THREE:
BE INSIGHTFUL
ABOUT INSTITUTIONAL
NEEDS AND
EXPECTATIONS
8
What Do Institutions Expect of Teams and Chairs?
• Respect for Time and Effort in Self-Study; Respect for the
Institution’s expertise about itself
• Collegiality with appropriate professional distance
• Fairness and Integrity
• “Compassionate Rigor”
• Expertise in Applying Standards and Interpreting Institutional
Conditions
• Thorough Preparation and Earnest Interest in the Story of the
Institution
• Consistency with Role Expectations and Conduct of the Visit
9
STEP FOUR:
KNOW YOUR
CHARACTERISTICS!
10
TEAM ANALYSIS OF THE
CHARACTERISTICS OF EXCELLENCE
• ALL TEAM MEMBERS SHOULD READ THE
CHARACTERISTICS THOROUGHLY AND FLAG
ISSUES PARTICULARLY RELEVANT TO THE AREA
EACH MEMBER WILL REVIEW AT THE INSTITUTION
UNDER STUDY
• ANALYZE SELF-STUDY IN LIGHT OF THE
CHARACTERISTICS
• PREPARE INTERVIEW QUESTIONS DRAWN FROM
THIS ANALYSIS
11
STEP FIVE:
KNOW YOUR ROLE
AND PLAY IT WELL!
12
Role of the Team Chair
13
Role of the Team Chair
LEADER
Team and Institution
look to YOU as the embodiment of
the Middle States system and
arbiter of the ethics of peer review
FACILITATOR/COACH
Develop Strategy for the Team Visit
Review with Team the Role of Team Members and
Ground Rules for Visits
Review with Team Characteristics of Evidence
and related accreditation materials
ORGANIZER
Set Dates for Visits
Conduct Preliminary Visit
Manage Communications
Final Report
Review Visit Arrangements
Review Documents
Make Team Assignments
Chair’s Brief
Analyze SS
Oral Exit Interview
Eval. Cmte. Meeting
14
When Team Chairs Go Bad…
 Runaway Team – Chair Loses Control of Team Direction
 Bully on the Team – Chair Fails to Confront/Resolve
 Institutional Hide ‘n’ Seek – Chair Ignores Reality
 Scrooge – Chair Keeps Team Working Past Reasonable
Hour
 Leviathan – Chair As Control Freak
 Terminator – Chair Bullies/Threatens Institution
15
Role of An Evaluator
•
You are Invited because of your expertise, but remember the “peer” in “peer
review”
•
Expected to be thoroughly prepared in advance of the visit:
– KNOW Characteristics thoroughly and other MSACHE documents
– ANALYZE Self-Study (see Handbook pp. 24-26)
– BE READY to engage the interview process appropriately
•
Cooperate with Team Chair in Site Visit Assignments, and maintain collegiality
with team members
•
Prepare each interview with care, and record notes carefully
•
INTERVIEW, do not monologue!
(If you hear yourself talking for more than 3 minutes, stop! Ask questions, don’t
make speeches.)
•
Refrain from making statements comparing this institution to yours
•
Avoid Prescriptive Pronouncements
•
Make Every Effort to Understand the Institutional Mission, Goals and Strategies
•
Make sure that evaluative comments are rooted in Characteristics
16
When Good Teams Go Bad…
•
The Inspector General Persona and ‘Gotcha!’ Games
•
“Well, at MY Institution here’s how it’s done….”
•
“I am the Great Oz and I know what you need to be doing…”
•
“Let’s slam them on this …”
•
“Could You Believe That President??”
•
“I’ve arranged a small meeting with a few disaffected faculty to hear
what’s REALLY going on at this place.”
•
“Forget the Characteristics, this place is a mess and we need to fix it
or shut it down.”
•
“They’re lying.”
17
STEP SIX:
ANALYZE THE SELF-STUDY
AND PREPARE FIELD WORK
BASED ON THIS ANALYSIS
18
PRIOR TO THE VISIT:
TEAM ASSIGNMENTS
•
Chair: Establish a pre-visit work plan for the team to guide members
in preparation
– Identify key areas for each team member to review, but stress that all team
members must read the entire self-study and participate in the full review
– Ask team members to prepare written outline of the issues and questions
they are likely to review on the campus visit
– Review the team outlines and questions to be sure that all team members
understand their roles and are heading in the right direction prior to the
visit
•
Team:
– Study Characteristics and Designs to create mental framework for analysis
– Read and Analyze Self-Study
– Prepare written outline of issues to probe during campus visit and likely
questions to ask
– Identify relationship to Characteristics for issues
– Communicate with team chair about any questions/concerns regarding
role, issues, preparation
19
Sample Matrix for Team Assignments
Area of Inquiry
Questions
Self-Study
Sections
Characteristics
Team Member #1
Outcomes
Assessment
Chapters 2 and 3
Standards 8 and 14
Team Member #2
Governance and
Administration
Chapters 1 and 5
Standards 4 and 5
Team Member #3
Planning and
Resources
Chapter 1,4,5,6,7
Standards 1,2,3,7
Team Member #4
Student Services
Chapter 6,7
Standards 5,7,9,11
Team Member #5
Admissions and
Enrollment Mgmt
Chapters 1,2,3,6
Standards 6,8,9,11
Team Member #6
General Education
and Undergrad
Prgms
Chapters 2,3,4
Standards
11,12,13,14
Team Member #7
Faculty and
Graduate Programs
Chapters 2,3,4,7
Standards
10,11,12,13,14
20
Understanding the Self-Study
• Designs: “The self-study process enables the
campus community to examine the institution’s
strengths and challenges relative to
accreditation standards. The process also
enables each institution to develop solutions to
problems, and to identify opportunities for
growth and development. The aim is to
understand, evaluate and improve, not simply
to describe or defend.”
(emphasis added) (p. 3)
21
Understanding the Self-Study
• Designs:
Self-Study =
• Continuous Process in Planning Cycle
• Accountability to Constituencies
• Measurements of Goal Achievements
• Meet Accreditation Standards
• Fulfill Institutional Mission
22
Problem Set: Self-Study Meltdowns and
Team Response Strategies
PROBLEM
ACTION STEP
•
Self-Study lacks the basics
•
Chair discuss with MSA staff
•
SS is disorganized and badly written
•
Chair Discuss with MSA staff and Institution
prior to the visit, correct as possible in advance
•
Gaps in Data
•
Be prepared to identify what data sets are
necessary for the team visit to be successful
and request the data
•
Too superficial/too self-congratulatory
without analysis/too descriptive without
analysis
•
Team must probe beneath the writing to get the
insightful analysis that the document may lack.
Need to distinguish between writing style
preferences of team members and real
problems with the self-study
•
•
Overly self-critical where data and
information suggests otherwise
Chair and team discuss gaps between reality
and statements in the report and reasons
•
•
Political tensions on campus come through
in report and create analytical skews in the
narrative and data
Team cannot get drawn into campus political
issues; need to pierce through the tension to
get at facts and analysis of performance and
outcomes
•
Self-study is not truthful in part, fact only
discovered through the team visit
•
Chair must discuss issues with the institutional
president as soon as the facts are in hand;
consult with MSA staff if necessary
•
Team learns during visit that the self-study
was largely written by administration with
little campus engagement
•
Problems of collegiality in the self-study
23
process can be addressed in the team report.
Problem Set: Self-Study Meltdowns and
Team Response Strategies
PROBLEM
•
Institution plans a selected topics
self-study but needs a
comprehensive review
ACTION STEP
•
•
As Designs makes clear, this kind of problem
should be addressed in the early stages of selfstudy preparation, see pp. 10-11.
–
Institution should submit proposal for selected
topics 24 months in advance
–
Commission staff evaluate proposal and
compliance with all accreditation standards
–
(Option 1) Chair’s Preliminary Visit should
review the plan and the documentation
supporting compliance with standards NOT
addressed in the selected topics self-study
• Chair and “Generalist Evaluator” write a
report, see p. 11
–
(Option 2) Chair and Generalist Evaluator
review the comprehensive compliance
documentation during the regular visit, with
team focused on special topics, and
Chair/Evaluator write report on compliance
with other standards
In all events, the Chair should consult as early as
possible with the MSA staff if there are questions,
differences of opinion, or new data on this topic
24
Problem Set: Self-Study Meltdowns and
Team Response Strategies
PROBLEM
ACTION STEP
• Institution chose a selected
• Chair should seek guidance
topics model of self-study and
from MSA staff. Might be
the report seems fine, but
appropriate to address the
during the team visit it becomes
problem in the team report.
clear that there are major
Campus visit should ensue on
problems at the institution that
the basis of the report given,
a focused self-study ignored,
and that report should be
whereas a comprehensive selfaddressed, but the gaps
study would have addressed
should also be illuminated and
the problems
addressed for further action.
• Team member(s) ignore
• Chair is in charge of the team
selected topics and insist on
and responsible to ensure that
pursuing other avenues of
the team stays focused
inquiry
25
STEP SEVEN:
INTERVIEW, OBSERVE,
REVIEW: MAKE GOOD
USE OF TIME ON CAMPUS
26
MAXIMIZING TIME ON THE CAMPUS VISIT
INTERVIEWS, OBSERVATIONS, REVIEWS
• Chair: Prepare interview list and schedule well in advance in
consultation with the institution and the team; changes can and will
occur during the visit, but have a game plan in advance and change
strategically
• Team: All members should have a prepared set of issues and
interview questions for the appointments
• Observe: An important part of the campus visit is observation of the
environment, facilities, availability of services, general conditions;
• Review: A large part of time on campus is also spent reviewing
source documents that support assertions in the self-study; the team
should come prepared with a list of topics that bear additional
research time when on campus
27
How to Conduct an Interview
During a Team Visit
•
Preparation: Interviews test and illuminate the self-study. Good interviews start with
thorough knowledge of the self-study and institutional materials. Chair makes interview
assignments according to the expertise of the team, but team members also identify
potential interviews, and the final schedule reflects dialogue between team chair and
members about assignments.
•
Develop Interview Objectives and Questions: For each interview, the team member should
know the objectives of the interview, develop clear questions, and be sure those questions
relate to the self-study and accreditation criteria.
•
Seek Dialogue, NOT Cross-Examination: Questions should be designed to evoke analysis
and dialogue. This is not a cross-examination or interrogation, and questions that make
the interviewee feel put on the spot are generally not helpful.
•
Set the Context and Explain the Objectives: At the outset of the interview, be sure to get
the names and positions of everyone in the room. Greet the individual or group in a
friendly way, and remind all of the purpose of accreditation. Frame the objectives of the
interview and general topics under review in this session. Set the time limit as well.
•
Avoid Monologue and Prescription: Team members must not use interview times to lecture
the interviewee, monologue about your opinions, or offer prescriptions about what the
institution should do. You are there to learn, test assertions constructively and push
analysis by institutional participants.
•
Take Careful Notes: Interviews form an important part of the background for the team
report, and often a statement in an interview can shape the team’s understanding of some
dimension of the self-study. Notes taken during the interview facilitate accuracy and
28
completeness.
STEP EIGHT:
WRITING A GREAT
TEAM REPORT
29
TEMPLATE FOR TEAM REPORTS
(See pp. 32-38 “Chairing and Hosting”)
I.
Context and Nature of the Visit
a) Institutional Overview
b) Scope of Institution
c) Self-Study Process
II.
III.
Affirmation of Continued Compliance With Eligibility Requirements
Compliance with Federal/State/Other Regulatory/Other Accreditation
IV.
Compliance with Accreditation Standards
For each standard (or by major heading in the self-study, with standards identified) the
report should follow this outline:
a) Summary of Evidence and Findings
b) Significant accomplishments/progress/exemplary practices
c) Suggestions for Improvement
d) Recommendations
e) Requirements
V.
Summary of Team Recommendations and Requirements
Standard
Recommendation
Requirement
30
Suggestions, Recommendations, Requirements:
What’s the Difference?
•
SUGGESTIONS
–
•
•
may reflect ideas for improvement in performance within accreditation standards based on the professional
experience of team members or the guidance of CHE publications
RECOMMENDATIONS
–
identify areas where the institution needs improvement in order to continue to meet accreditation standards
–
must state specifically how the institution might be at risk of non-compliance;
–
team report must be specific about the standard and how the institution needs to improve;
–
note that a recommendation warrants Commission follow-up with a progress letter or monitoring report
REQUIREMENTS
–
address specific points where the institution fails to meet the accreditation standards
–
and set forth steps the institution should take to comply;
–
specific citation to the standard, required element and failure to meet the requirement;
–
note that a requirement necessitates Commission action of postponement, warning,
probation or show cause
31
STEP NINE:
KNOW YOUR MSA
RESOURCES!
32
Resources for Team Support
•
•
•
•
All of the Middle States Documents
Middle States Staff
Experienced Evaluators
Team Chair
33
STEP TEN:
GOOD HOME TRAINING
34
Good Home Training
• Things that shouldn’t need to be said, but
sometimes have to be:
– Leave your biases at home.
– If you have a conflict of interest, reveal it before the visit.
– Respect the privacy and reputation of the institution. Do not ‘share’
what you have learned about the institution when you get home.
– Keep your sense of humor apparent, but mild. This is not the time to try
out for Second City.
– Do not be dour, or angry, or difficult, this is a TEAM and we all need to
get along.
– Bar bills should not be necessary at all, but pay for your drinks
personally if you choose to have a drink.
– Same for movies and personal phone bills.
– There are no job opportunities for you at this school.
– If you hate the experience, do not volunteer for this gig again.
– If you love the experience definitely sign-up again.
– Be open to learning new things you didn’t anticipate.
35
– Get to bed at a reasonable time each night.