Transcript AIR Forum - Bethune-Cookman University

Institutional Effectiveness
& Assessment @ B-CU
Dr. Helena Mariella-Walrond
VP of Institutional Effectiveness
Cory A. Potter
Director of Assessment
Administrative Support Services
Academic and Student Support Services
Building Capacity
Capacity: Our ability to seek solutions to our challenges so that we
can achieve our goals and objectives.
Capacity building: Should provide faculty and staff the skills and
tools needed (state proper outcomes, identify assessments – direct
and indirect, collection of data, analysis of data, use of data to drive
improvement) and strengthen our ability to work together..
Creating the common
vision through multiple
perspectives that matches
Dr. Jackson’s vision
Framework for Institutional
Effectiveness
• Purpose
• Goals
• Components of IE
• Accountability
Institutional Effectiveness
Purpose:
To systematically engage in a process designed
to monitor and document the degree to which
the institution is progressing together towards
accomplishing its Mission.
Annual IE Report
Accreditation Reports
Annual Assessment Report
IE Goals
1. Inform the strategic planning process
2. Promote effective allocation of resources (human & fiscal)
3. Engage faculty and staff in the continuous improvement of
the institution’s programs and services
4. Provide accountability to stakeholders
Components of IE
Planning (Mission-driven –
plan and review mission
simultaneously, integrated
into the annual planning
and budgeting cycle)
Budgeting (driven by
Mission and Division plans)
Assessment
• Academic Program
Assessment
• Administrative &
Educational Support
Services Assessment
• General Education
Assessment
• University Assessment
Feedback Mechanism /
Accountability Annual
Reports
(University, Division, Unit)
1. Annual IE report
2. Annual assessment
report
3. Accreditation reports
How is the IE process organized and coordinated? (Institutional Research, Fiscal, Assessment,
IE)
How is the IE process supported? (leadership support, IT, faculty and staff development,
policies and procedures manual, reporting templates)
Accountability
1. Who is the audience?
2. What are the products? (IE report, Assessment report,
Accreditation reports)
3. How do we most effectively communicate (convey information) to
stakeholders?
4. What is the process for getting a good product?
The Art of Explanation
“The single biggest problem in
communication is the illusion that it’s
taken place.”
G. Bernard Shaw
Source: Keynote Speech AIR Forum Orlando, May 2014, Lee LeFever, Common Craft
A Culture of Evidence
“Accrediting agencies – both at the institutional level and the
programmatic level – are now operating in a ‘culture of
evidence’ that requires institutions to qualitatively and
quantitatively demonstrate that they are meeting student
learning goals and effectively marshalling human and fiscal
resources toward that end.”
Michael Middaugh
Author of Planning and Assessment in Higher Education
Outcome Statements
Assessment
Transparency
 Specific
Use of Results
 Targeted to audience
 Provides examples
 Describes improvement
 Defines next steps
Assessment
Plans
 Descriptive
 Defines Measures
 Easy to download
COMMON TO ALL
 Clearly worded
 Prominently posted
 Updated regularly
 Receptive to feedback
Evidence of
Efficiency & Quality
 Interpreted results
 Contextualized to B-CU
 Presented graphically
 Disseminated
Assessment
Resources
 Easy to download
Current Assessment
Activities
 Defines activities
Source: National Institute for
Learning Outcomes Assessment (NILOA)
Assessment @ B-CU
Assessment is an integral part of the annual planning activities at B-CU, as
it determines whether or not the objectives of education are being met.
The main goals in the assessment of Administrative Support Services and
Academic & Student Support Services are to:
1. document what support of learning is taking place and,
2. use the results of assessment activities to improve the student learning
environment.
Annual Planning & Assessment Report
Guide
Template
Annual Planning and Budgeting Cycle
Collect, aggregate,
disaggregate data for
Academic Year.
Share data with Staff in
Pre-Planning for
Academic Year.
Staff analyzes data
and makes
recommendations for
unit improvement.
Staff reviews and
revises Annual Plans.
Vice-Presidents, Deans,
Department Chairs and Directors
submit Annual Plans for the
upcoming academic year.
B-CU collects assessment data
to enhance outcomes in an
ongoing, staff-driven
continuous improvement cycle
by adhering to an assessment
plan which, in turn, drives the
strategic plan.
Elements of the Report
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Description of the Support Services in the Unit
Mission Statement
Resources Used
Support Specifics
Outcomes
Annual Planning and Assessment Grid
Assessment Plan Methods and Procedures
Analysis and Evaluation
Action Plan
Rubric for Assessing the Annual Assessment Report
Description of the Administrative /
Academic and Student Support
Services
In this section, provide a brief overview of your unit. Be sure
to describe the functions, structure, and purpose of your
support services.
Mission
Remember that the mission of the unit must be aligned with
the Mission of the University and demonstrate that it
supports the Mission of the University.
The Mission of Bethune-Cookman University is to develop global
leaders committed to service, life-long learning and diversity by
providing a faith-based environment of academic excellence and
transformative experiences.
Mission
Outcomes
• List the expected outcomes for your area (administrative or
academic and student support services) as related to Division Key
Performance Indicators (KPIs) and Bethune-Cookman University’s
Strategic Goals.
• The professional staff in each unit are to work together to
establish a limited number of overarching outcomes for the unit
as a whole.
Outcomes
Administrative / Academic & Student Support areas typically
have outcomes that focus on the critical functions, services,
and processes that impact the student learning environment
with an emphasis on satisfaction, efficiency and quality.
S.M.A.R.T. Outcomes
Specific - The outcome is clear among all choices and not bundled with multiple
outcomes
Measurable - The outcomes contains quantifiable terms
Attainable - The outcome is reasonable give the area's resources and influence
Relevant - The outcome must be relevant to the area's mission, responsibilities,
and all people affiliated with the area
Time-bound - The period of time for accomplishing goal is reasonable
Types of Assessments
Satisfaction
• General (institutional)
• Specific (satisfaction with particular area)
Direct Measures
• Volume of Activity (# of persons served)
• Level of Efficiency (average response time)
• Quality average (# of errors)
External Validation
• Audits (financial, IT)
• Public Health Inspector
• Fire Marshall
Outcome Examples
Outcome 1: The Office of the registrar will process transcripts
in a timely manner.
ELOA:
90% of transcript requests will be processed within 2
business days.
Measure/Tool: Log of transcript requests (monthly)
Outcome 2: The Office of the registrar will be in compliance with
FERPA regulations.
ELOA:
100% of employees will answer 90% of the questions
correctly.
Measure/Tool: Survey
Resources Used
Briefly describe resources, including personnel, used to accomplish
the outcomes. It may be beneficial to review expenditures during the
fiscal year (July 1 – June 30) to categorize where resources were
allocated within your unit.
Areas to address in your brief description:
• Number of personnel utilized to accomplish each outcome
• Categories of expenditures that consumed a substantial portion
of your budget (with computations and details when available)
• Physical space/facilities utilized
Support Specifics
Teaching - Briefly describe how your unit supports the teaching
and learning enterprise.
Research - Briefly explain how your unit engages in activities
related to research and how your unit supports research at the
University.
Community and Public Service - Briefly state any unit goals
related to community and public service and how your unit supports
service at the University.
Annual Planning & Assessment Grid
Purposes:
• Show the summation and results of data analysis
• Tie the assessment plan to any identified, targeted
strategic goals
• Reveal the area’s expected levels of achievement
• Report if the targeted level of achievement was met
• Explain how your area plans on using the results in terms
of actions
Annual Planning & Assessment Grid
Annual Planning and Assessment (Previous Academic Year)
Mission Statement:
Targeted Strategic Goal(s):
Targeted Division Key Performance Indicators:
Targeted Institutional Student Learning Outcome(s):
Outcomes
Expected Level
of Achievement
Measure/Tool
Results
Use of Results (to
include analysis and
Action Plan)
Met
Not
Met
Annual Planning & Assessment Grid
Annual Planning and Assessment (Current Academic Year)
Mission Statement:
Targeted Strategic Goal(s):
Targeted Division Key Performance Indicators:
Institutional Student Learning Outcome(s) addressed:
Outcomes
Expected Level of Achievement Measure/Tool
Assessment Plan Methods & Procedures
Describe the methods and procedures used to assess the area. Be sure to use the
questions below to thoroughly cover methods and procedures involved in your
assessment plan.
1. Who was/will be involved in the process of reviewing and revising the
assessments?
2. What is the difference between the previous program assessments and the
revised assessments?
3. How will the assessments be collected?
4. Who will collect the assessments?
5. When will the assessments be collected?
6. Who will aggregate and disaggregate the data?
7. Who will analyze the data?
8. How are / will data (be) used to drive improvement?
9. How do you want to share the information with stakeholders?
Analysis and Evaluation
Write out a well-reasoned description of your conclusions
based on the data by explaining the significance and
impact of the results.
Action Plan
Briefly summarize the actions for improvement. Describe
what decisions need to be made, next steps that will be
taken during the following academic year, and explain how
the results are being used to inform unit changes.