Transcript Assessment Plan Workshop SUNY Oneonta March 22, 2012 Patty Francis Associate Provost for Institutional Assessment & Effectiveness.

Assessment Plan Workshop
SUNY Oneonta
March 22, 2012
Patty Francis
Associate Provost for
Institutional Assessment &
Effectiveness

Development of College’s Action Plan for Planning
and Assessment (Spring 2008)
◦ Endorsement by College Senate (12/2008)
◦ Approval by President’s Cabinet (Spring 2009)





Formation of Institutional Assessment Committee
(IAC) (Spring 2009)
Development of assessment guidelines by IAC and
approval by President’s Cabinet (11/2009)
Distribution of guidelines in 12/2009, with first
plans due June 1, 2010
Revision of guidelines by IAC in June 2011
Submission of first assessment reports in June 2011




All unit goals and objectives (i.e., Step 1)
Results of activity mapping exercise (i.e., Step
2) {optional}
Action plan summarizing those objectives
units intend to assess (and how) for the next
year (i.e., Step 3)
Brief description of how unit will “close the
loop” once assessment results are attained
(i.e., Step 4)





Establishing congruence among institutional
mission and goals, programmatic and unit
objectives, unit activities, and assessments
Linking goals and objectives to outcomes
through action plan
Assessment as an ongoing, iterative process
Using a variety of meaningful measures,
both quantitative and qualitative, in search
of convergence
Using existing data sources as much as
possible
1.
2.
3.
4.
Setting goals and objectives: “What you
aspire to do and what you do”
Objective mapping: “How you do what you
say you do”
Assessment: “How you know you are doing
what you say you do”
“Closing the loop”: “What you do next based
on results”

Done correctly, assessment:
◦ Serves to align mission, goals, objectives, and
assessments across all levels of the institution
◦ Initiates a “never-ending” dialogue among staff
members regarding programmatic priorities,
objectives and effectiveness
◦ Offers multiple, rich opportunities for professional
interaction and development
◦ Provides (mostly) affirming data in support of
existing programs and services
◦ Provides a systematic, focused direction for change
and future activities
“What are you
attempting to do?”

Mission statements:
◦ Summarize the unit’s status and major
functions and objectives at present
◦ Should be congruent with existing higherlevel mission(s) (e.g., at the
institutional/divisional levels)
◦ Are intended primarily for internal
stakeholders
◦ Are optional in IAC guidelines but strongly
encouraged
“The Library’s mission is to provide
comprehensive resources and services in
support of the research, teaching, and
learning needs of the University community.”
“To serve Creighton University and the community
by exploring, designing, supporting and facilitating
learning, teaching and research opportunities
through the effective and creative use of technology
and media in accordance with the mission of
Creighton University.”


Goals ->Objectives ->Outcomes = More General
(and Less Measurable)->More Specific (and More
Measurable)
Goals:
◦ Statements about general intentions/purposes that are
broad and more long-range in scope and not directly
measurable
◦ May come directly out of unit mission statement
◦ Usually developed at programmatic or divisional level and
often are in the form of a “process” statement (i.e., begin
with verbs like “establish,” “provide,” “enhance”)

Objectives:
◦ More specific than goals
◦ Typically there are multiple objectives for each
goal
◦ Usually developed at the unit level to reflect
“upper-level” goals

Outcomes:
◦ Very specific statements that “translate” into
assessable measures
◦ Process- vs. results-oriented statements
◦ Two kinds, depending on assessment stage
 “Expected outcome” refers to anticipated results of
assessment – should include criterion to be used in
determining success
 “Actual outcome” refers to actual results of assessment
Goal
Enhance student learning through civic, professional
and research engagement.
Objective
Students will learn to be responsible global citizens.
Expected Outcome
80% of students will participate in a universityapproved community service activity with an intercultural emphasis.
Goal
Develop and implement processes that deliver value
to the institution and campus community.
Objective
To identify and carry out strategies that result in the
strategic allocation of resources.
Expected Outcome
During 2007-08, the total funds expended on
institutional priorities will increase by 20%.
Goal
Enhance public service and image throughout the
region.
Objective
Promote and facilitate faculty involvement in the
local public schools.
Expected Outcome
At least 40% of full-time faculty will participate in
activities sponsored by schools in the community.
Goal
Create a culture of communication that ensures
positive, supportive recognition of the school
throughout all relevant constituencies.
Objective
Establish a research-based plan that defines
university marketing and communications needs.
Expected Outcome
Plan is developed and approved by the President’s
Cabinet.




Examine and review stated unit objectives for all
constituencies served
Elicit and discuss staff members' perceptions of
unit objectives
Analyze and compare unit objectives with
college/divisional Mission Statement and planning
documents, those at comparable institutions, and
criteria/standards of certification agencies or
national associations as appropriate
Develop or revise unit Mission Statement that
reflects objectives and that staff members
understand, agree with, and support through their
actions







Institutional effectiveness performance
indicators
Documentation of all services and programs
offered
Tracking of use of services (and by whom)
Constituent satisfaction with
services/programs
Direct impact of services/programs on
constituents (including student learning if
appropriate)
Adequate resources to support planned
activities
Consultation with other campus units (when
joint activities are required)
Don’t try to do everything at
once – typically, 5-7
objectives for each
assessment plan are plenty!
“How are you attempting
to meet your objectives?”



This step is basically an “audit” of existing
programs, services, and resources, in order to
determine if unit goals and objectives are
realistic
Involves a comprehensive review, being sure to
“map” objectives to existing activities
Possible results
◦ Redundancy – same objective being addressed by
multiple programs
◦ Gaps – specific objective not being addressed by any
program; could lead to:
 Identifying new\reallocating existing resources
 Abandoning the objective, at least for the present
Objective/
Unit
Alumni
Affairs
Fundraising
Grants
Community
Relations
Objective #1
5
5
5
5
Objective #2
5
5
5
5
Objective #3
2
3
3
5
Objective #4
2
1
1
2
Objective #5
1
2
1
1
Note. Extent to which unit addresses objective is ranked on a scale of 1-5, with higher scores
indicating more emphasis.



Determine the extent to which unit objectives
are reflected in unit activities, with the aim of
assuring that all objectives can be met
through those activities
Review and analyze coherence and interrelatedness of unit activities in order to
assure the most efficient use of resources
Identify resources that are necessary to
support activities aimed at realizing all unit
objectives and make sure those resources are
available
“How successfully are you
meeting your objectives?”


To collect data that clearly demonstrate
whether or not the unit is meeting its
objectives
Requires a priori identification of appropriate
strategies\measures for each objective and
statement of expected outcome (i.e., what is
the unit aiming for?)
◦
◦
◦
◦
Utilization of services/programs (i.e., quantity)
Performance measures (i.e., quality)
Satisfaction surveys
Comparisons with other units (i.e., benchmarking)





Develop a detailed action plan linking objectives to outcomes,
specifying strategies/actions intended to accomplish each
objective and, for each action, a timeline, person/persons
responsible, resources required, measures to be used, expected
outcomes, and actual outcomes once assessments have been
conducted
Use a wide variety of information sources, including existing data
as much as possible
Develop and administer satisfaction surveys to internal and
external constituent groups
Establish criteria for unit effectiveness through comparisons with
information provided by similar units at other institutions or
other relevant sources (e.g., certification agencies, national
organizations)
Units whose functions are evaluated through SUNY-wide
measures (e.g., the Student Opinion Survey, the National Survey
of Student Engagement) should include these measures as
performance indicators in their assessment plan
Goal
To provide high quality and comprehensive television
broadcasting experiences to our student staff
members and interns.
Objectives and Measures
1.
2.
3.
4.
To expand our formal internship program.

Number of students enrolled in internships

Equipment upgrades that meet standards of benchmark
institutions
To enhance technical aspects of the program’s facilities.
To increase partnerships with academic programs, high
schools, and community organizations.

Number of academic programs with co-curricular requirements

Percentage of students who meet minimal competency levels
To establish minimal competencies for each of the
program’s student learning outcomes.
“How can you improve
what you’re doing?”



To feed back the information obtained from
the outcomes assessment into the unit’s
objectives and activities
Will lead to the continuation of successful
practices and to the discontinuation\revision
of unsuccessful practices
May also lead to new objectives





Reach overall conclusions regarding unit
effectiveness, based on comparisons between
expected and actual outcomes and with special
consideration given to the different constituent
groups served by the unit
Identify major strengths and weaknesses of unit
operations revealed through assessment
Make major recommendations for changes in unit
activities based on assessment outcomes
Analyze relationship between available resources and
unit/program success
Revise objectives and identify new outcome measures
as appropriate for next assessment round
The Action Plan



Goals and Objectives for Unit
Strategies or Actions Intended to Accomplish
Goals and Objectives
For Each Action:
◦
◦
◦
◦
◦
Timeline
Person/Persons Responsible
Resources Required
Expected Outcomes
Actual Outcomes (Once Actions are Complete)
Goal: Plan and deliver integrated information services to enable members of the campus community to access information
when and where they need it.
Objective
1.
2.
Actions/Strategies
Target
Completion
Date
Resources
Required
Person(s)
Responsible
Expected Outcome
Develop and
implement a
strong foundation
of IT
infrastructure
and sound fiscal
planning.
A.
Build life-cycle
replacement funding
into planning at every
level of investment in
IT.
A.
01/01/10
A.
$150,000
annually
A.
AVPIT,
VPFM
A.
20% of all IT replacement
needs (except laptops) are
funded every 5 years
B.
Budget standard
amount each year per
FTE to support
replacement of
faculty/staff desktop
computers.
B.
01/01/10
B.
$175,000
annually
B.
AVPIT,
VPFM
B.
1/3 of faculty/staff
desktops are replaced
every 3 years
Provide faculty,
staff, and
students with
reliable access to
computing,
information, and
network services,
both on- and offcampus.
A.
Ensure that core
production services
are available and
accessible during
scheduled hours of
operation.
A.
Ongoing
A.
$150,000
annually
A.
CIO
A.
Average daily availability
exceeds 99%
B.
Expand student
residential network
bandwidth.
B.
01/01/10
B.
$100,000
B.
CIO,
VPSA
B.
Bandwidth is increased to
64 mbs
C.
Ensure that wireless
network is reliable and
accessible.
C.
10/01/09
C.
$50,000
annually
C.
AVPIT,
CIO
C.
Average uptime is 99%
during campus on-hours
and 95% during off-hours
Actual
Outcome
What’s the Difference?



Comprehensive process consisting of four steps
described earlier in presentation (i.e., mission to
outcomes), following common guidelines
A unit’s first assessment plan:
◦ typically takes a broader, longer view of the unit’s
functions and intentions
◦ will provide information for each of the four steps
◦ will delineate all goals and objectives for a unit, but
action plan will specify sub-set of those to be assessed
each year
In subsequent years, annual assessment plans
will highlight changes in assessment approach
and specify those objectives to be assessed
during the coming year



Required of administrative units at the
College for a number of years (although not
in a consistent format across divisions)
Relatively short-term and even post hoc in
nature, and not necessarily guided by unit
mission and broader goals
Likely to include description of events and
achievements not included in assessment
plan



IAC intentionally developed assessment
planning process that, ultimately, would easily
“translate” into annual reporting format
Much correspondence exists between “action
plan” components and annual report grid
Eventually, once assessment planning
becomes routine, there may be little
distinction between that process and annual
reporting
◦ Reasonable to suggest that assessment plan could
be incorporated into the annual report