Transcript Institutional Assessment: Connecting the Puzzle Pieces

Christine Licata
Senior Associate Provost
RIT
Anne Wahl
Director of
Student Learning
Outcomes Assessment
RIT
Institutional Assessment:
Connecting the Puzzle Pieces
MSCHE Presentation December 11, 2011
Workshop Outcomes
1. Self-assess infrastructure design and
processes
2. Review program level assessment
framework
3. Determine which tools and strategies to
adopt or adapt
Institutional Assessment
Major “pieces” of the puzzle:
1. Corner pieces
 Infrastructure
2. Center pieces:
 Student learning outcomes assessment
 Administrative unit effectiveness
Institutional Assessment
Infrastructure - 4 Corner Pieces
Corner Piece: Strategic
Vision RIT’s Strategic Plan
Key Results Area #3:
Demonstrate
Educational Excellence
Performance Commitment
11a. Meet/exceed outcome goals for
program-level student learning
FY 2009
FY 2010
FY 2011
FY 2012
FY 2013
Act
Goal
Act
Goal
Act
Goal
Act
Goal
Act
Goal
i. Expected program student
learning achievement levels NA
met or exceeded.
NA
40%
NA
NA
55%
NA
65%
NA
75%
NA
90%
85%
NA
85%
NA
85%
NA
100%
ii. Assessment results and
processes guide planning
and improvement.
NA
Corner Piece: Strategic
Vision - Provost’s Academic Quality Dashboard
Output Indicators: Learning Outcomes
Corner Piece:
Governance Structure
Three Central Committees
1. Student Learning Outcomes Assessment
Committee (SLOAC) (Academic Programs)
2. University Assessment Council (UAC)
(Administrative Units)
3. General Education Committee (GEC)
(General Education Framework)
Corner Piece:
Governance Structure
1. Student Learning Outcomes Assessment
Committee (SLOAC)



Key Support Areas:
Program level assessment
Annual progress report



Nine colleges represented
Meet monthly
Formal charge & work plan
Corner Piece:
Governance Structure
2.
University Assessment Council (UAC)



Key support areas:
Institutional Effectiveness Map
Annual Progress Report



All divisions represented
Meet every-other month
Formal charge & needs assessment
Corner Piece:
Governance Structure
3.
General Education Committee
(Standing Committee of Academic Senate)






Nine colleges represented
Meet weekly
Formal charge
Key support areas:
General Education Framework
General Education Assessment Plan
Corner Piece:
Leadership and Coordination
Centralized Support and Leadership for Assessment






One office responsible for Student Learning Outcomes
Assessment – in Provost’s Office
Permanent budget
Building capacity - staff: Director, Senior Assessment
Associate, Assessment Management System
Coordinator, and Senior Staff Assistant
Director co-chairs UAC and is a member of SLOAC, GEC,
Data Warehouse Steering Committee, Survey Council
Developing resources and website
Faculty stipends
Corner Piece:
Leadership and Coordination
http://www.rit.edu/academicaffairs/outcomes/
Corner Piece: Existing
Resources and Data Sources
Examples of Existing Resources

Institutional Research and Policy Studies Office
Data warehouse and student surveys

Teaching and Learning Center
Workshops and events for faculty

Cooperative Education Office
Co-op evaluation

Assessment Management System (TaskStream)
Corner Piece: Existing
Resources and Data Sources
Samples of Existing Data Sources
Assessment Resources
ASQ Admitted Student Questionnaire
Program-level Assessment of Student Learning
Co-operative Experience Evaluation
Delaware Study
Campus Climate Survey
Employee Satisfaction Survey
CSI – Freshmen Survey
NSSE
Noel Levitz Student Satisfaction Survey
Academic Program Review (2015)
Alumni Survey
Student-to-Student Survey
Enrollment Budget Projections and Forecasting
Internal Audits
Space Utilization Study
Student Evaluations of Teaching
Institutional Research Data
Middle States Standards
8
14
14
10
7,9
7
7, 9
1, 7, 9, 12
1, 7, 9
1,2,11
7, 11
9
2, 3
3
3
7, 10
1, 2, 7
Center Piece: Program-Level
Assessment Framework
Program-Level Student Learning Outcomes:
Planning
Implementing
Using Results
Sustaining
Center Piece: Program-Level
Student Learning Outcomes
Planning

University-wide engagement: Student
Learning Outcomes Assessment
Committee (SLOAC)

Common language: Program-Level
Assessment Plan
Center Piece: Program-Level
Assessment Plan
Program
Goals
List
programlevel
goals
Student
Learning
Outcomes
Students will
be able to:
(task,
capability,
knowledge,
skills, and
dispositions)
Use
measurable
verbs.
Academic Program
Profile
Align to the RIT essential
outcomes Check all that
apply.
 Critical Thinking
 Ethical Reasoning
 Integrative Literacies
 Global
Interconnectedness
 Creative/Innovative
Thinking
Data Source
Measure
Curriculum
Mapping
Assessment
opportunity
Course or
Experience
Method
measures
Assignment
rubric
Benchmark
Timeline
Standard,
target, or
achievement
level
(usually a %)
Statement of
student
success
Identify
when and
how data
are
collected
aggregated
and
analyzed
Data
Analysis
Key
Findings
Use of Results
Action Items and
Dissemination
Identify
who is
responsible
and list key
findings
Identify how
results are used
and shared. List
recommendations
or action items
Center Piece: Program-Level
Student Learning Outcomes
Implementing

Building relationships: from 1:1 to “moving in”

Leveraging accreditation

Providing support and resources for :




Benchmarking
Designing instruments
Developing rubrics
Conducting data analyses
Center Piece: Program-Level
Student Learning Outcomes
Use of Results

RIT Annual Progress Report (3 samples)

College-level action plans

SLOA office resources/support
Progress Report Sample
Emerging Assessment Program
BS Criminal Justice
PROGRAM OUTCOME 1
PROGRAM OUTCOME 2
Program Level Student Learning Outcome (SLO)
Demonstrate knowledge of key concepts,
Utilize critical thinking skills to apply
policy issues, legal and ethical issues
knowledge to social issues in crime and
surrounding crime and justice
criminal justice policies
Assessment Method
Course: Seminar in Criminal Justice Final
Field Experience Capstone Paper (mapped
Examination (mapped to SLO)
to SLO)
Achievement Benchmarks and Results
Benchmark: 70% of students will earn a
grade of C or better
Results: 81% met goal
Use of Results
 Faculty recommend moving from 1
comprehensive final to mini-exams to
provide additional assessment
opportunities and feedback.
 Re-assessment in spring
Benchmark: 70% will earn a grade of C
or better
Results: 100% met goal

Faculty to review rubric and rating
process to determine alignment of
rubric to SLO and consistency of
grading
Progress Report Sample
Developed Assessment Program
BS Game Design and Development
PROGRAM OUTCOME 1
PROGRAM OUTCOME 2
Program Level Student Learning Outcome(SLO)
Apply current technical concepts and practices
Incorporate accepted game industry processes
within the computing disciplines to the field of
towards the construction of a game design and
game design and development
development application or prototype
Assessment Method -Programming for Digital Media Course- selected Homework and Project
rubric (mapped to both SLOs)
Achievement Benchmarks and Results
Benchmark: 80% will achieve a 3 (acceptable) or Benchmark: 80% will achieve a 3 (acceptable)
higher average score across rubric items
or higher average score across rubric items
Results: 83% met goal
Results: 63% achieved 3
Use of Results
 Data was disseminated to faculty & Chair.

 Results discussed at faculty retreat.
 No action required as benchmark was met, but
information was utilized in the Semester
Conversion Process.

Analysis showed the content was too
condensed at week 8- limited time for
polished prototype experience. Course
modified- content spread over 10 weeks.
Re-assessment will take place in next cycle.
Progress Report Sample
Highly Developed Assessment Program
BS Mechanical Engineering
PROGRAM OUTCOME 1
PROGRAM OUTCOME 2
Program Level Student Learning Outcome (SLO)
Work independently and collaboratively while
Enhance skills through formal education and
demonstrating the professional and ethical
training, independent inquiry, and professional
responsibilities of the engineering profession.
development.
Assessment Method
 Employer Evaluation of Co-op Experience
Student Co-op Self-Evaluation: students rated
 Student Co-op Self-Evaluation
need/value of continuous learning
Achievement Benchmarks and Results
Benchmark: 100% will earn a 4 or above
Benchmark: 100% rank this as a 4 or above in
Results: 100% scored 4 or above by employers in: terms of importance
ethics, teamwork, leadership, respect. Students
Results: 100% of students ranked the
importance level as a 4 or higher
scored themselves above 3, but
below the levels reported by employers (4).
Use of Results
 Discrepancy between student and employers •Results suggest students understand and feel
suggest a need to offer more opportunities to prepared for continuous learning throughout
develop team building and leadership skills.
career.
 Further discussion by faculty in fall 2011 to
•Results are guiding strategic decisions at the
determine how further skill building in
department level about the future of engineering
teamwork and leadership can be intentionally education and approach to marketplace needs.
incorporated into curriculum.
Center Piece: Program-Level
Student Learning Outcomes
Sustaining

Technology



Assessment Management System
Website
Integrated processes
Academic program review
 Annual reporting
 Communication - regular sharing and dissemination of
findings and use of results


Closing the Loop


Input to budget
Informs strategic planning
Center Piece:
Administrative Unit Effectiveness

University Assessment Council (UAC)

Assessment Management System

Institutional Effectiveness (IE) Map

Next Steps:


AMS Annual Report
Develop unit audit
Center Piece: Administrative
Unit Effectiveness: IE Map
Administrative Unit:
Mission:
Contact Name:
Administrative
Division
Strategic Plan
Unit
Goal
Key Results Area
Objectives
(If
Align to the RIT’s Key
or
Applicable)
Results Areas - check
Outcomes
all that apply OR list
other Strategic
Initiative.
 KRA 1 Renowned
for Student Success
 KRA 2 Maximize
Opportunities for
Innovation,
Creativity, Research
and Scholarship
 KRA 3 Execute with
Organizational
and/or Operational
Excellence
 KRA 4 Achieve
Highest Levels of
Stakeholder
Satisfaction
 Other Strategic
Initiative - please list
Academic Year:
Data Source
Benchmark
Timeline
Achievement
Next Steps or
Unit Activities
or
&
of
Actions
Method/
Standard
Person(s)
Outcome/
What, if any, steps
Measure
Statement of Responsible
Objective
or actions will take
Describe
Success/
Indicated Indicate level of place as a result of
metrics or
Measureable Year, Cycle
achievement
the assessment
instruments
Outcome
and/or
findings
used to
Semester
measure results
Not Met
Approaching
Met
Exceeded
Center Piece: Institutional
Assessment Self-Assessment
Self-assessment tool for institutions
Questions