Transcript Planning for Improvement

Building bridges to constituents:
Communicating through performance
indicators and electronic portfolios
Trudy Banta, Victor Borden, & Susan Kahn
Sharon Hamilton
Policy and governance issues for electronic
student portfolios
Overview
 Planning for improvement
 PIs @ IUPUI
 The tip of the assessment iceberg
 iPort: The institutional portfolio
 Placing PIs in context
 ePort: The electronic student portfolio
 Demonstrating student learning
 Completing the cycle
 Linking accountability to improvement
Planning for Improvement
Planning & Budgeting
1.
2.
3.
4.
Mission, Vision, Goals developed
Unit goals aligned
Annual reports on web
Programs based on assessable goals,
with performance indicators
5. Biennial planning/budgeting hearings
conducted
Assessable
Outcomes
4. Proposing improvement initiatives
5. Improving assessment methods
 Web-based data
 Electronic portfolios
Culture
Of
Evidence
Data Collection
Analysis
Evaluation
1. Academic and administrative program
reviews
2. Evaluation of process effectiveness
3. Assessment of learning outcomes
 in major
 in general education
4. Course evaluations
5. Student assessment
6. Constituent surveys
7. Management information and analysis
8. Program cost analysis
9. Web-based evaluation tools
10. Annual campus performance report
11. NCA accreditation
Instrumentation
external stakeholders
3. Applying findings in campus
improvement initiatives
of Findings
1. Reporting to internal constituents
2. Demonstrating accountability to
Application
Improvement
Implementation
(Everyone on campus
implements goals)
Planning and Budgeting
1.
2.
3.
4.
Mission, Vision, Goals developed
Unit goals aligned
Annual reports on web
Programs based on assessable
goals, with performance indicators
5. Biennial planning/budgeting hearings
conducted
Implementation
Everyone on campus
implements goals
Evaluation
1. Academic and administrative program reviews
2. Evaluation of process effectiveness
3. Assessment of learning outcomes
 in major
 in general education
4. Course evaluations
5. Student assessment
6. Constituent surveys
7. Management information and analysis
8. Program cost analysis
9. Web-based evaluation tools
10. Annual campus performance report
11. NCA accreditation
Improvement
1. Reporting to internal constituents
2. Demonstrating accountability to
external stakeholders
3. Applying findings in campus
improvement initiatives
4. Proposing improvement initiatives
5. Improving assessment methods
•
•
Web-based data
Electronic portfolios
Planning for Improvement
Planning & Budgeting
1.
2.
3.
4.
Mission, Vision, Goals developed
Unit goals aligned
Annual reports on web
Programs based on assessable goals,
with performance indicators
5. Biennial planning/budgeting hearings
conducted
Assessable
Outcomes
4. Proposing improvement initiatives
5. Improving assessment methods
 Web-based data
 Electronic portfolios
Culture
Of
Evidence
Data Collection
Analysis
Evaluation
1. Academic and administrative program
reviews
2. Evaluation of process effectiveness
3. Assessment of learning outcomes
 in major
 in general education
4. Course evaluations
5. Student assessment
6. Constituent surveys
7. Management information and analysis
8. Program cost analysis
9. Web-based evaluation tools
10. Annual campus performance report
11. NCA accreditation
Instrumentation
external stakeholders
3. Applying findings in campus
improvement initiatives
of Findings
1. Reporting to internal constituents
2. Demonstrating accountability to
Application
Improvement
Implementation
(Everyone on campus
implements goals)
PIs @ IUPUI
Performance Indicators as the tip of the
assessment iceberg
PIs @ IUPUI: Background
 Status of PIs in the early 1990s
 Kells (OECD)


Borden & Banta (NDIR 82)


Failure of PIs for national accountability in Europe and
elsewhere. Movement toward quality assurance
Greater possibilities for PIs if they emerge from
comprehensive planning efforts
Gaither, Nedweck, & Neal (ASHE ERIC)

Increasing demand for accountability and continuing
inability of HEIs to communicate about performance to
external audiences
 The Rise of CSFs, KSFs, KPIs
 Rockhart (1979) Critical Success Factors (CSFs)
 NACUBO/ Peat Marwick – 67 Indicators
 Dolence (1989) KPIs for enrollment management
Balanced Score Card (BSC)
 Kaplan & Norton propose business model
 Financial performance
 Customer service and satisfaction
 Process effectiveness and efficiency
 Organizational learning
 Ruben (1999) applies BSC to HEIs





Teaching/Learning - Programs/Courses, Student Outcomes
Service/Outreach - University, profession, alumns, state,
prospective students, families employers
Scholarship/Research - Productivity/Impact
Workplace satisfaction - Faculty/staff
Financial - Revenues/expenditures
Problems with Current Systems
 Despite Ruben’s challenge to focus on
mission critical activities, most focus on
administrative functions
 There is a disconnect between
performance indicators and core
performance objectives
 What is the organizational link between
the accountability focus of campus
executives and the performance of
mission critical functions?
Making the Connection
 Doing assessment right (Banta, 2002)
 Participative planning


Implementation


Assessment must be explicitly tied to institutional
functions and be associated with goals that faculty and
administrators value
Assessment is most powerful at the unit level where
changes will most immediately affect the faculty and
students
Improving and sustaining

while the results of assessment may be used to respond
to various external accrediting and stakeholder
constituents, it will first and foremost provide the means
by which the organization can continuously assess and
improve itself

Excellence in Teaching and Learning

Attract and support a better prepared, more diverse student
population.
 Support and enhance effective teaching
 Enhance undergraduate student learning and success
 Provide effective professional and graduate programs and support
for graduate students and post-doctoral fellows.

Excellence in Research, Scholarship, and Creative Activity

Conduct world-class research and creative activities relevant to
Indianapolis, the state, and beyond.
 Provide support for scholarly activity and external funding.
 Enhance infrastructure for research and creative activities.

Excellence in Civic Engagement, Locally, Nationally, and
Globally


Enhance capacity for civic engagement.
Enhance civic activities, partnerships, and patient and client
services.
 Intensify commitment and accountability to Indianapolis, Central
Indiana, and the state.
Collaboration – Diversity – Best Practices
PIs @ IUPUI: Performance
Objectives
PI Development Process
 Vested interested groups (handout)
 Brainstorming session
 What evidence would convince a (neighbor,
prospective student, legislator) that you we were
doing well?
 What evidence do you (or could you) use in your
unit to assess performance?
 3-4 performance indicators per objective
 Three level rubric (green, yellow, red)
 Adapted Nominal Group Technique for voting
Status and Results
 In Handout
 Summary
of performance objectives and
performance indicators, with measures
shown for one set of indicators
 Published in…
 Campus
annual performance report
 Chancellors State of Diversity address
 Institutional portfolio
iPort
The Electronic Institutional Portfolio
“Institutional Portfolio”
A focused selection of authentic
institutional work, combined with
narrative analysis, that demonstrates
specific institutional achievements and
shows learning and improvement over
time—i.e., “institutional effectiveness”
IUPUI’s iPort
 Begun in 1998 as part of national grant



funded project
Organized around main components of
institutional mission and major strategic
priorities
Intended for purposes of both internal
improvement and accountability to multiple
stakeholders
Used as platform for accreditation self-study
in 2002
Is on the Web—available to all
What do we gain from work on
an electronic institutional portfolio?
 Occasions for focused, campus-wide
discussions of areas needing improvement
 A means of demonstrating institution’s
commitment to accountability and
effectiveness
 An easy, accessible medium for
communicating with stakeholders about
accomplishments and effectiveness
Next Steps
 Develop new iteration/update as annual
performance report
 Update the design and technological
infrastructure
 Foreground links to other relevant
campus Web sites
 Incorporate more direct evidence of
student learning through interface with
student electronic portfolio
ePortfolio
The Electronic Student Portfolio
“Unbundling”
Testing
Syllabus
In Touch
Schedule
Federated Searching
Gradebooks
Digital content
Ereserves
Full Text articles
Authn/Authz
Security
Workflow
Comm.
Tools
Storage
Learning Matrix
Learning Profile
Content Manager
Advising
Research Manager
Resume/Vita Builder
Knowledge Mapper
Other
Services
Authn/Authz
Security
Workflow
Comm
Tools.
Storage
Other
Services
ePortfolio Goals
 Support students' learning of and engagement with
the PULs over their entire undergraduate experience
at IUPUI beginning in the freshman learning
community and culminating in the capstone
experience
 Assist both faculty and students to reach a clearer,
more coherent understanding of how various aspects
of the curriculum support students' increasing
mastery of the PULs
 Contribute to assessment of student learning of the
PULs at the levels of the individual student, the
course, program, and institution
ePortfolio and the PULs
The ePortfolio is organized around IUPUI's
Principles of Undergraduate Learning
1. Core communication and quantitative skills
2. Critical thinking
3. Integration and application of knowledge
4. Intellectual depth, breadth, and adaptiveness
5. Understanding society and culture
6. Values and ethics
ePortfolio Levels of Competence
 Introductory: What all undergraduate
students at IUPUI should know and be able to
do in relation to the PULs within the first 26
credit hours
 Intermediate: What all undergraduate
students at IUPUI should know and be able to
do in relation to the PULs within the first 56
credit hours
 Advanced: What all baccalaureate students
at IUPUI should know and be able to do in
relation to PULs in their major or profession
or academic program
Learning Matrix
Planning for Improvement
Planning & Budgeting
1.
2.
3.
4.
Mission, Vision, Goals developed
Unit goals aligned
Annual reports on web
Programs based on assessable goals,
with performance indicators
5. Biennial planning/budgeting hearings
conducted
Assessable
Outcomes
4. Proposing improvement initiatives
5. Improving assessment methods
 Web-based data
 Electronic portfolios
Culture
Of
Evidence
Data Collection
Analysis
Evaluation
1. Academic and administrative program
reviews
2. Evaluation of process effectiveness
3. Assessment of learning outcomes
 in major
 in general education
4. Course evaluations
5. Student assessment
6. Constituent surveys
7. Management information and analysis
8. Program cost analysis
9. Web-based evaluation tools
10. Annual campus performance report
11. NCA accreditation
Instrumentation
external stakeholders
3. Applying findings in campus
improvement initiatives
of Findings
1. Reporting to internal constituents
2. Demonstrating accountability to
Application
Improvement
Implementation
(Everyone on campus
implements goals)