Transcript Going Public with Quality Assurance: Using the Web to

Using Electronic Portfolios to
Assess Learning at IUPUI
Trudy Banta
Sharon Hamilton
Susan Kahn
Indiana University-Purdue University
Indianapolis
Plan
Improve
Culture of
Evidence
Evaluate
Implement
PLANNING
1.
Campus mission, goals
2.
Unit goals aligned
3.
Programs based on assessable
goals with PIs
4.
Annual reports on the Web
PAII Evaluation Services
1.
2.
3.
4.
5.
6.
7.
8.
Assessment of learning
Surveys
Program reviews
Performance indicators
Program cost analysis
Web-based evaluation tools
Program evaluation/action research
Accreditation
IMPROVEMENT AGENDA
1. Reporting to internal constituents
2. Demonstrating accountability
www.iport.iupui.edu
3. Proposing improvement initiatives
based on findings
4. Improving assessment methods
Assessment of Individual
Student Development
1. Assessment of basic skills for use in
advising
•
•
Placement
Counseling
2. Periodic review of performance with
detailed feedback
3. End-of-program certification of
competence
•
•
Licensing exams
External examiners
Key Results of Individual
Assessment
1.
Faculty can assign grades
2.
Students learn their own strengths
and weaknesses
3.
Students become self-assessors
A Second Look
1. Across
students
2. Across
sections
3. Across
courses
1.
2.
3.
Where is learning satisfactory?
What needs to be retaught?
Which approaches produce the
most learning for which students?
Group Assessment Activities
1.
2.
3.
4.
5.
6.
7.
Classroom assignments, tests, projects
Questionnaires for students, graduates,
employers
Interviews, focus groups
Program completion and placement
Awards/recognition for graduates
Monitoring of success in graduate school
Monitoring of success on the job
Use of Results of Group
Assessment
1.
Program improvement
2.
Institutional and / or state peer
review
3.
Regional and / or national
accreditation
Assessment of Learning
1.
Faculty agreement on outcomes and
performance standards
2.
Evidence in syllabi and assignments
that outcomes are taught
3.
Collective faculty review of group
performance
4.
Use of findings to improve instruction
and curriculum
Methods of Assessment
1.
2.
3.
4.
5.
6.
7.
8.
9.
Paper and pencil tests
Individual or group projects
Portfolios
Observation of practice
Observation of simulated practice
Analysis of case studies
Attitude or belief inventories
Interviews and focus groups
Surveys
Now We Have
the
Press to assess with a test
2006
Commission on the Future of
Higher Education
1.
Need simple way to compare
institutions
2.
Institutions should be more accountable
for student learning
What happens to outcomes
assessment
undertaken
to guide improvements
in
teaching and learning?
Using Electronic
Assessment Methods
We can
1. Track progress in assignments
2. Evaluate contributions to group
projects
3. Conduct immediate process checks
to evaluate instruction
4. Assess the quality of written work
What is an e-portfolio?
A selection of purposefully organized
artifacts that support retrospective and
prospective reflection and that
document, assess, and enhance
student learning over time.
Why e-portfolios?
1.
2.
3.
For students

Track growth and development

Develop capacities for metacognition and selfdirected learning

Integrate and apply learning
For faculty

Enrich learning experiences

Base assessment on authentic performances
For programs and institutions

Assessment for admissions, improvement and
effectiveness, accreditation
Principles of Undergraduate Learning
1.
Core communication and quantitative skills
2.
Critical thinking
3.
Integration and application of knowledge
4.
Intellectual depth, breadth, and adaptiveness
5.
Understanding society and culture
6.
Values and ethics
ePort Learning Matrix
ePort levels of competence
What all undergraduate students at IUPUI
should know and be able to do within their:
1.
2.
3.
4.
Introductory: First 26 credit hours
Intermediate: First 56 credit hours
Advanced: Major, profession, or academic
program
Experiential: Co-curricular, extracurricular, work-based learning
Inside a Matrix Cell
Goal: To document
introductory competence
in written communication
Expectations:
1.You have identified a different
audience and purpose for at least
2 documents
2.You have selected information
appropriate to your audience and
purpose.
3.You have used two different
organizational styles appropriate
to audience and purpose
4. You have used appropriate
language, style, writing
conventions, and formats.
Features of ePort
1.
Resources
2.
Learner Profile
3.
Matrices
PUL
Customized
4.
Presentation Tool
5.
Reporting Tool
Dashboard/Learner Profile
• Tasks
• Profile
• Pre-Survey
• Preflection
• Upload
resources
• Reflection
Profile
Information
Connections to
self-assessment
tools
Connections to
Career Center;
Student Life and
Diversity
Connections to
other helpful sites
assess
interact interact
assess
assess
interact
assess
reflect
Criteria
Artifact
reflect
reflect
reflect
Artifact Artifact Artifact
Communication
Skills
Intro Intermed Adv.
Exper
Critical
Thinking
Intro Intermed Adv.
Exper
Integration &
Application of
Knowledge
Intro Intermed Adv.
Exper
Understanding
Society &Culture
Intro Intermed Adv.
Exper
Values &
Ethics
Intro Intermed Adv.
Exper
Etc ….
Aggregate
Crs grade Crs grade Crs grade Crs grade
Time/level/Linear
•
(Reports)
interact
Adoption strategies
1.
2.
3.
Small grants to departments and to
freshmen learning communities
Collaboration with users to develop
needed features and modify or
customize existing ones
Extensive faculty development and
technology support
Student Self-Assessment in the
IUPUI ePort
From Gateway Through Graduation
From Gateway Through Graduation
1.
First year:


2.
Preflection
Reflection
Capstone:


Retrospective
Prospective
Preflection
1.
What are your important goals for your life after
college? Think about your goals in areas like
career, citizenship, relationships, personal life, and
spiritual/religious development.
2.
Thinking of one or two of these goals, what do
you need to do and learn between now and
graduation to get there? What abilities, skills,
knowledge, and other characteristics do you need to
improve or develop?
3.
Which of the PULs seem(s) most important for
you to improve upon to reach your goals? Why?
Reflection
1.
Reflect upon the goals you had during your first
weeks at IUPUI.

Have your goals, or your ideas about how to reach
them, changed?

What have you learned that is relevant to the
goal(s) discussed in your preflection? What is the
evidence for this learning?

Are there additional PULs that you now see as
relevant to your goals?
Capstone
Matrix
LIFELONG POSSIBLE
LEARNING CAREERS
CURRENT
ROLE IN
SOCIETY
PRINCIPLE OF
UNDERGRADUATE
LEARNING
Artifacts
Artifacts
Artifacts
REASON FOR
BECOMING AN
ENGLISH MAJOR
Artifacts
Artifacts
Artifacts
Matrix Thinking
1.
2.
3.
4.
5.
Movement from focus on original
document to focus on original document in
relation to a new conceptual framework
See work in new focus
Results in greater depth of understanding
of original work
Students can assess their intellectual
growth since the original writing.
Development in reflective thinking
Development in Reflection

Ability to self assess

Awareness of how one learns

Developing lifelong learning skills
Ability to Self-Assess

Observing One’s Own Performance

Using Feedback and Evidence

Finding and Analyzing Patterns

Making Judgments
Islam Y107.doc exemplifies my ability to be a
critical thinker because I had to put forth
significant effort to separate my emotion from the facts
and research….I believe that objectivity and rationality
are at the core of every serious student – this paper
shows me that I can be a serious student. Every class
that I have taken in political science, English, and
philosophy has emphasized the importance of looking
past the surface of things. Additionally, my education in
the liberal arts has taught me that there is much more to
things than what my emotions tell me there are….My
emotions are central only to my own experiences, and
my critical thinking skills allow me to leap outside of my
own experiences.
Critical thinking is my most important PUL
because, no matter what goal I am
pursuing, critical thinking will be a
necessary step before that goal will ever be
reached….Thinking about working as an
advocate of the law in a globalizing world is
intimidating. The documents in this cell,
however, have calmed my angst so that I
feel as though I am capable of
accomplishing my goals. Two of the papers
in this cell are only one page in length. I’ve
come to learn that brevity and conciseness
coupled with poignancy and clarity serve as
valuable evidence that one has thought well
and critically. To be brief is difficult if
there is a lot to say. There is also a very
long document in this cell.…Longevity of
thought is sometimes necessary before
brevity of a statement can be made clear.
I no longer see what I have to offer as an English
job hunter in mere terms of degree possessed
and years of experience…I look at what I have to
offer in a larger context. Beyond the essentials in
my resume that I share with all other graduates, I
now see capacities in critical thinking,
communications, and multi-project analyses. All
these capacities can be supported with the
creative and scholarly material in my matrix.”
assess
interact interact
assess
assess
interact
assess
reflect
Criteria
Artifact
reflect
reflect
reflect
Artifact Artifact Artifact
Communication
Skills
Intro Intermed Adv.
Exper
Critical
Thinking
Intro Intermed Adv.
Exper
Integration &
Application of
Knowledge
Intro Intermed Adv.
Exper
Understanding
Society &Culture
Intro Intermed Adv.
Exper
Values &
Ethics
Intro Intermed Adv.
Exper
Etc ….
Aggregate
Crs grade Crs grade Crs grade Crs grade
Time/level/Linear
•
(Reports)
interact