Transcript Assurance of Learning: Why, what, who, when and how?

Assessment 101
(Academic and Administrative)
Marta Colón de Toro, SPHR
Assessment Coordinator
College of Business Administration
UPR-Mayagüez
Today’s Learning Goals
After completing this workshop you should be
able to:
Explain the importance and purpose of assessing outcomes,
both, academic and administrative.
2. Recognize accreditation standards on assessment.
3. Describe the process of assessment and the challenges it
represents.
4. Develop a plan to conduct assessment.
5. Identify and apply appropriate instruments to directly assess
outcomes in your area.
6. Identify ways to document assessment.
1.
2
Tell me…

Why are we here today?

Why are we here, at UPR-Mayagüez?
3
A Transformation Process
Input
Incoming
Student
• Demographics
• Academics
• Experience
• Talents
• Attitudes/Values
University Processes
Curriculum
Seminars and Competitions
Internship/Coop
Student Associations
Extracurricular Experiences
Environment/Services
Output
Completing
Student
• Knowledge
• Skills
• Abilities
• Experience
• Attitudes/Values
4
UPR-Mayagüez
Visión
"Nuestro compromiso es preparar a los mejores profesionales, llevar a cabo
investigación y divulgar el conocimiento en una universidad completa, ágil y
eficiente".
Misión
El Recinto Universitario de Mayagüez tiene como misión dirigir sus esfuerzos para
proveer a nuestra sociedad ciudadanos educados, cultos, capaces de pensar
críticamente y preparados profesionalmente en los campos de la agricultura, la
ingeniería, las ciencias naturales y sociales, las humanidades y la administración de
empresas, para que contribuyan al desarrollo cultural, social y económico de Puerto
Rico.
Este proceso está dirigido a dotar a nuestros egresados de una preparación técnica y
profesionalmente sólida y a desarrollar en ellos una actitud de compromiso con
Puerto Rico y con nuestro Hemisferio. Nuestros egresados deben poseer, además las
destrezas y los conocimientos necesarios que les permitan participar efectivamente
en la búsqueda de soluciones a los problemas que nos afectan, propulsar el
enriquecimiento de las artes y la cultura, la transferencia de tecnología, y defender
los valores que nos caracterizan como una sociedad democrática.
5
To achieve this mission, UPRM has set forth nine
primary goals (Cert. # 96-97-603):
1.
2.
3.
4.
5.
6.
7.
8.
9.
Have students become the central figures and “reason for
being”.
Become an integral institution characterized by excellence in
teaching, research, and service to the community.
Encourage an entrepreneurial spirit among students
Internationalize the institution.
Strengthen research and development.
Promote complete computerization.
Evolve into an institution that is operationally agile, efficient, and
auditable.
Encourage close collaboration with and provide ample service
to the community.
Establish a master plan with a focus on infrastructure and
physical equipment.
6
By the time of their graduation, UPRM
students will be able to:
Communicate effectively.
2. Identify and solve problems, think critically, and
synthesize knowledge appropriate to their discipline.
3. Apply mathematical reasoning skills, scientific inquiry
methods, and tools of information technology.
4. Apply ethical standards.
5. Recognize the Puerto Rican heritage and interpret
contemporary issues.
6. Appraise the essential values of a democratic society.
7. Operate in a global context, relate to a societal
context, and demonstrate respect for other cultures.
8. Develop an appreciation for the arts and humanities.
9. Recognize the need to engage in life-long learning.
1.
7
Introduction
To accomplish our Vision/Mission…
Everything we do should be:
…pertinent
…coherent
…strategic
…dynamic
…assessable
…intentionally designed!
8
Objective #1

Explain the importance and purpose of
assessment.
9
Objective #1
Assessment helps us ask ourselves..
Are we achieving our goals?
Academically
Are my students learning?
How much are they learning?
How well are they learning?
What affects their learning?
Support Units
Are we achieving our unit’s goals?
How much are we achieving?
How well are we achieving them?
What affects our services?
10
Objective #1
Assessment
…is a way of describing what we do
(student learning or support
services) to an identified audience
for clearly- articulated reasons.
…produces information useful for
communication and decisionmaking.
11
Objective #1
Purpose of Assessment
Assessment
Intended Outcomes
Actual Outcomes
1.
1.
2.
2.
3.
3.
4.
4.
5.
5.
Reveals the
Gap
12
Assess Learning to:
1.
2.
3.
Objective #1
Provide feedback and guidance to
individuals (students/employees).
Assist the unit members to improve
programs (courses and services).
Assure external constituents that the
institution meets its academic goals.
13
Objective #1
Levels of Assessment
Institutional Goals
Deanship Goals
Unit Goals
Objectives
14
Objective #1
Effective assessment…
Is directed by goals
 Uses multiple measures
 Show student development over time
 Uses multiple types of information

15
Objective #1
Challenges of Assessment
Motivating faculty, staff, students,
administration
 Needs realistic goal setting
 Data management
 Doing it for the right purpose
 Requires taking action
 Could be costly

16
Objective #2

Recognize accreditation standards on
assessment
17
MSCHE: Standard 7
Institutional Assessment
Objective #2
The institution has developed and implemented an
assessment plan and process that evaluates its overall
effectiveness in:
 achieving its mission and goals;
 implementing planning, resource allocation, and
institutional renewal processes;
 using institutional resources efficiently; providing
leadership and governance;
 providing administrative structures and services;
demonstrating institutional integrity;
 and assuring that institutional processes and
resources support appropriate learning and other
outcomes for its students and graduates.
18
Objective #2
MSCHE: Standard 14
Assessment of Student Learning

Assessment of student learning
demonstrates that the institution’s
students have knowledge, skills, and
competencies consistent with institutional
goals and that students at graduation
have achieved appropriate higher
education goals.
19
Objective #2
Other Accrediting Bodies






Business – AACSB
Engineering – ABET
Chemistry –
Nursing
Library
Research grants
20
Objective #3
•
Describe the process of assessment.
21
Objective #3
The Assessment Process
External Reality
Vision/Mission
Set Goals and
Design Strategies
Improve
Continuously
Enable
Achievement of Goals
Assess
22
Objective #3
For example…in a classroom
1.
2.
3.
4.
5.
6.
7.
Establish Learning Goals:
“The student will be able to discuss the importance of assessing student
learning.”
Identify or design enabling activities:
 Brief presentation on topic
 Brainstorming
 Q&A
Assurance of Learning Standards
Select:
 Indicator: “Participant will present 2 arguments pro/con of assessment w/o
consulting notes.”
 Instrument: Focused Listing
Assess:
 List 2 arguments in favor and 2 arguments against doing assessment.
Measure, interpret and report results (75% - listed 2/2, 15% 1/2, 10% 2/0…)
Introduce change
Revise plan (Change goal, teaching methodology, student’s preparation…)
23
Objective #3
For example…in an office
1.
2.
3.
4.
5.
6.
7.
Establish Goals:
“Increase student participation in cultural activities.”
Identify or design enabling activities:
 Promotional campaign in classrooms
Select:
 Indicator: “Increase students participation in cultural activities by
20% by the end of Academic Year 2006-2007.”
 Assessment Instruments
 Actual attendance records. - Quantitative
 Focus groups/Opinion survey - Qualitative
Assess: Measure
Interpret and report results (75% - listed 2/2, 15% 1/2, 10% 2/0…)
Introduce change
Revise plan (Change goal, teaching methodology, student’s
preparation…)
24
Objective #4
•
Develop a plan to conduct assessment.
25
Objective #4
Where to start?
PLAN!
Step#1 – Establish goals
• Should answer five questions:
1.
2.
3.
4.
5.
Are they aligned with the higher-level goals?
What type of measure do they address?
Are they realistic?
Are they outcomes-based?
Are they customer (student)-centered?
Step #2 - Identify/design activities to enable achievement of
each goal.
Step #3 - Select or design assessment instruments and
indicators.
Step #4 – Assess
Step #5 – Interpret and report results
Step #6 – Identify and introduce change strategies
Step #7 – Revise plan
26
Objective #4
For example: In class
Learning Goals
1. Leadership
2. Critical Analysis
3. Interpersonal
skills
4. Team Work
5. Problem Solving
Teaching Methodologies







Lecture
Case analysis and discussion
Oral presentations
Essays/Reports
Team Projects
Experiential Learning
Simulations
Extracurricular Activities


Seminars/forums
Field trips
27
For Example: In an office
Office Goals
1. Increase qty/qlty
2. Reduce errors
3. Increase
participation
4. Reduce time
5. Customer service
1.
2.
3.
4.
5.
Objective #4
Strategies
Train employees
Increase/try new
communication
channels
Improve technology
Re-engineer
processes
Relocate personnel
28
Objective #5

Identify and apply appropriate
instruments to directly assess outcomes.
29
Objective #5
Assessment Assumptions:






Quality of outcomes is directly, although not
exclusively, related to the quality of inputs.
Human behavior is goal oriented.
Feedback should be received early and often.
Assessment provides systematic inquiry and
challenges status quo.
Assessment does not require specialized training
Assessment efforts enhances performance and
personal satisfaction.
30
Objective #5
Assessment Techniques
Learner/Customer-Centered
 Teacher/Management-Directed
 Mutually Beneficial
 Formative
 Context-Specific
 Ongoing
 Rooted in Good Teaching/Administration
Practice

31
Objective #5
Classroom Assessment Techniques
Direct Measures










1 Minute paper
Muddiest Point
One-sentence summary
Direct paraphrasing
Application cards
Empty outlines
Pro-con grids
Class journals/Portfolios
What’s the principle?
Rubrics
Indirect Measures
Student evaluation of courses
 Alumni Feedback
 Internship/Coop

32
Administrative Assessment
Objective #5
Opinion/Satisfaction surveys
 Daily logs
 Attendance records
 Quantity/quality measures
 Process analysis
 Workflow analysis

33
REMEMBER…
 These
ideas are suggestions to be
adapted, not models to be adopted
 Don’t
ask if you don’t really want to
know or are not going to do anything
about it.
 Start
with something simple and then
add up.
34
Objective #6

Identify ways to document
assessment.
35
Documenting CATs
CAT’s

Diagnostic Tests
 Pre-Post Test
 Muddiest Point
 Minute Paper
 Application Cards
 Direct Paraphrasing
 Course Final General
Assessment
Documentation







Brief Description or
Sample copy
Sample of Grades
(H,M,L)
Sample Answers
Analysis of Outcomes
Conclusions
Recommendations
Actions Taken
36
Documenting administrative
assessment
Survey samples
 Descriptive statistics
 Institutionalize it through

Publications
 Requiring it for decision making
 Create position
 Assign resources

37
Analysis of Outcomes

Quantitative

Descriptive statistics



Average, Median, Mode, Standard deviation
Question by question analysis
Qualitative

Look for
Misconceptions
 Attitudes/prejudices

38
Conclusions/Recommendations
Summarize lessons learned
 List actions to be taken for improvement
 TAKE ACTION!!!

Refer pertinent recommendations to
appropriate forum
Implement changes
Assess again, and again, and again…
39
Stages of Grieving





Denial
Anger
Bargaining
Depression
Acceptance
On Death and Dying
by Elisabeth Kubler-Ross
40
Good luck!
Keep learning!
[email protected]
787-265-3801
41