Transcript Building a Culture of Continuous Improvement, 2011 to 2021

Continuous Improvement Plan
Building a Culture of Continuous Improvement
2011 to 2021
Monica Varner, Ph.D.
Director of Assessment and Institutional Effectiveness
Building A Culture of Continuous
Improvement
Meaningful Assessment
Accountability Framework
Student Learning Outcomes
Student Learning & Performance
Learning Outcomes, Measures,
& The Assessment Loop
Student Learning Portfolio
Faculty Development
Continuous Improvement Annual Report
Meaningful Assessment
“I’ll argue that most assessment
efforts have resulted in little
learning improvement because
they have been implemented
without a clear vision of what
“higher” or “deeper” learning is
and without understanding of
how assessment can promote
such learning.”
(Angelo, 1999: 59-60)
“Though accountability matters, learning still matters
most.” (Angelo, 1999: 59)
Accountability Framework
Essential Learning Outcomes
Robust Educational Programs
Concrete Levels of Performance
Cohesive Learning Community
General & Program Specific Goals
Assessment Across The Curriculum
Cumulative Learning and
High Level of Student Accomplishment
• Validation of Assessment Practices
• Commitment to Educational Improvement
• Visibility & Transparency
•
•
•
•
•
•
•
AACU, 2008. “Our Students’ Best Work: A Framework for Accountability Worthy of Our Mission” 2 nd Edition.
Essential Learning
Outcomes












Information Literacy
Inquiry and Analysis
Integrative Learning
Written & Oral
Communication
Critical Thinking
Creative Thinking
Quantitative Literacy
Ethical Reasoning & Action
Intercultural Knowledge
and Competence
Problem Solving
Civic Engagement
Foundations and Skills
For Lifelong Learning
Value Rubrics Project: Valid Assessment of Learning in Undergraduate Education, www.aacu.org.
Continuous Improvement
• Bloom’s Taxonomy Student Learning Action Verbs:
Knowledge: define, describe, recognize, label
Comprehension: explain, summarize, associate
Application: solve, illustrate, demonstrate
Analysis: compare, classify, contrast, infer
Evaluate: criticize, discriminate, interpret
Create: design, create, invent, change
https://www.google.com/search?q=revised+blooms+actio
n+verbs (English4fun-Blog:Bloom’s (Revised)Taxonomy)
Desired Student Learning Outcomes
Employer Rating of Desired Student Learning:
•
•
•
•
•
•
•
•
•
•
Problem Solving Diverse Settings
Ethical Issues In Field
Direct Experience With Community
Civic Knowledge, Skills, and Judgment
Critical Thinking & Analytic Reasoning
Liberal Arts and Sciences
Written and Oral Communication
Applied Knowledge In Real World Settings
Information Literacy
Innovation & Creativity
91%
87%
86%
82%
82%
80%
80%
78%
72%
71%
Hart Research Associates, It Takes More than a Major: Employer Priorities for College Learning & Student Success
(Washington, DC: AAC&U, 2013) LEAP Employer – Educator Compact: Making Quality A Priority As Americans Go To College. April. 2013
www.aacu.org/leap
Student Learning & Achievement:
Challenges To Consider
Knowledge of Human Cultures and the Physical and Natural World
ETS:
10% of college seniors test proficient in mathematics.
Intellectual and Practical Skills
ETS:
8% of college seniors test proficient in critical thinking.
9% of college seniors test proficient in written communication.
Integrative and Applied Learning
NSSE: 20% of seniors report having worked with a faculty
member on a research project.
49% of seniors report doing internships.
33% of seniors report participating in a senior culminating
experience (capstone project).
Finely, A. 2012. Making Progress? What We Know About the Achievement of Liberal Education Outcomes. (AAC&U, 2012)
What Impacts Learning &
Performance?
Psycho-Social Impact:
• Cognitive
• Emotional
• Social
• Motivation
• Socialization
• Prior Knowledge
• Social Capital
Organizational Impact:
• Institutional Practices
• Pedagogical Practices
• Learning Environments
(McAuliff, 2013; Pew, 2007;
Tinto, 2006-2007; Dika & Singh, 2002)
Student Learning Outcomes
Student Learning Outcomes Should:
• State clearly what students will know.
• State clearly what students will be able to do with what they know.
• Be clear and implementable goals.
• Include formative & cumulative learning.
• Be measureable & related to learning.
• Be able to be assessed by multiple measures.
Varner, M., 2012. Assessment Best Practices: Assessment Summit
(Suskie, 2009; NILOA, 2012; Anderson and Krathwohl, 2001)
Assessment Instruments &
Activities:
Indirect Measures:
Course Grades
Student Satisfaction Survey
Focus Groups
Graduation Rates
Graduate School Admission Rates
Job Placement Data
Direct Measures (Rubrics)
Objective Tests
Research Papers
Supervisor Rating
Oral Presentations
Student Reflections
Online Assessment:
Online Readiness Survey
Timely Performance Feedback
Peer Feedback
Multiple Assessment Strategies
Student-Content
Student-Peers
Student-Instructor
Student-LMS
Threaded Discussions
Group Work
Writing Assignments
Video Oral Presentations
(Varner, 2012; Suskie, 2009; Value Project: Valid Assessment of Learning in Undergraduate Education, 2012; CSU CTL: Online, 2012)
Student Learning:
Continuous Improvement
• Performance standards should provide
a clearly defined threshold for
student learning.
• Results should provide a
snapshot of overall student learning.
• Conclusion statements:
Student Learning Focused
Strengths and Weaknesses
Assessment Changes
Curricular Improvements
Varner, September 2012, Assessment Best Practices: Assessment Summit. (Suskie, 2009; NILOA, 2012)
Continuous Improvement
 Faculty Assessment Teams
 Assessment Liaisons
 Assessment Collaboration
 Assessment Planning
Assessment Peer Review Discussions
 Casual Conversations
 Collaboration & Communication
 Continuous Improvement
Continuous Improvement
Assessment Development:
 Assessment consultation with faculty, assessment representatives,
and academic program faculty from the Office of Assessment and
Institutional Effectiveness.
Assessment Participation:
 Faculty engagement with the assessment process.
 Faculty participation in assessment peer review discussions.
Assessment Satisfaction:
 Faculty who are involved with student learning assessment are
more satisfied with university assessment processes.
Continuous Improvement
Assessment Support & Training:
Mentoring
Assessment Workshops
Curriculum Mapping
Assessment Planning
Continuous Improvement Reports
General Education Program
Continuous Improvement
 Promote and Enhance
Student Learning
 Increase Opportunities
For Students To
Apply & Practice Learning
 Promote Interdepartmental
Communication
 Promote The Assessment
Process
 Maintain a Cohesive
General Education Program
 Promote Continuous Improvement
Continuous Improvement
• Assessment Planning
• Implementation & Analysis
• Curricular & Assessment
Improvements
• Continuous Improvement
Annual Reports
Academic Program Reviews:
Continuous Improvement Plan
Continuous Improvement Reports
Assessment Peer Review Reports
Curriculum Mapping
Student Satisfaction Surveys
 Post-Assessment Student Results
Continuous Improvement
Using Data for Improvement:
Enhanced Student Learning
Higher Retention Rates
Higher Graduation Rates
Increased Program Quality
Enhanced Usage of Resources
References
AACU, (2008). “Our Students’ Best Work: A Framework for Accountability Worthy of Our Mission” 2nd Edition.
Anderson, L.W. & Krathwhol, D.R. (eds.). (2001). A Taxonomy for Learning, Teaching, and Assessing: A
Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.
Angelo, T., (1999). Doing Assessment As If Learning Matters Most. 1999 AAHE Bulletin, vol. 51, no. 9, pp. 3-6.
www.aahe.org
California State University, (2013). Quality Online Learning and Teaching. Center For Teaching and Learning.
www.calstate.edu
Dika, S. and Singh, K.(2002). Applications of Social Capital in Educational Literature: A Critical Synthesis.
Review of Educational Research, 72(1), 31-60.
Finely, A. (2012). Making Progress? What We Know About the Achievement of Liberal Education Outcomes.
(AACU, 2012). www.aacu.org
Hart Research Associates, It Takes More than a Major: Employer Priorities for College Learning & Student
Success (Washington, DC: AAC&U, 2013) LEAP Employer – Educator Compact: Making Quality A Priority As
Americans Go To College. April. 2013 www.aacu.org/leap
References
National Institute for Learning Outcomes Assessment, (2012).
www.learningoutcomesassessment.org
Pew, S, (2007). Andragogy and Pedagogy as Foundational Theory for Student Motivation in Higher
Education. Volume 2, Student Motivation. Insight A Journal of Scholarly Teaching.
Suskie, L. (2009). Assessing Student Learning: A Common Sense Guide. 2nd Edition. Bolton, MA:
Anker Publishing Company Inc.
Tinto, V. (2006-2007). Research And Practice Of Student Retention: What Next?. Journal of
College Student Retention, Vol. 8(1) 1-19.
Value Rubrics Project: Valid Assessment of Learning in Undergraduate Education, (Fall
2011/Winter 2012) Peer Review. Assessing Liberal Education Outcomes Using Value Rubrics.
www.aacu.org.
Varner, M & Housel, S. (2012). “From Confusion to Conversation: Improving Student Learning and
Assessment One Step at a Time,” at the 2012 National Assessment Institute, Indiana University
Purdue University Indianapolis, Indiana, October 28-30.
Varner, M., (2012). Assessment Best Practices: Assessment Summit; Faculty Development, PSCJ,
RSU.
Varner, M. (2012). “Building a Culture of Service: Civic Engagement/Service Learning,” at the
National Conference on Learner-Centered Teaching, Sponsored by Langston University, Purdue
University, US Department of Agriculture and the US Department of Education, Tulsa,
Oklahoma, April 3-4.