Comprehensive Assessment Reports
Download
Report
Transcript Comprehensive Assessment Reports
Comprehensive Assessment
Reports
1
Fred Trapp, Ph.D.
Administrative Dean, Institutional Research/Academic Services (Retired)
Long Beach City College
Cambridge West Partnership, LLC
Robert Pacheco, Ed.D.
Director Of Institutional Planning, Research and Resource Development
Barstow College
Outcomes for the Session
2
The participant will be able to
Describe the comprehensive assessment report concept.
Locate best practice examples from other colleges
through web links.
Discuss how the comprehensive report idea can be used
as part of the institution’s learning process and as a
means by which the institution provides quality assurance
to the public.
Indicate national trends and efforts of consortia/national
organizations to provide quality assurance about student
experiences and learning outcomes.
Please hold questions until the end.
3
Curriculum Map
4
Introduction
(Purpose and
Goals)
Where Can I
Go?
How Does the
Process Meet
Our Pledge to
the
Communities
We Serve?
What Might
Comprehensive
Reports “Look
Like”?
How Do the
Reports
Improve
Institutional
Learning?
5
6
7
ACCJC Institutional Effectiveness Rubric
8
Part III Student Learning Outcomes
Proficiency
Comprehensive assessment reports exist and are completed on a
regular basis.
What Did We Look At, With Whom Did We Consult?
9
ACCJC. Institutional Effectiveness Rubric
ACCJC. 2002 Standards
ACCJC. Themes
ACCJC. Guide to Evaluating Institutions
Professional literature
Efforts of national groups/institutes
Institutional web sites, listservs and colleagues
Guiding Questions
10
How can the report writing experience:
Help faculty explore the student learning process?
Determine the extent to which the curriculum is
working?
Where can time, energy and/or money be allocated for
continuous improvement in learning?
Exploit the writing process and dialogue about results
to gain broader institutional learning experiences?
Help meet our quality assurance pledge to the
community?
Illustration Selections
11
Council for Higher Education Accreditation (CHEA)
Annual Award for Outstanding Institutional Practice in
Student Learning Outcomes
Demonstrated commitment to & developed highly effective
practice to use SLO assessment
Willingness to share the practices they developed
Selection committee
Selection criteria
Articulation & evidence of outcomes
Success with regard to outcomes
Information to the public about outcomes
Using outcomes for improvement
Illustration selections (continued)
12
Cited by scholars and peers
Schools with assessment work cited in scholarly books and
articles
Schools with assessment work selected for presentation at
national conferences
Web presentations publicly available for you to ease drop upon
Prominent national movements/initiatives regarding
learning outcomes & assessment documentation
(including public quality assurance)
What Might Be Included?
13
Assessment focus- course, program, general ed, etc.
What outcomes were assessed?
How and when were they assessed?
Who was assessed?
What were the results?
Who reviewed the results, made sense of the them
and what conclusions were reached?
What are the implications for practice and/or policy
or future assessment work?
CC of Baltimore County (MD)
Course-level Reporting
14
Middle States Commission on Higher Education
Community College Futures Assembly, Bellwether
Award, 2008
Instructional Programs & Services for High Impact Course
Level Assessment
CHEA award winner, 2006
Institutional Progress in Student Learning Outcomes
National Council on Student Development (NCSD)
Exemplary Practice Award Winner
CC of Baltimore County (MD)
Course-level Reporting
15
Projects are at least three semesters long
Individual and high-impact courses (all sections) included
Project proposal by a faculty group
Measurable objectives
External review & approval in selecting methods/instrument &
analyzing results. Benchmarking should be included if possible.
Controls and sample size considered.
Course improvements based on data analysis
Reassessment expected
Results/report shared across the college and web posted
CC of Baltimore County (MD)
16
Learning Outcomes Assessment Final Report
Template
1.
2.
3.
4.
5.
Design & proposal for the LOA project
Implementation of design & data collection
Redesign of the course to improve student learning
Implementation of course revisions & reassessment of
student learning
Final analysis and results
eavesdropping
http://www.ccbcmd.edu/loa/CrseAssess.html
Two-page executive summaries available
CC of Baltimore County (MD)
17
CHEM 108
An initial “failure” turned to success and collaboration with a
four-year school
HLTH 101
Addressing an achievement gap with professional development
and increased communication with students
CRJU 101 and 202
Statewide group assessment development effort and creativity
in the interventions used
Program-level Reporting
18
North Central Association of Colleges & Schools,
Higher Learning Commission
CHEA award winner, 2008
Institutional Progress in Student Learning Outcomes
Hocking College (OH)
19
Learning outcomes data collected in a student E-
portfolio
Direct internal and external evidence (1 to 10 measures)
Indirect evidence (1 to 4 measures)
Evidence drawn from samples of student work for
faculty to apply an agreed upon holistic rubric
Eight general education outcomes (student success skills)
Discipline-specific exit competencies or outcomes
See the Hocking wheel
20
Hocking College (OH)
Program-level Report
21
eavesdropping
Cloud
reference, not college URL as links are
broken there
Various reports available in each program profile
Curriculum
matrix
Criteria statements (exit competencies)
Instructional Program Outcomes (assessment plan)
Trend Charts for performance criteria
Hocking College (OH)
Program-level Report
22
Example reports and analysis
Culinary Arts Technology
Forestry Management Technology
Nursing Technology
Mesa College (AZ)
General-Education Reports
23
North Central Association of Colleges & Schools,
Higher Learning Commission
CHEA Award winner, 2007
Institutional progress in Student Learning Outcomes
Mesa College
General Education Report
24
Multiple outcomes assessed
Annually
Annual Report elements
Executive
Summary
Methodology
Results & observations (GE & workplace)
Indirect measures findings
Appendices of past results
Mesa Community College (AZ)
25
General education studies completed 2007-08; 2005-06
Numeracy
Scientific inquiry
Problem solving/critical thinking (2008-09)
Information literacy
Workplace skills (CTE) (2009-10)
General education studies completed 2006-07; 2004-05
Arts & humanities
Cultural diversity
Oral communication
Written communication
Mesa College (AZ)
General Education Reports
26
eavesdropping
http://www.mesacc.edu/about/orp/assessment/index.html
Annual
Nine
reports and summaries available
years of history and experience
14 years of assessment work
Capital CC (CT)
General-Education Reports
27
New England Association of Schools and Colleges,
Commission on Institutions of Higher Education
Cited in the Art and Science of Assessing General
Education Outcomes: A Practical Guide (AAC&U,
2005)
eavesdropping
http://www.ccc.commnet.edu/slat/
Annual reports and summaries available
Portland CC (OR)
General Education Reports
28
Northwest Accrediting Commission
eavesdropping
http://www.pcc.edu/resources/academic/learningassessment/
One general education theme a year
Learning Assessment Focus for 2009-10- Critical Thinking &
Problem Solving
Physical Science, Geology and General Science
Bioscience Technology
Management and Supervisory Development
Culinary Assistant Program
Truman State University (MO)
Various Reports
29
Southern Association of Schools and Colleges, Commission
on Colleges
eavesdropping
Assessment work began in 1970
http://assessment.truman.edu/
Assessment Almanac- A compilation of results from each year’s
assessment work (versions from 1997 to 2009 are posted)
General Education outcomes are assessed in the context of the
major field of study
Portfolio Project- required of all seniors to show best work assessed
by faculty for the nature & quality of the liberal arts and sciences
learning outcomes (versions from 1997 to 2008 are posted)
Authorship
30
Course-level, program-level & general education
Teaching faculty study team with technical assistance from
institutional research or assessment committee
No “lone ranger” authors
Institutional summary
Academic administrator with assistance from
Learning outcomes coordinator or assessment committee
Compilation of work accomplished in one or two academic
years across the institution
Putting the Reports to “Work”
31
To all affected participants
Campus committees
Curriculum, assessment, resource allocation group, unit
(department) leadership, general academic and college
leadership
Campus fairs, brown-bag lunches, poster sessions for
information sharing
Faculty professional development programs
Accreditation self-study committee work groups
Local governing board presentation
College web site for the public
Reports & a Learning Organization
32
Learning organization
Environment that promotes a culture of learning
Individual & group learning enriches & enhances the
organization as a whole
Systematic problem solving using data for decisions
Learning from experiences in assessing organizational
performance
Comparing yourself to others (benchmarking) and borrowing ideas
Adriana Kezar ed. Organizational Learning in Higher Education New Directions for
Higher Education. No. 131, Fall 2005. Jossey-Bass.
Reports as Institutional Learning
& Resource Allocation
33
The assessment data sense-making process = a faculty learning
experience
Linking results to future interventions = a learning experience
Using results to inform an intervention, then reassess = a
learning experience (accomplished one or more terms later)
Reference for future assessment work and other groups on
campus
Reports as Institutional Learning
& Resource Allocation
34
Hocking College (OH)
Annual summary
Improvements in the program in the previous year brought on by
study of assessment results
Expenditures of time, money & materials for the assessment
program
Requests for assistance in implementing assessment
Recommendations for altering the institution’s assessment process
Transition from evaluating individual students to assessing
groups of students & the curriculum experience
Reports as Institutional Learning
& Resource Allocation
35
Community College of Baltimore County (MD)
Learning Outcomes Assessment Advisory Board
Links findings in assessment reports to other college-wide
initiatives and professional development opportunities
Use of assessment processes and (findings) results
Challenged faculty to reexamine prompts used in assessment
Clarity of written prompt & extent it supports program goals
Common assignment options and common rubric increases faculty
understanding and buy-in
Builds faculty unity toward common goals
Public web page enhances communication and accessibility to
information
Reports as Institutional Learning
& Resource Allocation
36
CHEA award winner, 2010
Institutional Progress in Student Learning Outcomes
Reports as Institutional Learning
& Resource Allocation
37
Feedback & recognition
Feedback rubric for annual assessment reports
Conversations
and action
Collection and analysis of evidence
Implementation of findings
Recognition (achievement & excellence)
Reports as Institutional Learning
& Resource Allocation
38
Seal of Assessment Achievement
Academic programs earning this recognition have
demonstrated in their annual report that
learning outcomes have been assessed through two or more
methods, and
findings have been discussed among
the faculty.
Reports as Institutional Learning
& Resource Allocation
39
Seal of Assessment Excellence
Academic programs earning this recognition have
demonstrated
a thorough implementation of assessment plan(s)
the reporting of meaningful assessment data
the discussion of findings among faculty
and perhaps students
the use of findings to showcase
student achievements and
to make curricular adjustments.
Reports as Institutional Learning
& Resource Allocation
40
Mesa College (AZ)
Results Outreach Committee
Promotes use of outcomes data in relation to faculty
development, pedagogy and academic climate
Groups of faculty offer a proposal for summer or academic year
work above the course level
Resulting report placed on the web and used for campus
discussion and action
Report as Quality Assurance
41
NILOA
2010 Webscan report Exploring the Landscape: What
Institutional Websites Reveal About Student learning
Outcomes Assessment Activities
2010 Connecting State Policies on Assessment with
Institutional Assessment Activity
2011 Transparency Framework (online)
eavesdropping
www.learningoutcomeassessment.org
Report as Quality Assurance
42
Promising Vehicles for Expanding Information to
the Public
Brief
narrative report from annual assessment
reports
Simple statistical reports on learning
outcomes or surveys
Best practices stories supported by
assessment
Peter Ewell. Accreditation & the Provision of Additional Information to the Public about Institutional
and Program Performance, CHEA, May 2004
Quality Assurance to the Public
43
Voluntary System of Accountability
APLU & AASCU (520 public institutions, award 70% of
bachelor’s degrees in the US each year)
College Profile (includes learning outcomes & links to campus)
Proactive initiative to document learning gains and average
institutional scores (choice of 3 national instruments)
Proactive initiative to illustrate unique campus learning outcomes
assessment work
Promoting a learning institution
eavesdropping
http://www.collegeportraits.org/
Quality Assurance to the Public
VSA Example, CSU Monterey Bay
44
http://www.collegeportraits.org/map
Cal Poly Pomona
http://www.collegeportraits.org/CA/CPP/learning_outcomes
Learning Assessment Examples
Quality Assurance to the Public
45
National Association of independent Colleges and
Universities
Assessment programs on campus tied to institution’s mission
eavesdropping
http://www.naicu.edu/special_initiatives/accountability/Student
_Assessment/id.514/default.asp
Pepperdine University
http://services.pepperdine.edu/oie/learningoutcomes/learning-outcomes-overview.aspx
Where Can I Go?
Resource
46
Filesanywhere.com
http://www.filesanywhere.com/fs/v.aspx?v=8a69668b5c6773a96f6d
Contacts & Questions
47
Robert Pacheco (Barstow College)
[email protected]
Fred Trapp (Cambridge West Partnership)
[email protected]
Questions and Comments
Session Evaluation
Outcomes for the Session
48
The participant will be able to
Describe the comprehensive assessment report concept.
Locate best practice examples from other colleges
through web links.
Discuss how the comprehensive report idea can be used
as part of the institution’s learning process and as a
means by which the institution provides quality assurance
to the public.
Indicate national trends and efforts of consortia/national
organizations to provide quality assurance about student
experiences and learning outcomes.