Transcript Comprehensive Assessment Reports

Comprehensive Assessment
Reports
1
Fred Trapp, Ph.D.
Administrative Dean, Institutional Research/Academic Services (Retired)
Long Beach City College
Cambridge West Partnership, LLC
Robert Pacheco, Ed.D.
Director Of Institutional Planning, Research and Resource Development
Barstow College
Outcomes for the Session
2
 The participant will be able to
Describe the comprehensive assessment report concept.
 Locate best practice examples from other colleges
through web links.
 Discuss how the comprehensive report idea can be used
as part of the institution’s learning process and as a
means by which the institution provides quality assurance
to the public.
 Indicate national trends and efforts of consortia/national
organizations to provide quality assurance about student
experiences and learning outcomes.

Please hold questions until the end.
3
Curriculum Map
4
Introduction
(Purpose and
Goals)
Where Can I
Go?
How Does the
Process Meet
Our Pledge to
the
Communities
We Serve?
What Might
Comprehensive
Reports “Look
Like”?
How Do the
Reports
Improve
Institutional
Learning?
5
6
7
ACCJC Institutional Effectiveness Rubric
8
 Part III Student Learning Outcomes
 Proficiency

Comprehensive assessment reports exist and are completed on a
regular basis.
What Did We Look At, With Whom Did We Consult?
9
 ACCJC. Institutional Effectiveness Rubric
 ACCJC. 2002 Standards
 ACCJC. Themes
 ACCJC. Guide to Evaluating Institutions
 Professional literature
 Efforts of national groups/institutes
 Institutional web sites, listservs and colleagues
Guiding Questions
10
 How can the report writing experience:
Help faculty explore the student learning process?
 Determine the extent to which the curriculum is
working?
 Where can time, energy and/or money be allocated for
continuous improvement in learning?
 Exploit the writing process and dialogue about results
to gain broader institutional learning experiences?
 Help meet our quality assurance pledge to the
community?

Illustration Selections
11
 Council for Higher Education Accreditation (CHEA)
 Annual Award for Outstanding Institutional Practice in
Student Learning Outcomes
Demonstrated commitment to & developed highly effective
practice to use SLO assessment
 Willingness to share the practices they developed

Selection committee
 Selection criteria

Articulation & evidence of outcomes
 Success with regard to outcomes
 Information to the public about outcomes
 Using outcomes for improvement

Illustration selections (continued)
12
 Cited by scholars and peers
 Schools with assessment work cited in scholarly books and
articles
 Schools with assessment work selected for presentation at
national conferences
 Web presentations publicly available for you to ease drop upon
 Prominent national movements/initiatives regarding
learning outcomes & assessment documentation
(including public quality assurance)
What Might Be Included?
13
 Assessment focus- course, program, general ed, etc.
 What outcomes were assessed?
 How and when were they assessed?
 Who was assessed?
 What were the results?
 Who reviewed the results, made sense of the them
and what conclusions were reached?
 What are the implications for practice and/or policy
or future assessment work?
CC of Baltimore County (MD)
Course-level Reporting
14
 Middle States Commission on Higher Education
 Community College Futures Assembly, Bellwether
Award, 2008

Instructional Programs & Services for High Impact Course
Level Assessment
 CHEA award winner, 2006
 Institutional Progress in Student Learning Outcomes
 National Council on Student Development (NCSD)
Exemplary Practice Award Winner
CC of Baltimore County (MD)
Course-level Reporting
15



Projects are at least three semesters long
Individual and high-impact courses (all sections) included
Project proposal by a faculty group
Measurable objectives
 External review & approval in selecting methods/instrument &
analyzing results. Benchmarking should be included if possible.
 Controls and sample size considered.




Course improvements based on data analysis
Reassessment expected
Results/report shared across the college and web posted
CC of Baltimore County (MD)
16
 Learning Outcomes Assessment Final Report
Template
1.
2.
3.
4.
5.
Design & proposal for the LOA project
Implementation of design & data collection
Redesign of the course to improve student learning
Implementation of course revisions & reassessment of
student learning
Final analysis and results
eavesdropping



http://www.ccbcmd.edu/loa/CrseAssess.html
Two-page executive summaries available
CC of Baltimore County (MD)
17
 CHEM 108
 An initial “failure” turned to success and collaboration with a
four-year school
 HLTH 101
 Addressing an achievement gap with professional development
and increased communication with students
 CRJU 101 and 202
 Statewide group assessment development effort and creativity
in the interventions used
Program-level Reporting
18
 North Central Association of Colleges & Schools,
Higher Learning Commission
 CHEA award winner, 2008

Institutional Progress in Student Learning Outcomes
Hocking College (OH)
19
 Learning outcomes data collected in a student E-
portfolio


Direct internal and external evidence (1 to 10 measures)
Indirect evidence (1 to 4 measures)
 Evidence drawn from samples of student work for
faculty to apply an agreed upon holistic rubric


Eight general education outcomes (student success skills)
Discipline-specific exit competencies or outcomes
See the Hocking wheel
20
Hocking College (OH)
Program-level Report
21
 eavesdropping
 Cloud
reference, not college URL as links are
broken there
 Various reports available in each program profile
 Curriculum
matrix
 Criteria statements (exit competencies)
 Instructional Program Outcomes (assessment plan)
 Trend Charts for performance criteria
Hocking College (OH)
Program-level Report
22
 Example reports and analysis
 Culinary Arts Technology
 Forestry Management Technology
 Nursing Technology
Mesa College (AZ)
General-Education Reports
23
 North Central Association of Colleges & Schools,
Higher Learning Commission
 CHEA Award winner, 2007

Institutional progress in Student Learning Outcomes
Mesa College
General Education Report
24
 Multiple outcomes assessed
Annually
 Annual Report elements
 Executive
Summary
 Methodology
 Results & observations (GE & workplace)
 Indirect measures findings
 Appendices of past results
Mesa Community College (AZ)
25
 General education studies completed 2007-08; 2005-06
Numeracy
 Scientific inquiry
 Problem solving/critical thinking (2008-09)
 Information literacy
 Workplace skills (CTE) (2009-10)
 General education studies completed 2006-07; 2004-05
 Arts & humanities
 Cultural diversity
 Oral communication
 Written communication

Mesa College (AZ)
General Education Reports
26
 eavesdropping

http://www.mesacc.edu/about/orp/assessment/index.html
 Annual
Nine
reports and summaries available
years of history and experience
14 years of assessment work
Capital CC (CT)
General-Education Reports
27
 New England Association of Schools and Colleges,
Commission on Institutions of Higher Education
 Cited in the Art and Science of Assessing General
Education Outcomes: A Practical Guide (AAC&U,
2005)
 eavesdropping
http://www.ccc.commnet.edu/slat/
 Annual reports and summaries available

Portland CC (OR)
General Education Reports
28
 Northwest Accrediting Commission
 eavesdropping
 http://www.pcc.edu/resources/academic/learningassessment/
 One general education theme a year
 Learning Assessment Focus for 2009-10- Critical Thinking &
Problem Solving
Physical Science, Geology and General Science
 Bioscience Technology
 Management and Supervisory Development
 Culinary Assistant Program

Truman State University (MO)
Various Reports
29
 Southern Association of Schools and Colleges, Commission
on Colleges
 eavesdropping


Assessment work began in 1970
http://assessment.truman.edu/
Assessment Almanac- A compilation of results from each year’s
assessment work (versions from 1997 to 2009 are posted)
 General Education outcomes are assessed in the context of the
major field of study


Portfolio Project- required of all seniors to show best work assessed
by faculty for the nature & quality of the liberal arts and sciences
learning outcomes (versions from 1997 to 2008 are posted)
Authorship
30
 Course-level, program-level & general education
 Teaching faculty study team with technical assistance from


institutional research or assessment committee
No “lone ranger” authors
 Institutional summary
 Academic administrator with assistance from


Learning outcomes coordinator or assessment committee
Compilation of work accomplished in one or two academic
years across the institution
Putting the Reports to “Work”
31
 To all affected participants
 Campus committees

Curriculum, assessment, resource allocation group, unit
(department) leadership, general academic and college
leadership
 Campus fairs, brown-bag lunches, poster sessions for




information sharing
Faculty professional development programs
Accreditation self-study committee work groups
Local governing board presentation
College web site for the public
Reports & a Learning Organization
32
 Learning organization
 Environment that promotes a culture of learning
 Individual & group learning enriches & enhances the
organization as a whole
Systematic problem solving using data for decisions
 Learning from experiences in assessing organizational
performance
 Comparing yourself to others (benchmarking) and borrowing ideas

Adriana Kezar ed. Organizational Learning in Higher Education New Directions for
Higher Education. No. 131, Fall 2005. Jossey-Bass.
Reports as Institutional Learning
& Resource Allocation
33
 The assessment data sense-making process = a faculty learning
experience
 Linking results to future interventions = a learning experience
 Using results to inform an intervention, then reassess = a
learning experience (accomplished one or more terms later)
 Reference for future assessment work and other groups on
campus
Reports as Institutional Learning
& Resource Allocation
34
 Hocking College (OH)
 Annual summary
Improvements in the program in the previous year brought on by
study of assessment results
 Expenditures of time, money & materials for the assessment
program
 Requests for assistance in implementing assessment
 Recommendations for altering the institution’s assessment process


Transition from evaluating individual students to assessing
groups of students & the curriculum experience
Reports as Institutional Learning
& Resource Allocation
35
 Community College of Baltimore County (MD)
 Learning Outcomes Assessment Advisory Board


Links findings in assessment reports to other college-wide
initiatives and professional development opportunities
Use of assessment processes and (findings) results
Challenged faculty to reexamine prompts used in assessment
 Clarity of written prompt & extent it supports program goals
 Common assignment options and common rubric increases faculty
understanding and buy-in
 Builds faculty unity toward common goals
 Public web page enhances communication and accessibility to
information

Reports as Institutional Learning
& Resource Allocation
36
 CHEA award winner, 2010
 Institutional Progress in Student Learning Outcomes
Reports as Institutional Learning
& Resource Allocation
37
 Feedback & recognition
 Feedback rubric for annual assessment reports
Conversations
and action
Collection and analysis of evidence
Implementation of findings
 Recognition (achievement & excellence)
Reports as Institutional Learning
& Resource Allocation
38
 Seal of Assessment Achievement
 Academic programs earning this recognition have
demonstrated in their annual report that


learning outcomes have been assessed through two or more
methods, and
findings have been discussed among
the faculty.
Reports as Institutional Learning
& Resource Allocation
39
 Seal of Assessment Excellence
 Academic programs earning this recognition have
demonstrated
a thorough implementation of assessment plan(s)
 the reporting of meaningful assessment data
 the discussion of findings among faculty
and perhaps students
 the use of findings to showcase
student achievements and
to make curricular adjustments.

Reports as Institutional Learning
& Resource Allocation
40
 Mesa College (AZ)
 Results Outreach Committee
 Promotes use of outcomes data in relation to faculty
development, pedagogy and academic climate

Groups of faculty offer a proposal for summer or academic year
work above the course level

Resulting report placed on the web and used for campus
discussion and action
Report as Quality Assurance
41
 NILOA
 2010 Webscan report Exploring the Landscape: What
Institutional Websites Reveal About Student learning
Outcomes Assessment Activities

2010 Connecting State Policies on Assessment with
Institutional Assessment Activity

2011 Transparency Framework (online)
 eavesdropping

www.learningoutcomeassessment.org
Report as Quality Assurance
42
 Promising Vehicles for Expanding Information to
the Public
 Brief
narrative report from annual assessment
reports
 Simple statistical reports on learning
outcomes or surveys
 Best practices stories supported by
assessment
Peter Ewell. Accreditation & the Provision of Additional Information to the Public about Institutional
and Program Performance, CHEA, May 2004
Quality Assurance to the Public
43
 Voluntary System of Accountability
 APLU & AASCU (520 public institutions, award 70% of
bachelor’s degrees in the US each year)

College Profile (includes learning outcomes & links to campus)
Proactive initiative to document learning gains and average
institutional scores (choice of 3 national instruments)
 Proactive initiative to illustrate unique campus learning outcomes
assessment work
 Promoting a learning institution


eavesdropping

http://www.collegeportraits.org/
Quality Assurance to the Public
VSA Example, CSU Monterey Bay
44
 http://www.collegeportraits.org/map
 Cal Poly Pomona

http://www.collegeportraits.org/CA/CPP/learning_outcomes
 Learning Assessment Examples
Quality Assurance to the Public
45
 National Association of independent Colleges and
Universities


Assessment programs on campus tied to institution’s mission
eavesdropping
http://www.naicu.edu/special_initiatives/accountability/Student
_Assessment/id.514/default.asp
 Pepperdine University
 http://services.pepperdine.edu/oie/learningoutcomes/learning-outcomes-overview.aspx

Where Can I Go?
Resource
46
 Filesanywhere.com
http://www.filesanywhere.com/fs/v.aspx?v=8a69668b5c6773a96f6d
Contacts & Questions
47
 Robert Pacheco (Barstow College)
 [email protected]
 Fred Trapp (Cambridge West Partnership)
 [email protected]
 Questions and Comments
Session Evaluation
Outcomes for the Session
48
 The participant will be able to
Describe the comprehensive assessment report concept.
 Locate best practice examples from other colleges
through web links.
 Discuss how the comprehensive report idea can be used
as part of the institution’s learning process and as a
means by which the institution provides quality assurance
to the public.
 Indicate national trends and efforts of consortia/national
organizations to provide quality assurance about student
experiences and learning outcomes.
