Transcript Outcomes of the Majors: On Being Deliberate and Explicit

Origins and Development of the Lumina
“Degree Qualifications Profile (DQP)”
Peter Ewell
National Center for Higher Education Management Systems
(NCHEMS)
SHEEO Annual Meeting
July 13, 2012
To increase the proportion of
Americans with high-quality
degrees and credentials to
60 percent by 2025.
How Is Quality Reflected in Goal 2025?
 Increasing the number of degrees requires
attention to quality and transparency
 Learning is valued by employers
 High-quality degrees are essential element to a
knowledge economy
•.
Background
 Qualifications Frameworks in Many Other Countries
 Bologna Process Common Outcomes Benchmarks (e.g.
“Dublin Descriptors”)
 AAC&U LEAP Outcomes Statements and Rubrics
 State-Level Outcomes Frameworks in U.S. (e.g. UT, WI,
CSU, ND, VA)
 Some Alignment of Cross-Cutting Abilities Statements
Among Institutional Accreditors
Lumina Degree Profile
• Three Degree Levels: Associate, Bachelor’s, and Master’s
• Five Learning Areas: Specialized Knowledge,
Broad/Integrative Knowledge, Intellectual Skills, Applied
Learning, and Civic Learning
• Framed as Successively Inclusive Hierarchies of “Action
Verbs” to Describe Outcomes at Each Degree Level
• Intended as a “Beta” Version, for Testing, Experimentation,
and Further Development Beginning this Year
How the Panel Approached Its Work
• Wide Literature Review (Other National QFs and
Outcomes Adopted by U.S. Colleges and Universities)
• Emphasis on Application and Integration (as
Distinctively “American” Undergraduate Attributes)
• But Confined to Things that Institutions Actively Teach
(Therefore Few Values or Attitudes Included)
• Emphasized Civic Learning as an Area Particularly
Important for a Functioning Democracy
An Example: Communication Skills
Associate Level: The student presents substantially
error-free prose in both argumentative and narrative forms
to general and specialized audiences
Bachelor’s Level: The student constructs sustained,
coherent arguments and/or narratives and/or explications
of technical issues and processes, in two media, to
general and specialized audiences
Master’s Level: The student creates sustained,
coherent arguments or explanations and reflections on his
or her work or that of collaborators (if applicable) in two or
more media or languages, to both general and specialized
audiences
An Example: Engaging Diverse Perspectives
Associate Level: Describes how different cultural perspectives
would affect his or her interpretations of prominent problems in
politics, society, the arts, and/or global relations
Bachelor’s Level: Constructs a cultural, political, or technological
alternative vision of either the natural or human world, embodied in
a written project, laboratory report, exhibit, performance, or
community service design; defines the distinct patterns in this
alternative vision; and explains how they differ from current realities
Master’s Level: Addresses a core issue in his/her field of study
from the perspective of either a different point in time, or a different
culture, political order, or technological context, and explains how
the alternative perspective contributes to results that depart from
current norms, dominant cultural assumptions, or technologies—all
demonstrated through a project, paper, or performance
What Happens Next?
 Growing Number of Lumina-Funded Follow-On
Projects Designed to “Test Drive” the DQP (HLC,
WASC, SACS, CIC, AASCU, AAC&U, etc.) Involving
More than 120 Institutions
 Other Efforts Consistent with DQP that are Not
Directly Funded by Lumina (e.g. MA)
 Results of Projects (and other efforts) Will be Used to
Refine the DQP Further in 2014.
Testing The DQP :
The Context, Opportunities, and
Challenges
Debra Humphreys
Association of American Colleges and Universities
[email protected]
www.aacu.org
National Context
National Context
1) Increasing Demand for Public Higher Education
2) Declining Funding Sources
3) Declining Public Confidence and Skepticism about
Value
4) Lack of Clarity about What a Degree Actually
Represents
5) Lack of Solidly Researched Assessment Tools for
Full Range of Competencies at High Levels
DQP and Quality Collaboratives Respond Directly to 4
and 5.
The Quality Collaboratives Project and the DQP Also
Respond to Long-term Trends:
The World is Demanding More
There is a demand for more numbers of college
educated workers.
There is a demand for engaged and informed
citizens, who are knowledgeable about themselves and
the world around them
There is also a demand that those educated workers
and citizens have higher levels of learning and
knowledge, and some new and different skills and
abilities.
Quality Collaboratives
QC is a three-year project funded by Lumina Foundation and the
Hewlett Foundation and based on the DQP.
• 9 states and 20 individual campuses working in state systems
and/or 2-year/4-year partnerships
• Campuses and systems are testing the use of the DQP to map,
assess, and document student achievement across levels of
learning .
• Project outcomes: new assessment frameworks and approaches;
policy frameworks to track student progress and achievement;
and models of faculty development and leadership.
Opportunities
• DQP builds on research about how people learn and is designed
to ensure both knowledge and ability to integrate and apply
• DQP builds on LEAP and other efforts that have credibility
among faculty and includes outcomes already endorsed by
accreditors and employers
• DQP Combines Vision with Strategy—Focuses on what students
actually are required to do (projects, papers, research)
Opportunity to Do Assessment Right and Increase Achievement
and Completion at the Same Time!
Challenges
•
Progress on articulating outcomes, but not enough connecting of
outcomes explicitly to gen ed or major requirements;
•
We aren’t building faculty capacity/support fast enough;
•
Assessment approaches still localized rather than systemic; more
disciplinary than cross-cutting;
•
Progress on assessment, but methods are still very local and reporting
of results too confusing for public;
•
Pressures to advance completion at reduced costs present dangerous
opportunity to skip over defining quality or assessing achievement in
meaningful ways.
What is the Role of SHEEOs?
Becoming Facilitators as well as Regulators
•
Use participation in DQP projects as opportunity to facilitate
intercampus collaboration about common goals and assessment
approaches (statewide mtgs, task forces, working groups);
•
Draw positive attention to the work of faculty and academic
administrator change agents (focus on mid-level campus leaders);
•
Use participation in nat’l initiatives to draw public attention to the
need for broad learning outcomes in the knowledge economy;
•
Use or modify LEAP VALUE Rubrics to provide statewide
frameworks for assessing student work.
SHEEO Annual Meeting | July 13, 2012
Assessing Student Learning:
The Work in Progress in Massachusetts
Richard M. Freeland, Commissioner
The Work in Progress in Massachusetts
The Challenge

Develop a system-level program of
learning outcomes assessment that:
Allows Massachusetts to compare
educational results with other states
Does not depend on
standardized testing
The Work in Progress in Massachusetts
Why This Is Important
The Accountability Movement
The Educational Imperative
The Problems with Standardized Tests
The Work in Progress in Massachusetts
The Context

Part of
A comprehensive initiative to
strengthen public higher education
in Massachusetts

Relationship of college completion
agenda to student learning agenda
The Work in Progress in Massachusetts
The Approach

Working Group on Student Learning
Outcomes and Assessment

Two phases of work:
 Campus-level Assessment: 2009–2010
 System-level Assessment: 2010–2011
The Work in Progress in Massachusetts
Working Group Report: Key Principles

Acknowledge pre-eminent role of faculty

Must become integral to teaching/learning process

Should include common elements but allow
room for variation

Must be feasible for wide use in terms of cost and
faculty workload

Must be useful for program improvement

Must be useful for communicating results to public
The Work in Progress in Massachusetts
The Concept
 Grounded in embedded assessment
 Value of LEAP Framework and
VALUE Rubrics
 Allow for use of other measures
(direct and indirect assessments)
The Work in Progress in Massachusetts
Implementing the Concept

In Massachusetts:
 Becoming a LEAP State
 Building a collaborative
planning structure
 Acquiring external support
The Work in Progress in Massachusetts
Implementing the Concept

State Partners:
 The Boulder Conference
 Support of SHEEO and AAC&U
 Crafting a multi-state compact
 Ongoing partnership with SHEEO
Questions