Transcript Support Model Rollout

Blackboard Outcomes System:
Enabling Continuous Improvement
1
I. Critical Issues in Higher Education:
Accountability, Accreditation, and Need for Insight
Key Issues
2
Increasing demands for accountability
Commission on the Future of Higher Education
“Colleges and universities must become more transparent about cost, price
and student success outcomes and must willingly share this information with
students and families.”
• Creation of “consumer-friendly information database on higher
education” to include student learning outcomes.
• Tie federal funding to institutions
sharing performance data.
National Association of State Universities and Land Grant
Colleges recommends Voluntary System of Accountability (VSA)
using measures that:
•
•
•
•
3
Improve educational outcomes of students
Focus on value added
Are transparent or publicly disclosed
Are compared against an institution’s past performance
State Challenges
The Pappas Report
July 2006
“A recent (June 20, 2006) report by Education Research Center utilizes a
Cumulative Population Index to estimate the probability that a student in the
9th grade will complete high school on time with a regular diploma. North
Carolina was at 66%, which is below the national average.”
“The increasing diversity will further challenge preparation as there are
substantial educational achievement gaps among different socio-economic
groups. NCCS reports that…12.76% of white enrollment were graduates
in 2004-5, compared to 9.68% Black and 9.03% of Hispanic
enrollment.”
4
Accreditation focus on learning outcomes
SACS Standard 3.3.1. The institution Identifies expected
outcomes for its
educational programs
and its administrative
and educational support
services.
Establishes and evaluates
program and learning
outcomes
Assesses whether it
achieves these outcomes
CYCLE
IMPROVE
Provides evidence of
improvement based on analysis
based on analysis of those
results
 Ongoing (not just once each accreditation period)
 Institution-wide (academic units, administrative units)
SACS Standard 2.5
5
II. Drive for Continuous Improvement
6
Higher Education Wants to Understand its Effectiveness
1. What specific knowledge, abilities, and experiences should the
graduates of each of our programs possess?
2. How do we see (measure) these outcomes in our students –
developing from entry to exit and then demonstrated postgraduation?
3. How do institutional services and policies affect these outcomes?
4. Do specific student attributes (e.g. race, gender, nationality,
financial aid status) affect achieving these outcomes?
5. How does course sequencing, pedagogy, co/extra-curricular
activities, and technology affect these outcomes?
6. What is the location of our longitudinal data supporting the
answers to these questions?
7. Do stakeholders at our institution agree on the answers?
7
Cycle of assessment in higher education
• Plan program
learning outcomes
• Plan course
outcomes
• Define
measurement
methods
• Establish rubrics
• Discussions
• Survey employers
• Interview graduates
• Collect examples of
student work
• Run course
evaluations
• Construct portfolios
CYCLE
IMPROVE
• Review survey results
• Look for demographic trends
on data
• Discussions
• Develop program change
recommendations
Drive for continuous
improvement in institutional effectiveness:
• Academic outcomes - teaching and learning
• Administrative outcomes
8
Multiple levels of assessment
Institution
Program
Course
9
– Strategic planning, institutional
research, accreditation self studies,
general education
– Program assessment,
curriculum planning,
discipline-specific
accreditation, course design
– Course development,
testing, assignments,
portfolios, course redesign
III. Obstacles to effective assessment and
continuous improvement
10
Obstacles to effective assessment and continuous
improvement
 Lack of visibility into assessment data and knowledge
– Missed opportunity to learn and improve
– No repository for data over time
 Fragmented technology applications
– Disconnected individual systems for planning, measuring, and
improving
 Lack of connection between different assessment
processes at multiple levels of institution
– Duplication and overlap of activities and effort
 Current processes are costly and time consuming
– Paper and pencil
11
Fragmented and Disconnected Technology Applications
Assessment
Management
• Institutional and
Department Goal /
Objective Management
• Executive Dashboard
• Curriculum Planning
• Project Management
Tools
• 3rd Party Standards
• Document Management
• Communication and
Collaboration Tools
12
Enterprise
Test & Survey
Student Outcome
Planning/
Tracking
Institutional
Research
• Enterprise Test
• Evaluation Portfolios
• Export
• Enterprise Survey
• Learning Content &
Activity Alignment
• Evaluation
DataWarehouse
• Executive Dashboard
and Scorecards
• Bb LS Extraction,
Transformation,
Loading (ETL)
Institution Level
• Enterprise Course
Evaluation
• 3rd Party Instruments
• Course Test Aggregate
Reporting
Program Level
• Bubble Sheets (OMR)
• Secure Testing
• Student Performance
Monitoring
• SIS / SRS ETL
• Early Warning Systems
• Correlations
• Lesson Planning
• Analytics Engine
Classroom Level
• Ad-hoc Reporting
Manager
IV. Blackboard Outcomes SystemTM
13
A New Member of the Blackboard Academic Suite™
14
Co-Designed with Leading Partner Institutions
15
Guided by National Experts
 Mary Allen, Ph.D.
– Professor Emeritus of Psychology at California State University–
Bakersfield
– Former Director, California State University Institute for Teaching and
Learning.
– Author of “Assessing Academic Programs in Higher Education” and
“Assessing General Education Programs”
 Bob Mundhenk, Ph.D.
– Former Director of Assessment and Senior Scholar, American Association
of Higher Education
– Former Dean, Director of Assessment, VP of Student Affairs
 Ephraim Schechter, Ph.D.
– Former University Director of Assessment, NC State University
– Directed assessment components of North Carolina State University's
2004 SACS reaffirmation.
– Maintains the well-known meta-list Internet Resources for Higher
Education Outcomes Assessment
16
Expected Outcomes and Benefits
 Expected Outcomes
– Enable creation of “systemic and systematic” assessment processes
– Connect institutional assessment to program assessment to classroom
assessment
– More visibility of data and knowledge about outcomes
– Better evidence-based decision making
• Confidence in budgetary justification
 Benefits
–
–
–
–
17
Better learning
More efficient and effective assessment activities
Easier accreditation
Cost savings
Uses for the Blackboard Outcomes System
 Institution
–
–
–
–
Capturing Data for Institutional Research
Measuring Success of Strategic Plan Implementation
Managing Regional Accreditation Self Study
Providing Government Reporting
 Program
– Capturing Data for Program Reviews and Assessments
– Showcasing Curriculum Planning
– Managing Discipline-specific Accreditation
 Classroom
– Organizing Course Development
– Capturing Data for Course Redesign
– Improving Face-to-Face, Blended, and e-Learning Instruction
18
V. Key Capabilities
19
Provide critical information to make critical decisions
Reports
 Make evidence-based
decisions to improve
academic and
administrative
outcomes
 Inform analysis with
data:
 From across
departments
 Linking planned
outcomes and
measured outcomes
2005
20
2006
Measure student learning outcomes
Rubrics
 Employ rubrics to
communicate shared
expectations and measure
student learning outcomes
21
Encourage active and reflective learning
Outcomes Portfolios
 Apply rubrics to student work in
Outcomes Portfolios
 Measures course and program
learning outcomes
22
Collect and evaluate student work
Outcomes Artifact
 Collect student work for
evaluation
 Evaluate student work
 Apply rubric to measure
student learning
outcomes
23
Promote a holistic view of the curriculum
Curriculum maps
 Identifies gaps
and redundancies
in curriculum
 Inform other
departments
about
relationships
 Document
relationships
between program
outcomes and
course outcomes
24
Understand your stakeholders
Surveys
 Learn from oncampus and offcampus
stakeholders via
this email-based
solution
 Align questions
with standards,
goals, and
objectives
 Built-in flexible
analysis and
reporting
25
Reduce time and cost of assessment and accreditation
 Consolidate investments in numerous
disconnected technology with
institution-wide access to key
capabilities:
 Enterprise Survey
 Course Evaluation
 Outcomes Portfolio
 Rubric Builder
 Curriculum Mapping
Content Management
 Streamline documentation of
assessment processes for regional
and specialized accreditation
Improvement Initiatives
 Manage and track progress for
numerous assessment process
26
Questions and Answers
Contact Information: Karianne Naughton
[email protected]
28