Learning Assessment and Program Review: Finding Meaningful and Manageable Approaches Don Dodson, Senior Vice Provost Diane Jonte-Pace, Vice Provost for Undergraduate Studies Carol Ann.

Transcript Learning Assessment and Program Review: Finding Meaningful and Manageable Approaches Don Dodson, Senior Vice Provost Diane Jonte-Pace, Vice Provost for Undergraduate Studies Carol Ann.

Learning Assessment and Program Review: Finding Meaningful and Manageable Approaches

Don Dodson, Senior Vice Provost Diane Jonte-Pace, Vice Provost for Undergraduate Studies Carol Ann Gittens, Director, Office of Assessment

Acknowledgments and Purpose Today ACKNOWLEDGING FACULTY CONCERNS:



Faculty are not trained for assessment



Assessment and program review are time consuming

  

Effective assessment strategies are difficult to develop Departments have gathered lots of data but have few answers Faculty need guidance and support from experts PURPOSE TODAY:



Invite dialogue about assessment and program review



Identify ways to make them more manageable and meaningful

Defining Terms



Program Review Refers to periodic evaluation of an academic program based on a self study which examines all aspects of the program, including student learning outcomes.



Assessment Refers to collection and analysis of quantitative or qualitative evidence about what students actually learn in relation to specified objectives.

Levels of Assessment



Course

– – Involves designing and grading course assignments Determines what individual students are learning – – Provides feedback to students about their own learning Informs individual course modifications over time 

Program

– Asks how well students are achieving objectives of the major or minor – – Gathers direct evidence of student learning across program’s curriculum Reflects on program’s contribution to institution-wide learning goals 

Institution

– Evaluates overarching institutional learning outcomes (Core Curriculum)

What are Faculty Being Asked to Do for Assessment?



Grade student work in courses



Include program learning objectives on department syllabi

where appropriate



Complete Annual Learning Outcomes Assessment Report



Include Core learning objectives on new Core syllabi



Provide selected student work from Core courses

– Only 2-3 Core areas per year – – – Only a few selected courses Only a small number of students per course (approximately 5) Invitation to help score student work (not required!)

How Much Flexibility Do We Have Around Assessment?



Faculty determine course-level learning objectives, construct assignments, and grade assignments



Departments create departmental learning goals and objectives



Departments determine meaningful questions for program improvement



Departments design their own assessment plans and data collection strategies (with help, if needed)



Faculty can communicate Learning Objectives for Core through syllabus, website, or other vehicle

What’s the Mandate for Assessment and Program Review? Federal Regulation Accrediting Agencies Board of Trustees University Administration Student and Faculty Success Better Use of Resources Insight into Strengths / Challenges Desire to Improve Programs

Why Does This Feel So Burdensome?



Steep learning curve for individuals, institutions, and higher education generally



Many simultaneous initiatives:

– – Student learning assessment Program Review – – – Core Curriculum WASC self study Specialized accreditation self studies 

Inadequate communication and coordination



Some common correctable mistakes

Common Correctable Mistakes

Assessment

  

Setting more objectives than can reasonably be assessed Setting unclear objectives Assessing the same objective too frequently

 

Using methods that are unnecessarily time-consuming Designing add-on assessments rather than making use of existing course assignments or program activities Program Review

 

Not using program review data provided by Institutional Research Office Not integrating self study for University with self study for professional accreditation Help is available!

Institutional Support



Office of Assessment (Carol Ann Gittens)

– – Assistance refining learning goals and objectives for meaningful assessment Assistance streamlining approaches for manageable assessment – Assistance developing effective and efficient strategies and tools for evaluating student learning – – Assistance with data analysis and interpretation (quantitative and qualitative) Support with all assessment endeavors 

Provost’s Office (Don Dodson)

– Assistance with program review expectations and design 

Dean’s Offices

– Feedback and support for both assessment and program review

Additional Institutional Support



Core Curriculum

– –

Honoraria and summer workshops Stipends for Core scoring parties



Annual Assessment / Program Review

–

Course releases and funding



Faculty Development Workshops

Responses to Top 10 FSC Recommendations

  

Decrease frequency of program review and course goal assessment

 Provost’s Office will propose 8-year program review cycle to Academic Affairs Committee Frequency of course goal assessment is an individual faculty decision Frequency of program goal assessment is a program-level decision Frequency of institutional goal assessment (Core Curriculum) will require participation of faculty in individual courses, typically on a volunteer basis



Have assessment experts help departments streamline process

This service is available through the Office of Assessment and external consultants

 

Hire staff members to do program review

WASC requires full faculty involvement Institutional Research Office since 2005-06 has made relevant data available on compact disc to programs doing self studies. It will try to improve data currency and user training in future.

Responses to Top 10 FSC Recommendations 4.

 

Stop assessments that are not shown to be scientifically valid

Unrealistic standard that would probably stop most course grading as well Office of Assessment can assist with planning and data analysis



Improve structure and clarity of assessment tools given to departments from above

Other than Curriculum Alignment Matrix, departments are not required to use assessment tools given from above

  

Make program review specific to each department

WASC expects some consistency Consistency is desirable because departments may overlook key issues and because it helps administration determine priorities for program support Current guidelines state: “Programs may request exceptions to the official guidelines or calendar if such changes would be conducive to a more effective program review.”

Responses to Top 10 FSC Recommendations



Use existing data rather than creating new tools

To the extent possible, this is always desirable and is consistent with advice given by the Office of Assessment 8.

 

Improve assessment situation by negotiating with WASC

WASC doesn’t negotiate WASC Standards and Criteria for Evaluation are determined through consultation with all accredited institutions in the region, with an eye to both federal regulations and best practices in higher education



Deans read and provide feedback on program review reports

Agree

10. Make program review process work for professional accreditation needs

 Each school currently decides how integrate the two processes in consultation with the Provost’s Office. Advice: do one self study based on accreditation guidelines, with a supplement to address some specific University questions

SCU Web Resources



Office of Assessment: http://www.scu.edu/provost/assessment/



Core 2009: http://www.scu.edu/Core2009



WASC http://www.scu.edu/WASC (select Resources)

Learning Assessment and Program Review: Finding Meaningful and Manageable Approaches Don Dodson, Senior Vice Provost Diane Jonte-Pace, Vice Provost for Undergraduate Studies Carol Ann.

Transcript Learning Assessment and Program Review: Finding Meaningful and Manageable Approaches Don Dodson, Senior Vice Provost Diane Jonte-Pace, Vice Provost for Undergraduate Studies Carol Ann.