Institutional Strategies for Integrating Student Assessment with Academic Planning and Management Marvin W.
Download ReportTranscript Institutional Strategies for Integrating Student Assessment with Academic Planning and Management Marvin W.
Institutional Strategies for Integrating Student Assessment with Academic Planning and Management Marvin W. Peterson & Thomas E. Perorazio The National Center for Postsecondary Improvement, Project 5.2 The University of Michigan Ross A. Griffith, Wake Forest University American Association for Higher Education Assessment Conference Denver, CO - June 25, 2001 Session Purposes Discuss Initial Findings from the Case Study Phase of NCPI Research Project on Institutional Support for Student Assessment Provide Detailed Examples of Two Institutions That Have Integrated Assessment Into their Academic Management Processes Outline the Lessons Learned Applicable to Assessment Practitioners The National Center for Postsecondary Improvement Main Project Web Site http://www.stanford.edu/group/ncpi/ University of Michigan - Project Five http://www.umich.edu/~ncpi/ NCPI Project Area 5: Student Learning and Assessment 5.1 State Policies and Regional Accreditation Practices of Assessment for Student Learning 5.2 Institutional Support for Enhancing Student Assessment and Performance 5.3 Academic Programs and Students: Educational Reform and Innovation in Teaching, Learning, and Assessment Practices Metadata Project: Review of National Student, Faculty, and Institutional Data Resources Project 5.2’s Purpose To Examine How Postsecondary Institutions Support and Promote the Uses and Impacts of Student Assessment Information for Institutional Improvement Overview of Project 5.2 Phase One •Review & Synthesis of the Literature •Developed a Conceptual Framework of Institutional Support Phase Two National Survey of Institutions •Comprehensive Inventory Based on Conceptual Framework •1394 Usable Responses -- 55.1% Phase Three Case Studies of Institutions with Comprehensive Student Assessment Efforts Phase I: Conceptual Framework of Organizational and Administrative Support for Student Assessment Institutional Approach to Student Assessment Institution-wide Support for Student Assessment External Influences on Student Assessment Assessment Management Practices & Policies Assessment Culture and Climate Institutional Context Uses & Impacts of Student Assessment Phase II: National Inventory Research Questions What Approaches to Student Assessment Have Institutions Adopted? What Organizational and Administrative Support Patterns and Assessment Management Practices for Student Assessment Have Institutions Developed? How Have Institutions Used Student Assessment Information and What Impacts Has It Had? How Do Patterns of Student Assessment Approach, Organizational and Administrative Support Patterns and Practices, and Uses and Impacts of Student Assessment Vary by Institutional Characteristics? How Do External Groups Influence Institutional Adoption of Various Measures of and Approaches to Student Assessment? How Are External Influences on Institutional Approaches to and Organizational and Administrative Support Patterns and Practices for Student Assessment Related to the Use and Impact of Student Assessment Information? Phase III: Case Study Research Research Purposes 1) To Examine the Internal Dynamics of Institutional Support for Student Assessment 2) To Identify Institutional Strategies for Supporting, Promoting & Integrating Student Assessment 3) Assess the Culture & Climate for Using Student Assessment Data for Academic Improvement Seven Institutions With Extensive Involvement in Student Assessment •Review of Response Patterns to National Inventory of Institutional Support for Student Assessment (ISSA) •Campus Site Visits by Research Teams •Collection & Synthesis of Documents, & Records •Interviews & Focus Groups With Faculty, Administrators, & Students Integrating Student Assessment with Academic Management for Institutional Improvement Institution’s Academic Management Approach Assessment Purpose Structure Approach Links to Management Process/ Improvement Use in Academic Decision Making Assessment Leadership Support Coordination Discussion of Institutions Brief Institutional Storyline Institution’s Academic Management Approach •Primary Mission & Governance Pattern •Key Academic Management Processes Student Assessment Strategy Purposes of Assessment Assessment Structure Approach to Assessment Leadership, Support, & Coordination Links to Academic Management Processes Utilization in Academic Decision Making Summary - Culture for Student Assessment The Story of Institution Developed a Quality Initiative Based on a Baldridge Model Assessment Adopted as a Means of Academic Improvement Assessment Activity Gradually Integrated into Culture of Quality and Improvement for Many Years Primary Mission & Governance Pattern Vision •Emphasis on Culture of Quality; Sets Goals Statement of Mission •Focuses on Continuous Improvement •Committed to Using Assessment to Enhance Performance Cultural Core Values •Define the University; Drive the Mission & Vision Decision Drivers •Priorities Regarding Essential Functions Approach to Academic Improvement Key Academic Management Processes Strategic Planning Council Seven Step Planning Process KQIs Program Management Office of Assessment, Information, & Analysis Student Affairs Assessment Purpose -- NMSU Initiating Conditions Internal- President’s Focus on Culture of Quality (early 1980s) External- State Mandated Assessment by Governor (late 1980s) President’s Early Efforts Resisted Culture of Quality Focused on Academic Improvement in Undergraduate Education •Adapted From Baldridge National Quality Award Criteria Assessment Structure -- NMSU Formal, University-wide Strategy Uses Baldridge Quality Award Criteria As an Assessment Framework Comprehensive Approach -- Cognitive and Affective Learning College and Post-college Experiences Collected by by Office of Assessment, Information, and Analysis (OAIA) Seven Step Planning Process Assessment Approach -- NMSU Quantitative & Qualitative Data Gathered From Recruitment Until Post-graduation Battery of Cognitive Measures During College Years Regular Collection Schedule Data on Needs or Expectations of Students Using Local and National Surveys & Listening Posts Assessments are required--Explained in UG Catalog Student progress tracked and enforced through restrictions on registration and final transcripts Leadership, Support, & Coordination Leadership Long-term, Continuous Presidential Promotion Strong Support by Provost Through OAIA •Other Groups: •President’s Cabinet & Deans, Baldridge Category Council •Faculty Senate Assessment Committee •Student Success Task Force •Faculty Provide Significant, but Hesitant, Support Coordination •OAIA-- Compiles and Distributes Assessment Data •Formal Grid Details Methods, Uses, & Frequencies for Assessment Activity Links to Academic Management Process/ Improvement Strategic Planning Council Representatives From Administrative, Academic, and Student Affairs Assessment Info Used: •To Evaluate Organizational Performance on Strategic Initiatives (SI) •The Development of New Key Quality Indicators (KQI) for Each SI Annual Departmental Reports & Course Reviews Primary Link Between Action Planning & Assessment Deans Uses Assessment Data to Identify Areas for Improvement Focus Is on Student Performance and Experience Data President's Cabinet Reviews Reports & Provides Feedback to Unit Leaders Links to Academic Management Process/ Improvement OAIA Produces Three Types of Academic Information: 1) Outcomes and Performance; 2) Student and Stakeholder Satisfaction Ratings; and 3) Operational Effectiveness Measurement of of Key Quality Indicators Seven Step Planning Process •To Meet Stakeholder Requirements Data Is Distributed Through Dashboards and Profiles Tracking Systems to Indicate Deviations From Intended Targets Can Be Accessed Through Campus Network Utilization in Academic Decision Making-- NMSU Planning Phase I Review of Mission and Analysis of Progress on Current Goals Phase II Identification of Strategic Initiatives Phase III Action Planning and Development Actions Plans Are Developed to Improve Performance on Key Quality Indicators •Purpose Is to Increase Organizational Performance •Assessment Information Part of KQIs in Dashboards & Profiles Critical to Developing Agenda for Planning Efforts Utilization in Academic Decision Making Program Management •Annual Reports & Reviews •Benchmarking •Stretch Goals Quality Student Affairs-Academic Affairs Alliance Development of Student Life & Experiences Surveys/Studies Links to Educational uses of Assessment Information NMSU Applies Annually for Baldridge Quality Award Summary-- Culture for Assessment Strong, Central Administrative Support for Quality Approach & Assessment Baldridge Quality Criteria Drive Assessment Framework Integration of Assessment into Quality, Planning, and Program Management Processes Data-Driven System & Reports Key Coordination Role Played by OAIA -Commitment to Assessment •As a Means of Visibility for Quality Emphasis •Increasingly for Institutional Improvement The Story of Assessment Data Essential to Cycles of Institutional Planning, Program Review, & Evaluation Assessment Information Used by Decision Makers at All Levels to Evaluate Progress Towards Goals Administration Expects that Assessment Activity Will Occur for the Purpose of Academic Improvement Primary Mission & Governance Pattern No Formal Statements Regarding Assessment WFU Administration believes Assessment is a Necessary Management Tool to Maintain & Improve its Quality and Reputation •Obtaining Data Crucial to Decision Making Approach to Academic Improvement Key Academic Management Processes Institutional Strategic Planning Evaluation Committee Institution-wide Departmental Assessment Review Office of Institutional Research Student Life Assessment Purpose -- WFU Initiating Conditions •Internal- Foundations for Institutional Cycles of Planning Initiated by President & Vice-president •External- Accreditation Focus on Institutional Effectiveness Evaluate Strategic Plan The Plan for the Class of 2000 •Assessment Now a Management Tool in a Continual Process of Planning, Implementation, and Evaluation Program Planning & Evaluation Committees Use Assessment Data •To Address Major Campus Needs •Ongoing Strategic Planning Assessment Structure -- WFU Three Level, Decentralized Process 1) University-wide- OIR- Oversees Assessment of the Process •Administers National & Local Surveys to Students & Faculty •Analyzes, Stores, & Disseminates Data •Works With Evaluation Committee & Other Depts. •Only Office to Have Dedicated Funds for Assessment 2) Departmentally Based- Program Review Process •Primary Planning Unit on Campus •13-month Process, 7 Year Cycle for All Programs •Review of Curriculum & Department/ Program Performance •$2,500 Budgeted for Each Review 3) Student Life- Focuses on its own programs •Focus is on student experiences & satisfaction •Important for Intellectual Climate Planning Assessment Approach-- WFU Primarily Quantitative Focus on Satisfaction & Experiences Little Emphasis on Cognitive or Affective Measures Some Determined by Dept. In Departmental Review Institution-wide Surveys of Students, Faculty, & Alumni National Surveys Administered by Office of Institutional Research Also Some Locally Developed Instruments & Course Evaluations Departments Conduct Own Assessment of Programs & Satisfaction Significant Autonomy & Discretion Granted to Evaluate Their Students Leadership & Support Institutional Support from Executive •President’s and Provost’s Seminars, Conferences, & Retreats •Executive Council & Reynolda Cabinet Receive Reports & Use the Information in Strategic Planning Coordination OIR Has the Primary Role In: •Administering Institution-wide Student Assessments •Coordinating Program Planning •Conducting Data Analysis OIR Shares Its Information With Departments Upon Request Student Life Provides Its Data to Departments Information Systems Office Sends Course Evaluation Results to Department Heads Online OIR Role in Accreditation Allows It to Maintain Focus on Institutional Effectiveness Criteria Information Flow/ Link to Improvement Data Driven Approach to Management •Measuring Progress on Strategic Plan Goals Results in Steps Toward Improvement Annual Reports by Departments to Deans •Must State Progress on Goals and Set New Ones OIR & Departments Report to Evaluation Committee (EC) for the Strategic Plan Reviews Departmental Reports on How SA Info Was Used to Monitor Progress on Goals EC Monitors 27 Areas of Quality EC Members Serve As Liaisons to Other Committees and Departments Utilization in Academic Decision Making Institutional Strategic Planning •Prior Data Streams Produced the Development, Implementation and Evaluation of Plan for the Class of 2000 •Recent Student Life Data Critical to Next Wave of Planning on Intellectual Climate Program Review & Departmental Evaluations •One Full Cycle Completed •Departmental and Curricular Changes, including Program Restructuring or Elimination •Departments use OIR Data & their own to Improve Teaching & Learning Utilization in Academic Decision Making Student Life •Units Engage in Planning & Evaluation •Many Changes to Services Made Based on Satisfaction Survey Results Evaluation Committee •Assesses Laptop Initiative and Effect of Technology on Academics •Purpose Remains Measurement of Progress Towards Academic & Educational Goals Summary-- Culture for Assessment Academic Management is Well-designed Integrated Institutional Planning, Program Review, & Academic Evaluation Approach that is Improvement Driven Student Assessment Integrated into All Academic Management Processes Strong, Central Support for Data Driven Management Intensive Involvement of Departments in Program Review Key Support & Coordination Roles Played by OIR Commitment to Assessment is Embedded in Culture, but Not as an End in Itself Wide Support for Student Assessment Conclusions Lessons & Insights Need a Clear Understanding of: The Institution’s Academic Management Approach & its Key Elements The Role of Assessment in Their Various Elements Emphasis is on Integrating Student Assessment into Key Elements & Academic Decision Making Arenas: Assessment Used at All Levels of Institution Tied to Academic Improvement Embedded in Academic Management Policies & Practices Assessment Data Key to Management & Evaluation Processes Lessons & Insights An Effective Student Assessment Strategy Requires: Clearly Understood Purpose for Assessment Committed, Upper-Level Leadership Coordination Across Support Offices & Departments Flow of Information to Decision Makers at All Levels Link of Data to Problem Identification and Goal Setting Building a Culture that Values Assessment Requires: Long-term Effort Linking Assessment to Academic Management Processes Concerned with Academic Improvement Wake Forest Case Study Visit September 2000 Wake Forest Honored to Be Selected for the Case Study Faculty, Students and Administrators Were Most Cooperative in Participation of Interviews and Focus Groups Interviews Included President, Vice Presidents, Deans, Associate Provost, Advising Director, Director of Teaching & Learning Center, Director of Institutional Research, Chief Information Officer, Faculty Senate President, Chair of Curriculum Committee, Chairs of Academic Departments and Members of Evaluation Committee Focus Groups Included Four Faculty Members and Four Departmental Chairs Active in Assessment and 10 “Good” Students Lessons Learned Considerable Assessment With Decided Outcomes Took Place at Institutional Level, e.g. Program Review and Surveys of Students and Faculty Assessment at Departmental Level Is Decentralized and Varies Considerably Qualitative Assessment Could Be Emphasized More at the Institutional Level Combination of SACS 1995-97 Self Study and 199295 Strategic Plan Strengthened Assessment at Wake Forest