CLU Professional Development Day May 20, 2009 ISS Help Desk Survey Data Driven Decision Making Some Lessons Learned 2008-2009 With thanks and appreciation to Sue Bauer,
Download ReportTranscript CLU Professional Development Day May 20, 2009 ISS Help Desk Survey Data Driven Decision Making Some Lessons Learned 2008-2009 With thanks and appreciation to Sue Bauer,
CLU Professional Development Day May 20, 2009 ISS Help Desk Survey Data Driven Decision Making Some Lessons Learned 2008-2009 With thanks and appreciation to Sue Bauer, Eileen Leese, and Elena Montalto for designing and administering the surveys and tabulating summary results. Participant Demographics 34 21 Administrators Staff 31 71 Students 158 13 Adjuncts 38 34 Faculty 0 100 Lessons Learned Survey employees during school year. Use surveys tailored to employees and students. Test items before administering the survey. 200 2009 2008 June 2008 Lesson Learned Ask questions that will provide useful feedback. Quantitative Feedback June 2008 Connectivity Assistance 63 WebCT Assistance 38 Dissatisfied Students Account Assistance 43 CLU Website Assistance 47 N = 158 Qualitative Feedback June 2008 Connectivity/CLU Website 26 Dissatisfied Students WebCT13 HD Assistance 17 Lessons Learned Listen to all the feedback Surveys Echo Editorial Class Rep Meeting(s) Petitions Help Desk Calls Be proactive Communicate! Help Desk: Actions Taken • Enhanced Help Desk student worker screening, training, and monitoring. • Created binders with troubleshooting information for each student worker. • Developed a Help Desk knowledgebase. • Initiated call backs and email messages to clients to assess satisfaction and ticket completion. • Provided Help Desk ticket number to clients. Connectivity Wireless Wired Connectivity Wired Wireless Connectivity • Summer 08: Replaced over half of the 115 access points on the wireless network • Fall 08: Increased wireless network monitoring & Internet bandwidth from 30 to 45 bps; met with Res Life staff • Winter 08: Solicited an advertising campaign on student connectivity for 09-10 • Spring 08: Tested and installed new access points in Grace and Mt. Clef; received campaign materials WebCT Course Management System • Initiated CMS study & pilot-tested alternatives: Moodle & Blackboard • Noted student disappointment that not all faculty used WebCT; searching for a product that appeals to students AND faculty • Fixed bug with portal crashing March 2009 Lesson Learned IT’s about the entire department. Next Steps • Summer 2009: Finish analysis and develop actions plans from the 2009 IT survey; implement new personal learning system • Fall 2009: Administer LibQual survey; start “Get Plugged in” campaign • Spring 2010: Administer employee and student IT survey • Always: observe, act, assess, and communicate Hope springs eternal in the human breast; Man never Is, but always To be blest: The soul, uneasy and confin'd from home, Rests and expatiates in a life to come. -Alexander Pope, An Essay on Man, Epistle I, 1733 Melinda Wright Karissa Oien Seth Wagerman May 20, 2009 History of paper and pencil evaluations Melinda’s role in the process Scantron Machine Hand enter evaluations Time consuming Re-evaluation of current method CLU pilot of switching to online evaluations Course Evaluation possible vendors for online systems CoursEval Why is it a good fit for CLU? What it does for CLU? Process of new online evaluations http://courseval.callutheran.edu http://www.callutheran.edu/assessment/res ources/CourseEvaluations.php Undergraduate Graduate ADEP SPSS analysis of five select faculty members Pen and Paper Evaluations Online Evaluations Professor 1 Course A Course B Professor 2 Course A Course B Course C Professor 3 Course A Course B Professor 4 Course A Course B Course C Course D Professor 5 Course A Course B Course C Professor 1 Course A Course B Course C Professor 2 Course A Professor 3 Course A Course B Course C Course D Professor 4 Course A Course B Course C Professor 5 Course A Course B Repeated Measures Analysis Pen and Paper Evaluations Online Evaluations Professor 1 (MEAN SCORES) Professor 1 (MEAN SCORES) Professor 2 (MEAN SCORES) Professor 2 (MEAN SCORES) Professor 3 (MEAN SCORES) Professor 3 (MEAN SCORES) Professor 4 (MEAN SCORES) Professor 4 (MEAN SCORES) Professor 5 (MEAN SCORES) Professor 5 (MEAN SCORES) Sample Size = 5 pairs of professors’ scores Pen and Paper Evaluations Online Evaluations Professor 1 Course A Course B Professor 2 Course A Course B Course C Professor 3 Course A Course B Professor 4 Course A Course B Course C Course D Professor 5 Course A Course B Course C Professor 1 Course A Course B Course C Professor 2 Course A Professor 3 Course A Course B Course C Course D Professor 4 Course A Course B Course C Professor 5 Course A Course B Between-Groups Analysis Pen and Paper Evaluations Online Evaluations MEAN SCORES (Across all courses and Professors) MEAN SCORES (Across all courses and Professors) Sample Size = 100+ data points Foundations of Excellence Ensuring Student Success at CLU A Little Bit of History Non-major faculty advisors 1974-77 Learning Resources sections 1977-1982 Freshman Colloquium 1983-1995 CAPHE Grant 1996 Portfolio Advising Program (later the RIPP) 1998 Shared Freshman Reading 2004 WASC FYE Taskforce 2004 Foundations of Excellence 2008-2009 Policy Center on the First Year of College Policy Center provides: Yearlong process, structure, and support for campus review External reviewer Uses Foundational Dimensions and performance indicators Develop strategic action plan for campus improvement. http://www.fyfoundations.org/ Summer 2008 Steering committee chosen Steering committee conference in Ashville, North Carolina Co-chairs of nine Dimension Committees asked to participate Membership of committees discussed Dimension committees formed by September Timeline for reports stated Monthly meetings report findings, refine suggestions… Steering Committee Joan Griffin Kapp Johnson Halyna Kornuta Kristin Price Michaela Reaves Dimension Committee Chairs Philosophy: Kapp Johnson & Melissa Maxwell Doherty Faculty: Tim Hengst and Lorraine Purmort Learning: Andrea Huvard and Henri Mondschein All Students: Alan Goodwin and Marja Mogk Improvement: Halyna Kornuta and Angela Naginey Organization: Paul Witman and Melinda Roper Roles and Purposes: Guy Erwin and Juanita Hall Diversity: David Marcey and Anna Calderon Transitions: Marylie Gerson and Dane Rowley Campus Commitment 18 Staff/Administration 9 Alumni 9 Students 27 Faculty Remember: This is a review of the entire First Year Experience, NOT a review of the seminar! First Year Most Taken Classes Religion 100 English 111 Math 110, 115 Political Science 102 Spanish 101 Biology/Geology History 101 Philosophy Dimension Report Findings… CLU has no overarching First Year Philosophy Certain sub-groups have one, e.g. the Seminar, Orientation, etc. Action… Develop a Philosophy Statement Disseminate campuswide the newly created philosophy Correlate the philosophy statement to the university’s stated mission. Suggested Philosophy Statement The First Year Experience at California Lutheran University facilitates the transition of new students to a university life that fosters academic excellence and independent learning and living. The first year initiates the integration of student’s lives with the University’s intellectual, cultural, and social climates. Faculty Dimension Report Findings… CLU faculty teach first year students either in the seminar or in advising/class settings not the seminar CLU offers few rewards for teaching First Year Students Action… Add module to New Faculty Orientation on first year students All faculty are 1st responders Review faculty reward system/ART Provide workshop and travel funding Learning Dimension Findings 30% of faculty consistently teach First Years Adjuncts teach many first year students First Year learning goals are unclear High # of Freshman DFWI courses Out-of-class learning needs assessment First year courses experiences are inconsistent Learning Dimension Actions Communicate SLO’s Create a taskforce of faculty to develop specific learning goals for first year students. Emphasize importance of first year courses with smaller sections and full-time faculty Measure and analyze out-of-class learning data Suggested Learning goals Quantitative Skills Study skills Information processing Problem solving and independent thinking Research Communication Rigor All Students Dimension Findings Freshmen Athletes Freshmen Honors Students Under-represented Students (Multicultural & International Students) Non-Traditional Students & Students According to Housing Academically Under-prepared & Heightened Academic Needs Students Freshmen with Disabilities and Heightened Psychological Needs All Students Suggestions Actions A CLU Freshman Experience Mission Statement Adequate Personal Counseling Staffing Freshman Academic Advising The Freshman Seminar Dedicated First Year Experience Staff Member (or Freshman Dean) Improvement Dimension Findings Complete a self study to address six campuswide initiatives: · Registration · · · · · Advising Freshman Seminar Residence Life (including Judicial Affairs) Honors Program Students Taking Academic Responsibility (STAR) Improvement Actions Arrange for purposeful attendance at higher education events with a focus on the first-year experience. Invite external experts to review and share best practices of the first-year experience. Encourage and develop structures / opportunities for sharing campus-based knowledge/expertise about the first year and to develop a learning community. Develop a systematic approach to implementing the strategies for change. Organization Dimension First Year Organizational Structure Currently is… Freshman Seminar New Student Orientation Actions… Create and maintain an FYE website Conduct surveys for research Centralize First Year Student services Standardize FYE presentation Increase Funding. Diversity Dimension Findings Survey found the largest gaps between Students of Color and White student responses on questions relating to social class/economic status and to political perspectives. Underrepresented and Caucasian students are retained to their second year at a very similar rate. The 5 year average was 79% for Underrepresented students and 79.3% for Caucasian students. Diversity Suggestions Actions Increase faculty diversity Co-Curricular Integration/Connection Expansion of Co-Curricular Offerings Modify Religion 100 to focus on world religions instead of Christianity. Create a course on the basis of a multiplicity of world religions/of non-religious world views Respond to Diverse Students Concerns Re-evaluate CORE 21 Requirements for Global Perspectives and U.S. Diversity Roles and Purposes Dimension CLU does not fail, but does not succeed brilliantly either. There is much written material, in orientation and acculturation, and in the advising and mentoring process that deals with the overarching goals of higher education. Treatment of these topics is more implicit than explicit, including the syllabi used in freshman classes. Roles and Purposes Findings Improve use of printed materials to articulate the purposes and roles of higher education. Make the overall educational principles and goals explicit in student handbook/majors/catalog. Explore this dimension in the first-year programs Bring aspects of the roles and purposes dimension to the forefront of the Freshman Seminar curriculum and conversation. Transition Dimension Findings CLU maintains a high degree of accuracy in communication with students about enrollment and expectations Lack of clarity, consistency, and intentionality in the academic advising process. CLU uses new media effectively but lacks a virtual campus tour Transition Findings Develop websites for each academic department and each major. Expand aid offerings and student employment opportunities Use STAR/SOAR models to expand communication/advising to students and parents. Create online/virtual tour on CLU admission website. Expand the CLU 101 Program Increased outreach to community organizations programs. Consistent themes Create and disseminate a First Year Philosophy Evaluate Freshman Year Coordinator suggestion Improve Faculty Advising Model FYE website/virtual tour Now the work continues… PRIDE: Program Review, Improving Departmental Effectiveness Examine program effectiveness Different perspectives The big picture Showcase your department Take a closer look Your program Highlight what you do www.teachnet.ie “Stories from the Field” Dan Kuntz – Athletics Lisa L. Buono – Counseling and Guidance Pat Holmberg – Athletics California Lutheran University Athletic Department Program Operations: A Review Dan Kuntz Purpose The purpose of the California Lutheran University (CLU) Athletic Department review was to assess various operational areas for their effectiveness. The Assessment Requirement All Division II and III National Collegiate Athletic Association (NCAA) affiliated schools must complete the NCAA Institutional Self Study Guide (ISSG) once every five years. Submission of page with signatures – deliverable to NCAA – audit possible 1.President 2.Athletic director (AD) 3.Senior woman’s administrator (SWA) 4.Faculty athletics representative (FAR) 5.Director of financial aid 6.Director of admissions Instruments Institutional Self Study Guide (ISSG) survey Responses from stakeholders Feedback from the Faculty Athletic Policy Committee (FAPC) Data Collection Yes/No answers from stakeholders - Completed Comments from 11 stakeholders - Completed Faculty Athletic Policy Committee Feedback - Completed Post ISSG feedback from stakeholders [Next Step] Student and coach surveys [Next Step] Results Stakeholder Responses = 100% cooperation: President [Required]- Faculty Athletic Representative (FAR) [Required]- Director of Admissions [Required]- Director of Financial Aid [Required]- Athletic Director (AD) [Required]- Senior Women’s Administrator (SWA) [Required]- Student Athlete Advisory Committee (SAAC) president [Invited]Human Resources Representative [Invited]- Head Athletic Trainer [Invited]- Student Support Services [Invited]- Faculty Athletic Policy Committee (FAPC) [Invited - external review)- Assistant Athletic Director [Invited] 1. 2. 3. 4. 5. 6. 7. 8. 9. Process Timeline / Instruction memorandum Highlight sections of ISSG Calls/ Initial and follow up E-mails to respondents Electronic distribution of response process for study guide Electronic distribution of ISSG Electronic collection of ISSG responses Assimilation of responses into single ISSG for FAPC review Faculty Athletic Policy Committee review and feedback Action plans for improvement Visual Presentation of Athletic Department ISSG Operations Review Process President FAPC ISSG CLU Sports AD FAR Assistant AD SWA VP Enrollment Financial Aid Admissions HR SSS Stakeholders SAAC Trainer By Dan Kuntz Recommendations 1. 2. 3. 4. 5. 6. 7. 8. 9. Expand communication of the NCAA Division III philosophy Share how CLU athletics contributes to the educational process Clarify job description of the Senior Woman’s Administrator Develop a student athlete exit interview process - satisfaction Write sports medicine procedures for student athletes Increase athletic staff development Develop a student athlete handbook Survey female athletes Conduct an updated Title IX study once a tenth women’s sport is added Lessons Learned Transport your department in a new direction by using the BUS: Being inclusive Using staff ideas to design specific action plans from reviewer feedback Share operational innovations and improvements with others Leadership Reflection: Effective leaders embrace the abilities and passions of others to build exemplary organizational teams that create exemplary programs EXPLORING THE NEED FOR STUDENT-ATHLETE DEVELOPMENT PROGRAMMING AT CALIFORNIA LUTHERAN UNIVERSITY Patrick Holmberg P.R.I.D.E. Program Review: Investing in Departmental Effectiveness Purpose • The purpose of this assessment study was to explore the need for educational programs and services to best meet the developmental needs of California Lutheran University (CLU) student-athletes. Background •Student-athletes are young people in transition who, like most college-age individuals, must confront formidable developmental challenges (Ferrante, Etzel & Lantz, 2002). •The critical difference, however, is that these people must do so in an environment that presents a unique, complex set of demands (Engstrom, Sedlacek & McEwen, 1995). Data Collection • Qualitative interviews with CLU Athletic Department Administrators and sport coaches – Inductive-deductive content analysis • Quantitative data collected from a CLU student-athlete survey – 95 participants (25%) – NCAA Needs Assessment Survey – Flashlight – SPSS; descriptive and comparative statistical analysis Results •The following qualitative themes emerged: •Engagement •Personal Growth/Maturity •Athletic Development •Student-Athlete Retention Results •Quantitative findings indicated that first and second year studentathletes believed they used campus resources significantly (p = .04) less than upperclassmen. Recommendations . . . Academic Athletic Goals Build Trusting Relationships Campus Resource Awareness Lessons Learned . . . • Assessment is a good thing • Start small • Assessment is a process • Invite others to collaborate Dialectic Action Research Spiral (Mills, 2000) Questions? RETENTION -Different Perspectives -Different Challenges Cathy Alexander, Institutional Research Officer Maria Kohnke, Associate Provost – Registrar Stephanie Mercer, Enrollment Systems Manager Angela Naginey, Director of Retention “One Student at a Time” • A comprehensive university, provides a challenging academic environment servicing approximately 2100 undergraduates and 1300 graduate students Agenda • Address the Why • Undergraduate Process and Implementation • Graduate Process and Implementation • Next Steps • Questions Undergraduate Retention Background • Doing great things Current Programs in Place – Mandatory orientation for all first time freshmen – First year freshmen seminar class – Residence hall policies & activates – Task force to address retention issues – FTF retention rate was good but declining CLU Assessment System Comprehensive Approach to Retention • Needed to develop a focused, comprehensive plan that addressed all students • Needed more coordinated and focused strategies – For the traditional undergraduates: • Improved coordination and communication across departments • Improved communication to the entire campus community retention progress and goals – For the graduate/adult programs: • Get a beginning look at the numbers and process • Connect the data to the programs Definitions • Undergraduate Programs – Standard approach to define retention • Graduate and Adult Programs – Multiple definitions as we refined our approach – Baseline Data- who started during a 7 year period, remained active or completed – Streamlined Data – who started in an academic year, attended or completed in successive years First Time Freshmen Retention & Graduation Rates Entered Fall # Entering First Time Freshmen 1998 1999 2000 2001 2002 2003 2004 2005 2006 324 326 364 392 330 336 402 385 403 5 Yr. 2007 Avg. 424 382 Retention Rates % Retained 2nd yr 81.50% 82.50% 78.80% 79.60% 82.40% 78.00% 84.30% 78.20% 76.70% 81.40% 79.90% % Retained 3rd yr 70.10% 72.40% 67.90% 67.90% 73.00% 69.60% 71.60% 68.80% 67.20% % Retained 4th yr. or Graduated 64.80% 67.80% 66.50% 66.80% 71.80% 67.30% 69.70% 67.80% 70.20% 68.40% Graduation Rates % Graduated 4 yrs 56.50% 55.20% 57.10% 59.90% 59.40% 53.90% 57.20% 57.10% % Graduated 5 yrs 63.00% 65.00% 62.40% 61.70% 68.50% 60.10% 64.10% % Graduated 6 yrs 63.30% 67.80% 63.50% 63.30% 69.70% 65.50% UG Retention Planning Process Task Timing CLU created Director of Retention position November 2007 Director implemented targeted retention strategies November/December 2007 Expanded attrition reports with Institutional Research January 2008 Created Undergraduate Retention Committee February 2008 Drafted Undergraduate Retention Committee purpose March 2008 Held Retention Summit for Retention Committee July 2008 Purchased retention communication and tracking software August 2008 Drafted Traditional Undergraduate Retention Plan September 2008 Coordinated and hosted Retention Consultant Assessment October 2008 Retention Committee and Strategic Planning Steering Committee approved Retention Plan October 2008 Created Graduate/Adult Student Retention Committee November 2008 Undergraduate Retention Plan • Main Goal of the Plan: • Increase the four year graduation rate • Strategies: • Select and implement retention software with an Early Alert component • Evaluate, improve and enhance academic advising • Explore and identify possibilities of improved customer service and office communication of student services Undergraduate Plan-Next Steps “One Student at a Time” • Enhanced awareness and visibility – Enhanced relationships – Retention and graduation rates – Retention purpose & goals • Implement Plan – Foundations of Excellence in the First Year Project – Customer service training – Faculty Enrollment Committee reviewing faculty advising – Enhanced communication and tracking of students – More focused approach for specific attrited populations Summary Data on Attrition Traditional Undergraduate Programs Attended Fall 2004 2005 2006 Total % Total % Total % Total Attrition 190 11% 232 13% 260 14% Freshman 141 74% 178 77% 199 77% Transfers 49 26% 54 23% 61 23% Enrollment Status After CLU: Transferred to another institution 151 80% 198 85% 193 74% Did not enroll at another institution/unknown 39 20% 34 15% 69 27% Attended 1 other institution 60 40% 106 53% 162 84% Attended 2 other institutions 53 35% 62 31% 27 14% Attended 3 or more other institutions 38 25% 25 13% 4 2% Attended graduate school 1 >1% 1 >1% 0 0% Transferred to: 2 year institution 103 68% 125 63% 98 51% Another 4 year Institution 48 32% 73 37% 95 49% 4 year public 31 65% 49 67% 68 71% 4 year private 17 35% 24 33% 27 29% Another California School 102 68% 141 71% 139 70% A school outside California 49 32% 57 29% 54 30% Graduated with a Bachelors 10 6% 1 >1% 0 0% Graduate Background • Data Drivers – Course scheduling, marketing, enrollment management, program review, accreditation, grant writing, new program development, faculty hiring justification • Lack of required systematic reporting • Lack of resources • Lack of comparative research on masters/postbaccalaureate certification programs Graduate Timeline Task Timing Associate Provost for Graduate/Adult Programs and Accreditation appointed Provost’s Office and Registrar surveyed students who had not enrolled in a course during the 2004-2005 academic year Director of Marketing for Graduate/Adult position created Fall 2004 Enrollment Systems Manager for Graduate/Adult Programs position created Director of Retention position was created to address issues of retention for all students Institutional Research Office reporting standardized for Graduate/Adult Student Learner Programs Purchased retention communication and tracking software Fall 2007 Created Graduate/Adult Student Learner Retention Committee Fall 2008 Beginning institutional inventory of retention practices Spring 2009 Spring 2005 Spring 2007 Fall 2007 Fall 2007 Summer 2008 Graduate/Adult Reporting Evolution Baseline Total Active Master/Credential Program Completed Withdrew N % % % Business Administration 987 28.20% 47.40% 24.40% Public Policy and Administration 84 28.60% 32.10% 39.30% Computer Science 52 82.70% 7.70% 9.60% Marital and Family Therapy 116 31.00% 57.80% 11.20% Clinical Psychology 74 16.20% 67.60% 16.20% 1,830 20.30% 48.90% 30.80% Teacher Preparation Special Education 393 35.40% 33.10% 31.60% Curriculum and Instruction 203 15.30% 39.40% 45.30% Educational Leadership 318 15.40% 43.70% 40.90% Counseling and Guidance 757 23.10% 57.70% 19.20% 4,814 24.10% 47.70% 28.20% Total Year to Year Retention & Grad Rates Streamlined Academic Year All MBA - New Students 0001 0102 0203 0304 0405 0506 0607 134 129 137 160 155 196 311 Retained or Graduated % 2nd Year 74.6% 81.4% 71.5% 82.5% 86.5% 83.2% 88.1% % 3rd Year 73.1% 72.9% 65.7% 71.3% 74.8% 76.5% % 4th Year 72.4% 68.2% 62.0% 71.3% 62.6% % 5th Year 73.1% 69.0% 59.1% 72.5% % 6th Year or more 72.4% 69.8% 61.3% Graduated % within 1 year 2.2% 8.5% 17.5% 16.3% 23.9% 19.9% 19.9% % within 2 years 36.6% 28.7% 33.6% 40.0% 50.3% 40.8% % within 3 years 54.5% 50.4% 46.7% 54.4% 60.0% % within 4 years 66.4% 61.2% 54.7% 64.4% % within 5 years 69.4% 66.7% 56.9% % within 6 years 72.4% Academic year = Summer, Fall, Winter and Spring of the same fiscal year Specialized Populations 3 Year Average Attrition Rate By 2nd yr 3rd yr 4th yr All MBA Traditional (N=650) MBA (N=396) 19.5% 14.4% 25.6% 28.2% 34.5% 36.8% Online IMBA MBA (FP) (N=127) (N=127) 4.7% 24.4% 4.8% 39.4% 4.8% 50.0% Term to Term Enrollment Mgmt. Fall MPPA 06FA 07FA 08FA Continuing students 31 34 49 36 New students 10 7 16 11 Fall Total 41 172 41 204 65 3 yr. Avg. 282 113 Eligible - Not enrolled 2 4 3 4 December Graduates 3 6 4 3 9.7% 17.6% 8.2% 9.7% % new students 24.4% 17.1% 24.6% 24.1% Return rate-non grads/eligible students 93.9% 89.5% 94.2% 89.7% 132.3% 91.1% 130.0% 114.1% Graduation rate of continuing students Net Fall as a % of Summer Graduate/ADEP – Next Steps – Created the Graduate/Adult Learner Retention Committee – Defining benchmarks for graduate and adult learner programs – Creating an institutional inventory of retention practices for our 20 different graduate and adult programs “One Student at a Time” What is my role? Questions/Comments? Feel free to contact: Cathy Alexander – [email protected] Maria Kohnke – [email protected] Stephanie Mercer – [email protected] Angela Naginey – [email protected]