Transcript Document
ABOUT THE ASSESSMENT CYCLE OFFICE OF INSTITUTIONAL EFFECTIVENESS • JANUARY 2012 Program Assessment: A Cycle Program assessment is the evaluation of student learning within a particular academic program leading to a defined objective, degree, certificate, diploma, license, or transfer to another institution of higher education. Program assessment is an continuous cycle that is geared towards ongoing improvement. California Code of Regulations, Title V, § 5500(g) LATTC’s Assessment Model LATTC used the Nichols and Nichols’ (1995) model to provide a framework for its own modified cycle of developing and assessing student learning outcomes (SLOs) and service area outcomes (SAOs). There are five main components in this model: 1. 2. 3. 4. 5. Mission/Goals Intended Outcomes Means of Assessment/Criteria for Success Summary of Data Use of Results The Assessment Loop The figurative representation of the assessment loop, created by Dr. Janet Fulks of Bakersfield College, and referenced in LATTC’s SLO Terminology Glossary located at http://college.lattc.edu/slo/slo-glossary/, is also comprised of 5 parts, beginning with “develop or modify student learning outcomes” and leading to “develop, modify or review a curriculum, course, program, or service.” Except for the Mission/Goal alignment, included in the Nichols’ model, both methods incorporate the same basic premise: SLO development, data collection and analysis, reflection and refinement, and the continuance of the assessment and improvement cyclical process. SLO Integration Although the cycle of assessment and improvement continues around the “loop,” the results are also fully integrated into institutional planning and budgeting process. The Dean of Institutional Effectiveness is currently working on a district-wide program that will house outcomes assessment and program review data, provide easier access to information, and enhance the integrative planning process, among other features. The Assessment Cycle Mission & Goals, Intended Outcomes, Means of Assessment, Summary of Data, & Use of Results Mission and Goals A mission is a large, long-term end-result or achievement. There may be objectives, goals, strategies, executions and tactics all used to achieve the mission, but the mission is the biggest and most important thing to be accomplished. The mission statement is a what versus a how, and is very similar to a vision statement in that it has a future orientation. Intended Outcomes What will the student think, feel, know or be able to do as a result of a given educational experience? Student learning and service area outcomes are: The ends toward which effort and action are directed or coordinated. Although an outcome is the aim or an end, it is not necessarily the final achievement. That is the mission. Whats, not hows, but they are smaller than a mission. There can be a number of outcomes to be achieved in order to achieve a mission, but there is usually only one mission. Means of Assessment What are the criteria for success? What tools will be used to establish and measure success? What means of assessment will be employed? What is the expectation of success? Will a rubric be developed or is there one already in use? How and when will the evidence be collected? Who is responsible for the administration and evaluation of the assessment? How will the results be used? Triangulation The use of a combination of assessment methods, or triangulation, could be used to ensure validity of or as a means to improve assessment results. An example of triangulation is using surveys, interviews, and tests to assess the same student learning or service area outcome. Summary of Data After the assessment is administered, summarize the findings. How close were the results to the criteria for success? Other questions to consider when studying the data: Were there trends, patterns, or themes that emerged from the data? What skills (or portions of skills) did student universally understand? What were most common errors that students made? Are there any “wow” moments that you did not expect? Reassessment At the end of an assessment, it may be discovered that there were issues with reliability, the extent to which a measure, procedure or instrument yields the same result on repeated trials, or validity, the degree to which a study accurately reflects or assesses the specific concept that the researcher is attempting to measure, and, hence, reassessment is considered. Communicate First However, to ensure compliance with the college’s Assessment Management Plan (AMP), speak with the SLO Coordinator or Dean of Institutional Effectiveness before reassessing the same outcome for a subsequent semester. Use of Results: Closing the Loop Determine ways to use the data and make revisions to the curriculum, departmental processes, and the intended outcomes. After the data has been collected and analyzed, a departmental meeting should occur to discuss the results, areas for growth or opportunity, and brainstorm ideas and methods to address the areas. The purpose of this meeting is to stimulate meaningful dialogue and initiate change. Corroboration could lead to reassessing, revision, or continuance. Example of Closing the Loop, Pt. I Child Development Course (CTE Example): Some changes that could be implemented to address the assessment results is to visit a center with the entire class and collect data as a group and complete one demonstration score together. Any concerns or clarifications would be addressed in class. In addition, every single student must have a copy of the ECERS-R workbook; assessment cannot be done without it. Source: http://college.lattc.edu/slo/files/2010/10/CD_Assess.pdf Example of Closing the Loop, Pt. II Biology Course (Instructional Example): To address the difficulty that some students had with the level of abstractions in the evaluation of the results (completeness), chemistry could be made a prerequisite for anatomy and physiology. This is a common requirement for physiology courses elsewhere and would be entirely appropriate as it would give the students more background in abstract evaluation. Source: http://college.lattc.edu/slo/files/2010/10/Science_Assess.pdf Example of Closing the Loop, Pt. III Registrar’s Office (Student Services Example) With a In a move towards a quicker online application turnaround time, more staffing, specifically at peak times, during the beginning of the registration periods, will help improve the processing time from 48 hours to less than 24 hours. Source: http://college.lattc.edu/slo/slo-assessment-results/ Resources Chaplot, Priyadarshini. 2008. A guide to student learning outcomes and administrative unit objectives. Mt. San Antonio College. Retrieved January 21, 2012 from http://www.mtsac.edu/administration/senates/academic/documents/SLOAUOGUID EBOOK090808FINAL.pdf. Fulks, Janet. 2004. Assessing student learning outcomes in community colleges. Bakersfield College. Kelly, Bruce. 2008. Mission vs. goals vs. objectives vs. strategy vs. executions vs. tactics. Advertising Educational Foundation. Retrieved January 21, 2012 from http://www.aef.com/industry/careers/memos/8022. Los Angeles Trade-Technical College. 2012. SLO Website. Retrieved January 21, 2012 from http://college.lattc.edu/slo. Nichols, James O. and Nichols, Karen W. The Departmental Guide to Implementation of Student Outcomes Assessment and Institutional Effectiveness. New York: Agathon Press, 1995, 2000. The RP Group. 2010. SLO Glossary. Retrieved January 22, 2012 from http://www.rpgroup.org/resources/slo-glossary.