Learning Outcomes

Staff Development Day February 2003 Gavilan Community College

Expected Outcomes

       Describe learning outcomes Explain why we must attend to learning outcomes List and compare assessment methods Outline Gavilan’s approach Create a learning outcome Identify resources for further research Submit to the will of WASC

This is a subliminal message to promote

Bike To Work and School Day

on

May 15 th

“Pedal Peace, not war”

     

Perspective

In the past, year-end reports demonstrated proper management but not whether the programs directly aided student learning Colleges measured enrollments, participation rates, budget expenditures More recently have expanded to measure success, retention, persistence, graduation, and transfer Focus has also been on instructional delivery versus student learning Means have been mistaken for the ends (Barr & Tagg 1995) Learning outcomes now required by WASC

   

What are Learning Outcomes?

“Learning outcomes” not a new concept – – – Used in California in K-12 and vocational programs Used in other college systems (i.e. Maryland, England) Barr and Tagg called for learning outcomes in 1995 Focuses on what and how much students learn not on how we instruct or support them; outcome based Emphasis on higher more complex level of learning Clearly states educational intentions for students

Sounds like behavioral objectives!

  Learning outcomes differ from behavioral objectives by – – Being broadly stated Having instructors assess gains in skills rather than creating detailed list of specific topics and abilities to be mastered (Harden 2002) Some authors do not perceive that the difference between learning outcomes and behavioral objectives is important (Prideaux 2000)

The definition of insanity is doing the same thing over and over again and expecting different results so if at first you don’t succeed try, try again…

    

New Perspective

Shift from instruction as “core of the college” towards learning as central focus Potentially lessens threat to faculty by shifting focus of evaluation from teaching to learning “We must treat the college as a learning laboratory” – John Baker, VP SS Gavilan Some areas may be easier to evaluate than others such as Vocational programs Many Student Services will be more difficult to assess

Levels of Analysis

 Institutional  Department or Program  Class or Service  Assignment or Student Interaction

Gavilan Institutional Learning Outcomes

 Communication  Cognition  Information Competency  Social Interactions  Aesthetic Responsiveness  Personal Development and Responsibility

Gavilan Institutional Learning Outcomes

 Communication – Listening – Reading – Writing

Gavilan Instutional Learning Outcomes

 Cognition – Analysis & Synthesis – Problem Solving – Creative Thinking – Quantitative Reasoning – Transfer of Knowledge & Skills to a New Context

Gavilan Instutional Learning Outcomes

 Information Competency – Research – Technological

Gavilan Instutional Learning Outcomes

 Social Interaction – Teamwork – Effective Citizenship

Gavilan Instutional Learning Outcomes

 Aesthetic Responsiveness – – Differentiate between people who can sing and people who can’t  e.g. Brittney Spears vs. Norah Jones Distinguish between art that offends you, art that is cheesy, art you can’t understand, and art that is too expensive



Gavilan Instutional Learning Outcomes

Personal Development and Responsibility – – – – Students will develop individual responsibility, personal integrity, and respect for diverse people and cultures Self-management: Students will demonstrate habits of intellectual exploration, personal responsibility and physical well being Ethics and Values: Students will demonstrate an understanding of ethical issues that will enhance their capacity for making sound judgments and decisions Respect for Diverse People and Cultures: Students will respect and work with diverse people including those with different cultural and linguistic backgrounds and different abilities

Learning Outcome Example: Introduction to Statistics

Learning Outcome ILO Demonstrate proficiency with calculations of basic probabilities for simple events.

Utilize statistical software to analyze and interpret data.

Demonstrate a synthesis of concepts learned throughout the semester by a group project, which will require students to formulate a survey, collect data, analyze data, and derive inferences from the data.

cog Com, cog, info comp Com, cog, info comp, soc

Service → Learning Outcome ↓ Counseling Communication Cognition Student understanding and explaining goals/plans; use advice from counselor Understand GE patterns; Develop ed plan Financial Aid Student discussing and understanding requirements with staff Student explaining what is desired: registering, transcripts, adding, etc.

Calculating income, need, and unit loads Admissions and Records Calculating units and fees Information Competency Using web to gather program information Comprehension of forms and instructions Comprehension of forms and instructions Social Interaction Aesthetic Responsiveness Working with college personnel Working with college personnel/mentoring Form design, web page layout Form design, web page layout Working with college personnel/mentoring Form design, catalog, schedule web pages layout Personal Development And Responsibility Understand concept of balancing school, work, and family Provide accurate information; meet deadlines Provide accurate information; meet deadlines

Assessment Considerations

    Assessment has a role but is not an end in itself Assessments must feed back into practices All data collection must be preceded with consideration of how it will be analyzed and how well it can answer the question of service efficacy and desired learning outcomes Collect a few well-chosen pieces of evidence rather than obtaining all possible data

    

Data Considerations

Quantitative and Qualitative Direct and Indirect As students are exposed to many influences on learning, separating the effects of a service from other factors proves difficult How long does it take for learning to manifest itself in observable behavior? Must assess in many places and stages.

Not everything that can be counted counts and not everything that counts can be counted -Einstein

Example Method Randomly assign students to service Randomly assign students to receive information about services Correlate use with performance or skills measures Surveys Case study and journals Strength of Evidence Can claim causality Weaker causality claim Causality cannot be claimed, useful in conjunction with other information Ethical Consideration Denies access to a service that may or may not be effective for some students All students have access but some receive less information No restriction of access or information Causality cannot be claimed, useful in conjunction with other information Use student time to complete survey Causality cannot be claimed but complex and difficult to measure effects can be noted Confidentiality

      

Common Assessment Methods

Tests – Locally developed or Standardized Performances – Recital, Presentation, or Demonstration Cumulative – Portfolios, Capstone Projects Surveys – Attitudes and perceptions of students, staff, employers Database Tracked Academic Behavior – Grades, Graduation, Lab Usage, Persistence Narrative – Staff and student journals, essays, and interviews Embedded Assessment – Using grading process to measure ILO

Embedded Assessment Example

Criteria Joe Jane Liam Celeste Avg 4 5 4 4 Arithmetic with fractions Calculating basic areas Order of operations Right triangles 3 3 3 3 Total 12 4 4 4 16 3 5 4 17 5 5 4 18 3.75

4.25

3.75

Student Grade C B B A Down for grades, across for outcomes assessment –after Nichols

Assessment Strategies

 Track outcomes for every student all the time  Randomly select a set of classes to assess once per year

Gavilan’s Approach

     Educational Master Plan calls for each department to identify learning outcomes for at least one course Student Services held a retreat to address the application of learning outcomes Instruction is revising selected course outlines to include learning outcomes Next step is to develop more comprehensive and formalized communication among departments and between Student Service and Instructional divisions What is learned needs to be systemically integrated into ongoing services and evaluation

Teamwork

 The task of evidence compilation is an institutional responsibility and must be distributed across the institution  There is no “I” in team but there is a “me” if you take out the “a” and switch the “m” and the “e”

Writing Learning Outcomes

 Should be based on our Mission…  In an environment that nurtures creativity and intellectual curiosity, Gavilan College serves the community by providing a high quality learning experience which prepares students for transfer, technical and public service careers, life-long learning, and participation in a diverse global society

Writing Learning Outcomes

    …and our Values… Value 1: We value excellence in and promotion of comprehensive programs, services, and activities.

Because we value excellence, our objective is to develop learning outcomes and measures at the services and program level.

The measure we will use to determine whether we have met our objective is that students will demonstrate their learning in all programs according to institutional learning outcomes.

From Harriett Robles

   

Writing Learning Outcomes

Outcomes must be observable so you can measure or record them Could be quantitative – Play all major scales in eighth notes at 120 beats per minute (comm, cog, aesthetic responsiveness) Or not – – Compose in sonata-allegro form (comm, cog, info comp, aesthetic responsiveness) Accurately complete a FAFSA (comm, cog) Course level learning outcomes sound like behavioral objectives (or really are the same) but LO tie into institutional learning outcomes

  

Writing Learning Outcomes

Good verbs (Blooms’ Taxonomy): – – – – – – Compile Analyze Design Apply Demonstrate Explain – Compare Bad verbs Arrange Identify Classify Operate Solve Write Differentiate Calculate Formulate Predict Compose Assess Estimate Critique – Know Understand Appreciate Learn Good verbs are clear and observable, bad verbs are vague states of mind

Your Turn! 5 Minute Exercise

 Gather into groups with people you already know so you don’t have to spend time asking their name  Write 2 learning outcomes for a class and 2 for a service  Some of you will be randomly selected to share your outcomes

Class

5 minute exercise suggestions

Service MATH 154: Elementary Algebra PSYC 1A: Introduction to Psychology AH 16: Intravenous Therapy/Blood Withdrawal MUS 4A: Beginning Piano Math Lab Counseling Health Services Financial Aid ECOL 1: Ecology: Conservation of Natural Resources Admissions and Records

     

Institutional Concerns

Another education fad that will invoke hypnotic tendrils of smoke and little flame Sounds kindergarten Students have a fundamental right to services and if these services are not or cannot be related to learning outcomes, their existence could be unfairly jeopardized (Collins 2002) Uncritical application of business models and concepts to education Fear all this measuring will be a waste of effort and not enhance student achievement – Luna, Gavilan College Academic Senate Do students really care about learning? – Borden

Learning Outcome Resources

 Authors – – – – – Trudy Banta Patricia Cross Thomas Angelo James Nichols Victor Bordon

       

Learning Outcome Resources

WASC – www.wasc.org

American Association for Higher Education – www.aahe.org/teaching/portfolio_db.htm

Palomar College – www.palomar.edu/alp/ Novia Scotia Community College – www.nscc.ns.ca/studentportfolio Cascadia Community College – www.cascadia.ctc.edu/LearningForTheFuture/vanguard.asp

Raymond Walters College – www.rwc.uc.edu/phillips/index_assess.html

Alverno Colleg – www.alverno.edu

Gavilan College Curriculum and Research – – www.gavilan.edu/curriculum www.gavilan.edu/research

Learning Outcome Resources

 Curriculum Committee  Academic Senate  Student Services Council  Researcher  Reference Librarian

Conclusion

    The learning outcomes challenge can be another opportunity to improve our efforts and to better integrate with instruction We must take control of the learning outcomes mandate and create a process that benefits our students We must work together Learning outcomes is a plot concocted by Texas CEO’s to distract us from the budget crisis

   

Acknowledgements

Gavilan Student Services Division: – – John Baker Margery Regalado Gavilan Instructional Division – – – – Marty Johnson Marlene Dwyer & her Curriculum Committee Department Chairs Academic Senate, Enrique Luna Research and Planning Group – Harriett Robles, Brad Philips, Fred Trapp WASC – Darlene Pacheco, Barbara Beno

Riddle

Made from sheets Not meant for sleep It's bound to make you think With stories of smearing ink

Thank You