Transcript Slide 1

The Learning Outcomes Leaders
Meeting with Deans
“Improvement—if it is to happen—will occur where
faculty and staff have the most leverage to change how
they approach teaching and learning.”
-Ewell, Paulson, and Kinzie, 2011
Feb. 6, 2015
Laura Blasi, Ph.D., Institutional Assessment
Meeting Goals…
1. Examine prior successes and challenges looking at
the data we have from a college-wide perspective;
2. Review the Learning Outcomes Leader role and Work
Plan sent with the invite for this;
3. Collaborate on a quick needs assessment related to
our learning outcomes work at the college;
4. Analyze and make decisions – LOLs and deans
together as possible – using prior reported data; and
5. Discuss spring assessments and begin to develop a
plan for Assessment Day.
Most Students Experience Us as a College
(not a program)
•
Presentation to the Board of Trustees - 2014
Accomplishments,
Challenges, Opportunities
•
•
•
•
•
•
Strategic Goals
Essential Competencies
Faculty development
Annual review / workload
Curriculum Committee / catalog,
Viability and Academic Program Review (AS)
Assessment Day – 5 years of growth
– May 2010 – 11 College-wide Programs / Disciples participated (11% plans)
– May 2011 – 17 College-wide Programs / Disciplines Participated (25% plans)
– May 2012 – 61 Programs / Disciplines Participated (56% plans)
– May 2013 – 68 Programs / Disciplines Participated (100% plans)
– May 2013 – 64 Programs / Disciplines Participated (100% plans)
• IE Compensation Plan – a five-year cycle for assessing every outcome
• Courses mapped to program / discipline student learning outcomes
2003 - The SACS Visiting Committee recommended that
the College provide evidence as it relates to the quality of
student learning outcomes.
11 college-wide assessment plans: May 2010
17 additional assessment plans: May 2011
56 college-wide assessment plans were implemented and
improvement plans were developed - May 2012
However, the process was not yet uniform noted in a
SACS recommendation following a visit in February 2013.
2014 visit and 2015 decision by SACS we were responsive
Important Lesson
Curricular Mapping is the Key to Moving From Course
Level Assessment to Program Level Assessment
Course-Level
Assessment
• Assess student learning
outcomes at the end of the
course
• Assign a grade to individual
students
• Grading often involves only
one faculty member who is
teaching the course
Program-Level
Assessment
• Assess student learning
outcomes at the end of the
program
• Evaluate aggregate student
artifacts for purposes of
program improvement
• Evaluation involves faculty
teams across the program/
discipline
Note: The counts represent programs – the leader for each program took the survey.
Assessment Day 2014
Results / Findings
Documenting the Impact
• Because faculty work together and share resources and are
able to critique each other's resources students have better
assimilated basic interpreting skills needed to be successful as
students transferring as a junior to a university. Students also
demonstrate these skills in various internship opportunities
which they are more eager to apply.
• The group we assessed this year regarding demonstrating
information literacy showed mixed levels of competency based
on their performance on the research project. The
weaknesses we identified were: the ability to write a good
research question; and the ability to apply the information they
found back to the research question.
Making Improvements
•
Adjustments in our teaching and curriculum. Students seem to do well
with the skills that are being taught but struggle with application and
thinking/reasoning. We would like to provide more learning opportunities
in the classroom to support mastery of these higher level thinking skills.
Many faculty already do this, but we have discovered that many also do
not. The data collected support this as we look at the large number of
students that scored 1s on the competencies as they relate to thinking
and reasoning skills but earned As or Bs in the course.
•
We have some evidence to show that by implementing a “common
tutorial”, we should be able to increase student learning as documented
by our assessment. Faculty within the science discipline areas are
developing tutorials using a tutorial template and a scientific article
relative to the discipline area. The tutorial to be based on the discretion
of the discipline task force and faculty.
•
Improvement Plan: Projected Plans for Upcoming Year- • We determined
that further additions of strategies are needed such as further
explanation of problem instructions. • Determination of changes in group
work assignment. (Some activities may work best when students
complete as part of a team.) • Assessment at midterm of semester is
recommended to determine those students that may need academic
intervention.
Documenting the work with each
outcome being assessed at least
once in a five-year cycle
Discussion
Program and Discipline
Learning Outcomes Assessment
Successes, Challenges, Resources
Specific to
assessment you
have been part
of at the program
or discipline level
• Three or more successes
• Three or more challenges
• Three or more resources
 Deans’ Perspectives – AS / Gen Ed
 Faculty Members’ Perspectives – AS / Gen Ed
How we
support
LOLs / LOA
What was
learned from LOA
Question shared in pre-survey…. “How do college leaders/administrates understand
this role and what is expected from the Learning Outcome Leader?”
Learning Outcomes Leaders – Work Plan
Advice from Experienced
Learning Outcomes Leaders
• Include all faculty, full-time and part-time in the discussions and
Assessment Day. Create clear guidelines for the assessment.
Have an activity-driven agenda for Assessment Day so that
actionable decisions can be made based on the data.
• Have meetings other than on Assessment Day to inform the
faculty about the progress and steps being made; many faculty
hear about this work once a year and forget about it. Create
smaller committees where faculty (other than the PLOA leaders)
can work on different aspects of the PLOA work; this gets more
faculty to engage.
• Offer clear communication to faculty in your program/discipline *
Schedule meetings in advance to ensure participation * Partner
with Dean to make this work a divisional effort; add to divisional
meeting agendas.
x
X
Discussion
• Observations
• Making it Meaningful
• Next Steps
Discussion
• Spring Assessment Activities
• Assessment Day
• Long-term Planning
Challenges… momentum
Momentum, we have done so much good work
and need to stay focused.
Finding or creating best practices in program
assessment, and then educating faculty and staff
in order to ensure the process is implemented
efficiently and the data is accurate.
Ensuring that our program's learning program
matches with what employers require from our
students.
Challenges… meaningful assessment
In my opinion, the greatest challenge is to design
a single assessment which will provide a
meaningful results for all courses in a discipline.
Figuring out what data set to analyze. The college
is so big, there are lots of avenues that could be
pursued. I think it is difficult determining what
type of analysis will yield the greatest benefit.
I think the system that is in place is starting to
come together so I think at this point is it about
the right artifacts, the methodology at times.
Challenges… college-wide communication
Trying to engage in meaningful conversation related to
results, especially when this requires a willingness to
discuss curriculum. Aligning assessments among and
between disciplines so that student learning can be
seen in progress and growth could be measured.
Getting buy-in from everyone involved. Communication
has been the biggest problem I see.
Ensuring maximum college-wide faculty participation in
the process and not leaving program/discipline learning
outcomes assessment become the burden of the leader
and a few faculty.
From Assessment Day 2014 Post-Survey
Making the Day a Success
• Again, it all comes down to what the deans do or don't do
with regard to this work. If the deans take it seriously the
faculty will too. I have said that consistently since we
started doing this work and I'll probably say it again next
year.
• More opportunities to work across / with the disciplines.
We would like to have a college-wide information literacy
summit or big meeting.
• Training may be needed for leaders, deans, and chairs to
better understand how to guide the faculty. If the leaders
don't feel connected to the work, then the faculty
concerns/fears become are echoed by the leader
diminishing the quality.
Needs Beyond Assessment Day
• Time to examine our needs. Dissemination of
knowledge is not an issue; re-examining what we want
our students to know is most important. Retooling is
important at every junction.
• Stronger communication among faculty, faculty
development opportunities related to specific
assessments, and time spent developing clearly
measurable criteria in the rubric.
• More people who understand the value and importance
of assessment involved in the process. I think many see
it as something we have to do, but they do not
understand the possible benefits, because I'm not sure
how authentic the assessment has been and the results
have not been broadly shared by everyone in the
college.
Closing Conversation and Evaluation
• Results from meeting returned to you by Feb 20
• Please make Organizer corrections / additions by March 20
• Let us know by March 20 if you would like a Blackboard
shell to use to organize the work
• Deans ask for an Assessment Day Update prior to April 10
•
•
Valencia Faculty-Developed Assessment Glossary:
http://valenciacollege.edu/instassess/documents/lacGlossary.pdf