Transcript Scholarship of Teaching and Learning

Welcome to a Tutorial on the Scholarship of
Teaching and Learning (SOTL)
Almost every academician has heard the term “scholarship of teaching.” However, exchanges among
colleagues from many institutions reveal a heterogeneity of meanings, attitudes, and experience levels with
this topic. A comprehensive and coherent framework seems needed to accommodate the diverse strands of
thought and endeavor. Similarly, a treatment of SOTL-related topics that accommodates a broad continuum
of familiarity and experience seems called for.
The tutorial attempts to meet
those needs, by providing a
“primer” … a useful grounding
in the whole domain of SOTLrelated topics.
“The tutorial speaks to so many of the interests and concerns of
faculty members and administrators who are grappling with issues
of teaching, scholarly teaching, and the scholarship of teaching and
learning. You and your colleagues have done a tremendous job in
organizing and presenting these materials.” Barbara Cambridge
A group making the journey together along the continuum “from Alpha to Omega” will find interesting topics
for discussion and rich resources for further study. The tutorial is well enough indexed to also serve
encyclopedically for those who seek information about specific SOTL-related topics.
Materials for the tutorial originated in workshops and conferences sponsored by the American Association of
Higher Education (AAHE), the Carnegie Foundation for the Advancement of Teaching, the Professional and
Organizational Development Network in Higher Education (POD), the Lilly Conferences on College
Teaching and various individual colleges and universities.
Your hosts: Samuel B. Thompson, Craig E. Nelson, and Rita C. Naremore
Using the Tutorial
“It’s really impressive to see how much you have
compressed and organized into a manageable
‘package.’ I confess I was a bit daunted by the
number of frames at first, but the tutorial moves
along very smoothly. A very efficient way to learn
this stuff.”
Pat Hutchings
This is a multimedia tutorial including photos, graphics,
hyperlinks to sources, audio and video clips, and
interactive tasks as well as text.
The tutorial is organized in frames (not pages)
consisting of short blocks of information about the topic
in the frame title. The continuum is traversed by
proceeding serially along a main path through the
frames, stopping to examine embedded resources
whenever they seem of interest. There are many frames,
and it is not intended that the viewer intent on mastery
of content will travel through all of them in one, or even
two sittings. However, we have tried to make the
journey entertaining as well as informative. Travel
through the frames goes quickly, information is
conveyed in various ways, and many good stopping
places are evident.
Click at any point on the screen to advance to
the next frame. Move your mouse to show faint
arrows at the lower left corner of the frame.
Click on either of these arrows to show a menu.
To move backward one frame, click on
(Previous).
Jumps to frames elsewhere in the tutorial are
dark blue and underlined. To return to the
frame from which you jumped, use (Go, then
Previously Viewed). This two-step procedure
will probably be frequently used. Click on (Go
then Slide Navigator) to see a menu of frames
and to move to any point in the tutorial.
To end the tutorial, use (End Show)
Other types of embedded resources linked to
the main path are indicated by icons:
Website
Video
Audio
Respond to a task
(click on icon)
Acknowledgements
Individuals
Institutions
Moya Andrews
Charles Glassick
David Pace
Thomas Angelo
Samuel Guskin
Edward Redish
William Becker
Mary Huber
Eugene Rice
Simon Brassell
Pat Hutchings
Laurie Richlin
Barbara Cambridge Robert Hutchins
Leah Savion
William Cerbin
Dennis Jacobs
Anita Salem
Donetta Cothran
T. Mills Kelly
Donald Schon
Patricia Cranton
Carolin Kreber
Nancy Shanklin
K. Patrica Cross
Marcia Landen
Lee Shulman
Alix Darden
Suzanne Mabrouk
John Sullivan
Robert Diamond
Gene Maeroff
George Walker
Donna Duffy
Valerie O’Loughlin
Barbara Walvoord
Abilene Christian University
The Citadel
Elon College
Indiana University
Middlesex Community College
Notre Dame University
Rockhurst University
This project is sponsored in
part by an American
Association of Higher
Education (AAHE) Grant
Information and materials from more than 30 individuals across the United States and Canada as well as SOTL program
information from seven colleges and universities are used in this tutorial. All contributions are gratefully acknowledged.
© 2001 Samuel B. Thompson
Task A
1. How would you describe your experience with scholarship of teaching and learning?
Choose one: Extensive Moderate Hardly-Any None
2. OUTCOMES: List 3 topics or issues you hope to learn more about from this tutorial
STOP NOW! BEFORE YOU GO FURTHER, WE NEED YOUR RESPONSES TO
THIS FRAME. This task is DESIGNED to capture experience level of users and their
expectations/goals in freeform before influence by tutorial content. If you will be so
kind as to activate the button in the slide and type your responses, we’ll put the
information you provide to good use!
Continuum of Experience with SOTL
A Metaperspective
Αlpha
Omega
1A What & Why of SOTL
2A Initiating SOTL Programs
3A Bridges to Productivity
1B Origins of SOTL
2B Faculty SOTL Projects
3B Questions, Designs &
Resources
Boyer coined the term “scholarship of teaching” more than ten
years ago. The American Association of Higher Education
(AAHE) and the Carnegie Foundation for the Advancement of
Teaching have conducted initiatives to advance this scholarship
for several years. Now, a continuum of experience levels is
evident among academicians. At the “Alpha” end of the
continuum, people have heard the term but are unsure what it
means. These need perhaps a glossary and basic grounding in
the key developments over the last decade. In the middle are
people who may have launched campus initiatives or may be
wondering whether particular issues and projects could result
in scholarly products. At the “Omega” end, those at the
“cutting edge” of SOTL endeavor to construct and validate
developmental processes as sustainable bridges to carry faculty
from initial interest to stages of productivity that achieve goals
and count in reward systems.
This tutorial addresses the whole continuum of SOTL
experience from “Alpha to Omega.” The three sections of
the continuum correspond to the three units of this tutorial
and each phrase names a module … so two modules per
unit, six total. The first unit, features essential definitions,
concepts, rationale and other background. The second unit
features administrative “how to” and examples of campus
programs as well as faculty projects. The third unit
provides some research essentials to permit viewers to
launch their own projects. The “metaperspective”
afforded in considering the whole continuum permits
viewers to structure their own projects and campus
initiatives.
Alpha to Omega Continuum:Contents
Alpha
Omega
Unit 1A
Unit 2A
Unit 3A
•“The “What” of SOTL
•A Campus Example in Detail
•How Could I Do SOTL (Genres)
•Why SOTL?
•Task D: Refining Your Campus
Program
•Task G: Reflecting on Genres
•Task B: Reasons to Engage
•Examples on Other Campuses
•Approaches to Scholarship Via Classroom
Research Projects
•Task E: Applicable Features
•Where to Publish and Present
•Sources of External Funding
Unit 1B
•Task H: Classroom Research
•Overview
Unit 2B
•Reform Concepts
•Donetta Cothran – Kinesiology
Unit 3B
•Implementing Entities
•Valerie O’Loughlin – Medical Sciences
•Framing Questions
•Recent Articulations
•David Pace – History
•Task J: Examining Valerie’s Questions
•Task C: Critique of Origins
•William Becker - Economics
•Task K: Framing Your Question
and Evolution
•Rita Naremore – Speech and Hearing
Sciences
•Designs for Studies
•Dennis Jacobs – Chemistry
•Choosing Measures
•Leah Savion – Philosophy
•Guiding Questions in Choosing
Methodology
•Other Examples – Carnegie Scholars
•Task L: Designing Your Project
•Task F: Minute Paper
•Summary of Standards
Unit 1A
The What and Why of SOTL
•“The What”: A Conceptualization of Teaching-Related Activities
•Why SOTL?
•Task B: Your reasons to become engaged in SOTL
We seek understanding of what SOTL is and its relationship to teaching as well as reasons for becoming
engaged in SOTL.
A Conceptualization of Teaching Related Activities
Many conceptualizations of SOTL have been
offered over the years. This one is ours. Others
will be exhibited in the next module.
Every teaching-related act can be represented as
a point somewhere within the three elliptical
regions. Elliptical shapes are chosen solely for
convenience of representation. All deeds that
teachers perform in the name of carrying out their
teaching assignments can be imagined as
individual points within the large ellipse labeled
“teaching.” The centillions of such points include
acts of materials development, grading,
individual tutoring … every teaching activity in
or out of the classroom. Within this big red
ellipse of teaching acts, we can identify a subset
that meet special criteria for “scholarly teaching.”
The smallest ellipse (yellow) represents
“scholarship of teaching and learning.”
Scholarship of teaching and learning (SOTL) is
rooted in teaching but extends beyond it (out of
the red ellipse). SOTL involves creation and
dissemination of original work that makes a
useful contribution to knowledge and practice of
other teachers.
Teaching
Scholarly
Teaching
Scholarship of
Teaching &
Learning
Scholarly Teaching:
Teaching that entails certain practices of classroom assessment and
evidence gathering; teaching that is informed not only by the latest
ideas in the field but by current ideas about teaching generally and
specifically in the field; and teaching that invites peer collaboration or
review.
Scholarship of Teaching:
An act of intelligence or artistic creation becomes scholarship when it
possesses at least three attributes: it becomes public, it becomes an
object of critical review and evaluation by members of one’s
community, and members of one’s community begin to use, build
upon, and develop those acts of mind and creation.
(Definitions after Hutchings and Shulman)
More about Lee Shulman
More about Pat Hutchings
Though sharply defined in the picture, the boundaries of ellipses can
seem a bit fuzzy when classifying certain teaching-related acts. This
occasional fuzziness does not lessen the usefulness of this seemingly
simplistic conceptualization.
Relationship to Excellence
Faculty members should be encouraged to move
acts of teaching that are characterized by points in
the red toward the orange … in other words
toward scholarly teaching. To the extent that
teachers adopt practices of scholarly teaching,
their teaching will probably improve. Similarly,
their acts of scholarship will probably improve
their teaching, develop them professionally, and
add to our body of useful knowledge about
practice. However, we must stop short of
accepting either the proposition that scholarly
teaching implies excellent teaching or the
converse. Similarly scholarship of teaching does
not imply excellent teaching.
“Excellence” is often confounded with “scholarly
teaching” and “scholarship of teaching.” Scholarly
teaching practices do not necessarily confer
excellence. All seasoned faculty know of
colleagues who are excellent teachers … excellent
in terms of achieving superb outcomes in students
… without their meeting any of the criteria of
“scholarly teaching” Avoidable indignation and
opposition to SOTL initiatives are justifiably
engendered in faculty by the notion that “scholarly
teaching” and “scholarship of teaching” are
necessarily what one must do to be considered
excellent in teaching.
Plan
e
of th
eV
enn
Tea
ch
ing
An excellent deed of
teaching
A superior deed
of scholarly
teaching
An ineffective
deed of teaching
Dia
gram
Sch
o
teac larly
hing
Sch
olar
ship
Quality (or “excellence”) of any teaching-related
activity is an independent dimension not represented
in the plane of the Venn diagram.
No ascending hierarchy of excellence can necessarily be associated
with movement from left to right or bottom to top in the Venn diagram.
To represent excellence, we must invoke a third dimension
perpendicular to the plane of the diagram. In this three-dimensional
conception, each teaching-related deed represented by a point in the
diagram has associated with it an arrow (a “quality vector”)
representing excellence (or lack of it). The direction of the arrow
indicates whether the deed is above or below average in quality and the
length of the arrow indicates how far above or below average.
SOTL as Academic Activity vs. Campus Initiative
Into what realm beyond teaching does
scholarship of teaching extend? The rightmost
ellipse is the set of points representing deeds
that scholars perform in the name of research
and creative activity. Though research-based
activity to advance knowledge is the prevailing
notion of scholarship, George Walker tells us
that scholarship is not precisely synonymous
with research. Scholarship is reflective and
creative. It can result from reflecting on the
processes and outcomes of research or personal
teaching experience and creating something
new.
Boyer’s notion that scholarship of teaching and
learning spans the “tired teaching vs. research
debate” is naturally depicted in the position of
the yellow ellipse (with italicized text) between
the two big red ellipses.
Teaching
Scholarly
teaching
Scholarship of
teaching and
learning
Research and creative
activity
Distinction between scholarship of teaching and learning
as a domain of academic activity and as a campus
program:
•
SOTL as academic activity (yellow ellipse)
•
SOTL as a campus program can span all teaching
- related activities (leftmost three ellipses).
A SOTL program provides support to faculty for
movement toward the yellow ellipse; that is movement in
their teaching-related activities toward the right and
movement in their research-related activities toward the
left.
We can distinguish between SOTL as an academic activity and as a campus initiative. As an academic activity, what
scholarship of teaching and learning means in your institution defines the interior and boundary of the yellow ellipse. As a
campus initiative, what SOTL means may encompass the entire 3 leftmost ellipses. In other words, a SOTL initiative may
include all teaching related activities of faculty. Thusly conceived, a SOTL initiative can provide a resource-rich and
opportunity-laden home to faculty members who aspire SIMPLY to teach excellently as well as to those who perceive
themselves primarily as scholars (i.e.”researchers”). In fact, there is evidence that faculty who have not been attracted to
previous teaching improvement initiatives can be attracted to a SOTL initiative because of its research orientation. This
phenomenon may be visualized in the diagram as faculty converging to SOTL not only from the big red ellipse on the left but
also from the one on the right.
Why SOTL?
SOTL can improve teaching and learning … and more! SOTL has power to:
•
Unite Where Teaching and Pursuit of New Understanding are Shared Interests
Elaboration of the unifying potential of SOTL (Voice of Samuel Thompson)
•
Build Multidisciplinary Community
Elaboration of community-building via SOTL (Voice of Samuel Thompson)
•
Capture the Wisdom of Practice
Documentation of the deep and useful insights gained over time so that others can use them is encouraged by SOTL. The
documentation includes the teaching moves and materials that every seasoned faculty member has found to work especially
well.
•
Foster Mentoring of New Generations of Teachers
SOTL is a medium for the generativity that develops naturally in mid-life … seasoned teachers helping newer ones to learn
to teach more effectively.
•
Employ Reserve Intellectual Capacity
SOTL may increase the individual and collective scholarly productivity of the faculty, partly by providing an avenue to
products from the immense investments that already go into teaching and partly by providing an area where faculty
productivity can thrive when scholarly efforts in other areas are flagging or no longer interesting.
•
Recreate the Concept of Scholarly Excellence
SOTL brings the spirit and standards of scholarly excellence to teaching .. standards that have traditionally been applied to
other fields of scholarly endeavor but not to teaching.
Why SOTL? (Cont.)
SOTL Honors and Enriches the Seasons of Academic Life
Elaboration of the role of SOTL in the seasons of academic life (Voice of Craig Nelson)
• Power of Ideas
• Multiple and Competing Commitments
• Faculty Role
• Need to Step Out
• Students
• Marginality
• Public Self
• Courage
Knefelkamp L. L. (1990) Seasons of Academic Life: Honoring Our Collective Autobiography
Liberal Education 76. 3
Knefelkamp’s characterization of Seasons of Academic Life seems worthwhile reading for any faculty
member. This particular article may serve as a basis for a meaningful exercise in deepening understanding
of “Why SOTL?” The recommended exercise is to read Knefelkamp’s description of each season and then
pause to reflect on the possible roles that scholarship of teaching and learning may take during it and the
implications for the faculty member.
Task B: Reasons to Engage in SOTL
•
What reasons to become engaged in SOTL … the ones discussed in the tutorial
or others … are most important on your campus?
•
What reasons might be irrelevant to faculty on your campus?
We pause here for processing of the foregoing and for reflection. Please activate the button and e-mail your reasons.
We’ll compile categories of reasons and disseminate the results.
Unit 1B
Origins and Evolution of SOTL
•Overview
•Reform Concepts
•Implementing Entities
•Recent Articulations
•Task C: Critique of Origins and Evolution
As we progress to the second and subsequent modules, we’ll continue to broaden and deepen our concept of “what
SOTL is” and “why become engaged in SOTL.”
We invite your peer review and critique of this module. At the end of the module, we will ask you (Task C) to tell us
how you would modify the framework and which elements you would add or omit in each the first four categories
above. We’ll consider your inputs and may incorporate them in future versions of this tutorial in order to build a
better framework for everyone.
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
Implementing Entities
Recent Articulations
Classroom Assessment/
Research
AAHE
Scholarship of Teaching
Carnegie
Scholarly Teaching
New Epistemology
Pew
Model of SOTL
New American Scholar
Lilly
Relation of ST to SOTL
Conception of Teaching
Scholarship Assessed
The “work in progress” in this “tree” is our tentative attempt to build a succinct framework for essential background
in SOTL. Of course, the number of individuals, forums, articles, and books that have significantly contributed to
SOTL is too enormous to include comprehensively. And who is to say which of these is “essential”? So this whole
exercise is somewhat arbitrary and no doubt reflects the bias of our own limited knowledge and experience.
Not a New Idea
The quote is taken from Hutchin’s inaugural address as the fifth
president of the University of Chicago in 1928 and is presented
as evidence that the concept of SOTL existed long before Boyer
popularized the term. The last sentence suggests that faculty in
all departments should carry on experiments in undergraduate
teaching and learning … a fundamental activity of SOTL. The
sentence further suggests that graduate students preparing for
academic careers should learn from these experiments.
Note that Hutchins also presaged the plight of many
undergraduates, and the need for programs to prepare graduate
students to teach … programs like the Preparing Future Faculty
(PFF) initiative of the 1990’s.
In the first sentence, Hutchins also expressed a truth still not
internalized by many faculty who prepare graduate students in
doctoral institutions today … research productivity that
advances the discipline ends with the dissertation for most
Ph.D’s. Today, the vast majority (95% is a figure often quoted)
of new Ph.D’s who take positions in academe do so in teaching
institutions rather than research universities. In other words,
though they may acquire their Ph.D’s in research universities as
researchers, teaching is what they will primarily do in academic
careers.
“What is certain is that most Ph.D.’s
become teachers and not productive
scholars as well. [A Ph.D. candidate who
plans to be a teacher] must know his field
and its relation to the whole body of
knowledge. It means too that he must be
in touch with the most recent and most
successful movements in undergraduate
education, of which he now learns
officially little or nothing. How should he
learn about them? Not in my opinion by
doing practice teaching upon the helpless
undergraduate. Rather he should learn
about them through seeing experiments
carried on in undergraduate work by the
members of the department in which he
is studying for the degree….”
Robert Maynard Hutchins
“Scholarship of Teaching” Coined
Ernest Boyer popularized “scholarship of teaching” in a famous monograph Scholarship
Reconsidered: Priorities of the Professoriate. 1990 Princeton, NJ: The Carnegie
Foundation for the Advancement of Teaching.
Boyer argued that the concept of scholarship must be broadened to include not only basic
research but other kinds of intellectual work in which faculty engage. He suggested four
types of scholarship: discovery, integration, application, and teaching. All of the elements
of “Reform Concepts” build on Boyer’s theme of broadening the traditional view of
scholarship.
Boyer disseminated the notion of a scholarship of teaching but did not define clearly what
this scholarship would be. He thereby touched off a decade of academic thought and
controversy over this topic. As a result, our conceptualization of this scholarship is a bit
clearer today (but only a bit!)
Reform Concepts:
Classroom Assessment/
Classroom Research
Reform Concepts
The several reform concepts presented in this section of the tutorial may be viewed as multiple origins or histories of SOTL.
They are somewhat overlapping and sometimes difficult to isolate.
Classroom Assessment Techniques (CATs)
K. Patricia Cross and Thomas Angelo pioneered this “do-it-yourself” approach to assessing conditions of learning in
classrooms. Their 1993 book, Classroom Assessment Techniques: A Handbook for College Teachers (2nd Ed) JosseyBass is a famous work of scholarship of teaching and learning, reported to have sold more than 35,000 copies. Thousands of
college and university teachers use CATs in their classrooms today, especially versions of the “one-minute paper,” and there
is evidence of improved learning as a result. Angelo and Cross contributed to the current focus on learning but also hoped
their book would influence faculty to go beyond application of CATs in the classroom … that it would in fact spark
systematic classroom research which would in turn become a popular form of SOTL nationally … a form more useful to
faculty than traditional educational research. The adoption of CATs became widespread, the adoption of classroom research
much less so.
Classroom Research
In 1996, Cross with Mimi Steadman published a sequel book, Classroom Research: Implementing the Scholarship of
Teaching, Jossey-Bass placing CATs in a broader research context. Again the hope was to institutionalize classroom
research. However, as of yet, classroom research is still not nearly as widely employed as are CATs.
More on CATS
More on Classroom Research
Reform Concepts:
New Epistemology
“On the high ground, manageable problems lend themselves to
solution through the use of research-based theory and technique.
In the swampy lowlands, problems are messy and confusing and
incapable of technical solution. ...the problems of the high ground
tend to be relatively unimportant to individuals or to the society at
large, however great their technical interest may be, while in the
swamp lie the greatest problems of human concern. ...Shall [the
practitioner] remain on the high ground where he can solve
relatively unimportant problems according to his standards of rigor,
or shall he descend to the swamp of important problems where he
cannot be rigorous in any way he knows how to describe?”
Donald Schon
Donald Schon posed a “dilemma of rigor or relevance” for research epistemology. In this eloquent quote,
he calls for new research epistemology for the scholarship of teaching and learning based on reflection by
teachers on actions taken in their practice.
Schon, D. “The New Scholarship Requires a New Epistemology,” Change, Nov./Dec. 1995 27 ( 6) p. 28
Reform Concepts:
The New American Scholar
Adapting the learning cycle of David
Kolb to SOTL, Gene Rice sees
traditional scholarship … particularly
the so-called “social science model”
… as characterized by reflective
observation and abstract analytic
knowing. It therefore resides in the
lower right quadrant of his diagram.
He asserts that SOTL needs to be
rooted more in active practice and
concrete connected knowing. Said
another way, scholarship of teaching
and learning should take forms that
properly reside in the upper left
quadrant
CONCRETE
CONNECTED
KNOWING
ACTIVE
PRACTICE
REFLECTIVE
OBSERVATION
ABSTRACT
ANALYTIC
KNOWING
R. Eugene Rice. 1996. Making a Place for the New American Scholar. AAHE. p. 14
Reform Concepts:
Conception of Teaching
&
Scholarship Assessed
Conception of Teaching: The argument here … and Lee Shulman is a major proponent … .is that
teaching is not just technique (though technique has gotten the lion’s share of attention in teaching-improvement
efforts) but an enactment, rather, of our understanding of our disciplinary, interdisciplinary or professional field and
what it means to know it deeply. As Shulman wrote recently "a scholarship of teaching will entail a public account of
some or all of the full act of teaching—vision, design, enactment, outcomes, and analysis—in a manner susceptible to
critical review by the teacher’s professional peers, and amenable to productive employment in future work by
members of that same community."
Scholarship Assessed: Glassick, Huber, and Maeroff (1997) in Scholarship Assessed: Evaluation of
the Professoriate. Jossey-Bass review the history of scholarship in American academe and describe general standards
for all forms of scholarship. These standards “pull together” many of the concepts noted in this section. More detail on
the standards
Summary of Reform Concepts. All
reform concepts contain an implicit call to broaden and
generalize traditional notions of scholarship in order to make progress against relevant problems which vex
undergraduate teaching and learning at levels of individual teacher and course. George Walker advises us to avoid
unnecessary limitations in our notions of scholarship. The goal is to produce a body of scholarship that may deviate
from conventional forms, and that may not be applicable over disciplines, categories of student, or institutional type
but which nevertheless is more used by teachers and contributes more to the learning of students than does past
educational research.
Implementing Entities:
AAHE
Implementing Entities We focus on broad national initiatives. There are far more entities (including many academic
institutions) deserving acknowledgement for implementation than those included here.
AAHE (American Association of Higher Education)
The AAHE has sponsored many initiatives over the years
to advance teaching and its scholarship. Barbara
Cambridge directs the Teaching Initiatives. Major
initiatives include assessment, service-learning, peerreview of teaching and teaching-learning technologies. We
highlight faculty “roles and rewards” below because its
historic origin is associated with Boyer’s introduction of
“scholarship of teaching.”
AAHE FFRR (Faculty Forum Roles and Rewards) It
was recognized at the time of Boyer’s 1990 monograph,
that overhaul of the faculty reward system needed to go
hand-in-hand with advancement of scholarship of
teaching, if the latter was to be more than a transient topic.
The FFRR has met annually since 1990 to address faculty
roles and rewards.
Robert Diamond’s depiction of the factors bearing on
faculty work reflects the agenda of AAHE FFRR.
Initiatives have been undertaken in the 1990s’ to favorably
influence each of the five factors in the figure toward
greater incorporation of SOTL in faculty work.
Factors Influencing What Faculty Do
Diamond, R.& Adams, B. The Disciplines Speak AAHE 1995 p.7
Implementing Entities:
Carnegie
THE CARNEGIE FOUNDATION FOR THE ADVANCEMENT OF TEACHING
The Carnegie Foundation has a number of exemplary programs and resources that encourage scholarly teaching and
the scholarship of teaching and learning. Among the programs that may be explored
are: Cultures of
Teaching and Learning in Higher Education, Higher Education and the Development of Moral and Civic
Responsibility (co-led by Tom Ehrlich, former President of Indiana University), Preparation for the Professions,
Teacher Education, and the long running U.S. Professors of the Year. Four key programs to click on:
Rethinking the Doctorate. George E. Walker, Vice President for Research and Dean of the University Graduate
School at Indiana University, leads this new, five-year project.
Knowledge Media Laboratory. A gallery of course portfolios illustrating various approaches to the scholarship of
teaching. Relation between course portfolios and SOTL
Resources. Click on “e-Library” to read key SOTL references on-line and to access an annotated bibliography of
SOTL in higher education.
CASTL Carnegie Academy for the Scholarship of Teaching and Learning
Implementing Entities:
Carnegie (cont.)
CASTL
Carnegie Academy for the Scholarship of Teaching and Learning (CASTL)
•
Pew National Fellowship Program for Carnegie Scholars
For faculty from higher education - brings together outstanding teacher-scholars from a variety of academic fields and
institutions to pursue investigations of issues in teaching and learning.
•
Teaching Academy Campus Program
Coordinated for the Carnegie Foundation by AAHE. Institutions in higher education that are prepared to make a public
commitment to the scholarship of teaching and learning are invited to join the Campus Conversations program. As
progress is made, campuses are invited to apply for small grants for the Going Public phase. CASTL now is also
encouraging the formation of National Networks of Teaching Academy Campuses. The AAHE CASTL Campus Program
Web Center includes a number of resources for thinking about SOTL, including reports from campus conversations, a
list of going public grants and selected resources.
•
Work with the Scholarly and Professional Societies
Carnegie is encouraging societies to increase their emphasis on teaching and on the scholarship of teaching and learning.
This is a continuation of work under AAHE FFRR culminating in two publications which contain perspectives of
disciplinary societies on Boyer’s four scholarships: The Disciplines Speak AAHE, 1994 and The Disciplines Speak II
AAHE, 2001.
Implementing Entities: Pew
Peer Review of Teaching
Course Portfolios
Pew Charitable Trusts:
•CASTL (with Carnegie Foundation)
•Peer Review of Teaching
Peer Review of Teaching is an ongoing area of
work intertwined with the scholarship of teaching
and learning in several ways.
Peer Review
has been an emphasis of
AAHE since 1994 and continues to be supported
by the Pew Foundation and other grant sources.
The course portfolio has evolved as the principal
means by which peer review of teaching is
effected. Two notable current programs to advance
course portfolios, both supported by Pew, are
centered at the University of Nebraska Lincoln
and Samford University
Individual course portfolio projects and SOTL
projects are so closely related as to often seem
inseparable. Example of this inseparability.
Perhaps the best way to ensure that quality teaching is
recognized, valued, and rewarded is to improve the
means of identifying and documenting teaching
effectiveness. Course portfolios afford a
comprehensive yet efficient means of documenting the
intellectual work of teaching a particular course.
Through such a portfolio, a faculty member
documents course design and execution, including
results in student learning. In this way, teaching can
be understood and presented as a form of scholarship,
utilizing the accountability through peer review that
already exists in higher education. A course portfolio
can be used as an instrument for exhibiting teaching
effectiveness, a framework for cultivating scholarship,
and a vessel for conveying one's work to appropriate
publics, including promotion and tenure committees.
Implementing Entities:
Lilly Conferences on
College Teaching
Lilly Conferences have encouraged scholarly
work in teaching and provided a venue for
faculty members to discuss and disseminate
such work for more than 20 years.
Lilly Conferences
are retreats that combine workshops, discussion sessions, and major addresses, with lots of
opportunities for informal discussion about excellence in college and university teaching and learning. Internationally-known
scholars join new and experienced faculty members and administrators from all over the world to discuss topics such as
gender differences in learning, incorporating technology into teaching, encouraging critical thinking, using teaching and
student portfolios, implementing group learning, and evaluating teaching. The welcoming "Lilly Spirit" and the high level of
scholarly attention to teaching and learning enable everyone to contribute to the forum.
Recent Articulations:
Scholarship of Teaching
Scholarly Teaching
Scholarship of Teaching and Learning
This recent articulation (Shulman,Change 31 (4) 1999) is widely accepted. Note that it is not quite a definition; it offers
necessary conditions for scholarship but not sufficient conditions. The articulation does suggest a broadening of the
concept of scholarship as advocated in the Reform Concepts Applications of the broadened concept of scholarship.
Scholarly Teaching
This articulation (Hutchings & Shulman Change 31 (5) 1999) is also widely accepted. One of it’s principal features is to
include knowledge of recent and relevant developments in pedagogy as well as the traditional notion of recent and
relevant developments in the field as a criterion for scholarly teaching.
We have left the topic of “Implementing Entities”, and moved to “Recent Articulations” (the rightmost branch of the “origin
and evolution tree”. ) A decade after Boyer’s “Scholarship Reconsidered”, the meaning and implications of scholarship of
teaching and learning are topics of evolving thought and discussion. Many more entries than the four shown could be
included in the right branch and these would be no less worthy than the sources shown. However, our purpose is to provide a
sample rather than a comprehensive list. Our apologies to authors for failing to list their contributions.
Recent Articulations:
Exploring Scholarship of Teaching
The Model of Kreber & Cranton
Carolin Kreber and Patricia Cranton see
development of scholarship of teaching as
a process of reflection on experience and
research in teaching.
The second, third and fourth concentric
circles represent content, process, and
premise reflection respectively. The three
spokes partition the circles into regions
representing the three domains of
knowledge (instructional, pedagogical, and
curricular) shown on the periphery of the
figure.
Within each knowledge domain, learning
is characterized by various forms of
instrumental, communicative, and
emancipatory learning processes, which
are derived from Habermas’s three forms
of knowledge.
Development of SOTL in this
conceptualization is thus comprised of nine
distinct types of learning.
Recent Articulations:
Conceptualizing Scholarly Teaching
and Scholarship of Teaching
Laurie Richlin’s conceptualization reflects her extensive experience as co-director of the Lilly Conferences on College
Teaching and in helping faculty with teaching-related investigations. Her conceptualization shown in the slide is closely tied
to the ideas of classroom research. She partitions a developmental process for SOTL according to the aspects that can be
considered scholarly teaching and scholarship of teaching. Her steps in the part of the process called scholarly teaching are
similar to those in our classroom research schematic.
Richlin, L in Kreber, C. (Ed.) New Directions in Teaching and Learning 86 2001 (In Press)
Task C
Think about your own knowledge of SOTL. What additions would you
make to the framework “Origins and Evolution of SOTL” in each
category?
Reform Concepts
Implementing Entities
Recent Articulations
We pause to process, reflect on, and improve the “Origins and Evolution of SOTL.” WE NEED YOUR HELP WITH THIS!
Please suggest alterations to the structural framework of the tree as well as to the specific elements in the branches. Just click
on the button on the frame to e-mail your suggestions. We’ll be most grateful for your contributions, and they will result in a
better product for everyone.
Unit 2A
Initiating SOTL Programs
•A Campus Example in Detail ( Indiana University Bloomington)
•Task D: Refining Your Campus Program
•Examples on Other Campuses
•Task E: Applicable Features
We are leaving the left region (Unit 1) of the Alpha to Omega Continuum and transitioning to the middle region (Unit 2).
A Campus Example in Detail:
Using SOTL to Make Change Happen
•
•
•
•
•
Identifying Key Resources
Getting People Involved
Changing the Institutional Culture
Assessing SOTL Program Impact
Task D Refining Your Campus Program
An alternative path through this unit is to view “Assessing SOTL Program Impact” first. After studying “A Campus
Example in Detail”, you will be asked to reflect on whether any aspects of this example are applicable to your own campus
(Task D).
Three Stages of Campus Program Goals
Participation
Scaffolding
Scholarly Productivity
Campus program development may be viewed in a framework of three stages. The first stage is to build
awareness and participation. The second is to erect essential scaffolding to support faculty in this line of work.
The third is to maximize scholarly productivity.
It may be helpful to bear the above three stages in mind while studying this detailed example of a campus
program. To assess the extent to which goals of the first two stages are achieved, look for evidence distributed
through the frames of this unit. Evidence of scholarly productivity will be largely found in the following unit
“Faculty SOTL Projects.”
What are the Key Resources?
The Ingredients:
–
–
–
–
Massive administrative support
A dedicated director
A faculty advisory council
A core of faculty members who are willing
to engage in this work and share it with
others
Massive administrative support
Moya Andrews is committed to the campus SOTL initiative and
an architect of it from its inception. Her administrative support
is massive and visible to all. She provides a budget for the
SOTL initiative. However, she also contributes vitally by
staying abreast of SOTL-related developments and directing the
campus initiative. She makes public statements, writes
messages to faculty and invitations to department heads and
other key individuals. She often introduces featured presenters
at events.
Message to faculty regarding SOTL
Moya Andrews
Dean of the Faculties
and Vice Chancellor
for Academic Affairs
Massive Administrative Support (Cont.)
“Expanding our Vision of Indiana University’s Research
Mission: The Scholarship of Teaching and Learning”
Vice President Walker’s talk at the first SOTL main event was
subsequently delivered to representatives of research universities at
invitation of AAHE and distributed nationally.
Other SOTL resources by George Walker:
A statement that SOTL is important.
A brief message to faculty
regarding the rationale and significance of
SOTL.
The campus SOTL initiative received a huge boost in administrative
support when George Walker became a partner in it. From that time, the
SOTL initiative was impelled by the two primary academic offices on
the campus. This partnership facilitated the movement of faculty interest
toward SOTL from research as well as teaching. George Walker
describes some of the administrative support available for SOTL.
George Walker is currently a senior scholar
at the Carnegie
Foundation leading a new study: Rethinking the Doctorate
George Walker’s contributions exemplify what a distinguished
researcher and senior administrator can do to help advance teaching and
its scholarship.
George Walker
Vice President for
Research and Dean
of the Graduate
School
A Dedicated Director
A campus SOTL initiative needs a
person charged with responsibility
to take actions on ideas, to follow up
actions to ensure completion, to
convene groups, to plan and stage
events, to coordinate faculty and
administrative viewpoints, to
“cheerlead,” and to serve as a stable
point of contact. If the director is a
faculty member, release time is
essential. The SOTL initiative needs
to be a significant part of the
director’s portfolio.
Samuel Thompson
E-mail to Samuel
Director of the Campus
SOTL Initiative
The Faculty Advisory Council
•
•
•
•
Insure Representation Across Disciplines
Tap Respected Faculty
Let FACULTY Define Initiatives
Support FACULTY Initiatives
Success may depend on the initiative being of, by, and for the faculty. Advisory council members can steer
the enterprise. They can be inspired by the prospects of what the initiative can do and so contribute their
own time and effort without acknowledgement in tasks which they, themselves, often define. This is
“bottom up” infusion of energy.
Advisory council members should be chosen carefully. Ensure membership is representative. Primarily use
senior, respected faculty: they know the ins and outs of the institution and their presence validates and
lends prestige to the SOTL effort.
Make sure the initial SOTL thrusts on the campus reflect consensus of faculty on the advisory council. Let
faculty members feel empowered to influence key decisions. Advisory council members suggested
doubling the amount of the SOTL grants proposed by administration, defined and addressed the human
subjects issue for SOTL, and took the initiative to bring the research establishment into partnership.
Administration supported the advisory council initiatives and decisions.
A Core of Faculty Members Willing To Engage
in This Work and Share It With Others
Because this unit of the tutorial deals with launching campus SOTL initiatives, much emphasis is on
administrative considerations. However, all the administrative brilliance in the world will not attract or
sustain faculty in a SOTL initiative if they do not see relevance to academic issues important to them.
If a campus already has scholars of teaching and learning, these may provide examples or kernels of
issues which engage colleagues. But what if such scholars and issues are not readily identifiable?
Engaging faculty should not be a struggle. The vast majority of faculty have a natural curiosity about
teaching issues and a latent desire to do teaching-related inquiry. Making time and resources for reflection
and inquiry available to faculty in light of their teaching loads and other responsibilities might be a
challenge but as Shulman has suggested, students already provide faculty with more information than
they normally use. Thus additional work, if any, in information-gathering should be minimal.
A possible two-step procedure is to conduct a kind of “needs assessment” … formal or informal … to
identify campus issues for possible investigation; e.g. academic incivility. The second step is to connect
an identified need to language and issues that people care about … framing the issue to make it attractive
in the particular setting.
If engaging faculty in issues and framing them is of concern to you, you might keep this concern in mind
when viewing other examples of campus programs , the issues investigated in Faculty SOTL Projects, and
Framing the Question.
Getting People Involved
• Get Out the Crowd to Initial Events
Kick off the SOTL initiative with a “Celebration of Teaching.” Send personal invitations to winners of teaching
awards, recipients of teaching-related grants, teachers of pedagogy courses for graduate students, and all others who
have demonstrated interest in teaching in an identifiable way. Send personal invitations also to deans, chairs, and
P&T committee members. About 200 faculty showed up for our Celebration, more than in anyone’s memory for a
teaching-related event. Personal attention has continually proved important in building a SOTL community “one
faculty member at a time.”
• Showcase Campus Scholars
Identify and encourage faculty who wish to pursue scholarly inquiries into teaching or have already done so.
Make these visible as models and mentors in high profile events rather than bringing in outside speakers.
• Identify Campus-wide Needs and Interests
Use information-gathering opportunities to learn the needs, goals and activities of people and units across the
campus. How can SOTL help to fill their needs? Focus on issues for investigation that cut across disciplinary
boundaries.
• Partner with Stakeholders
Seek to make the SOTL initiative inclusive of all academic and non-academic units on the campus. Seek to involve
undergraduate and graduate students, professional staff in academic affairs and student affairs, librarians, and faculty
in all seasons of academic life. Seek as well to support and cooperate with all other teaching-related initiatives rather
than to compete with them. Think of ways and themes for collaboration that advance the goals of stakeholders as
well as those of the SOTL initiative.
Getting People Involved:
An Example
As an example of partnering with stakeholders, consider librarians. They generally see themselves as academics but do not
always feel that others see them in that light. When invited to join and help grow the SOTL initiative, they were very pleased
for the opportunity to participate with faculty in an obviously academic enterprise. The librarians enriched SOTL with talent,
initiatives, and resources.
Librarians:
•volunteered as consultants to individual faculty investigators needing assistance in locating unfamiliar teaching and
learning literature.
• gave presentations to show use of information technology to access SOTL-related resources.
• made free document delivery systematically available to scholars of teaching.
•acquired multiple copies of critical books and put them on reserve for “SOTLites”.
•served on working committees of the Faculty Advisory Council.
The librarians, in turn, found participation rewarding both individually and collectively.
Getting People Involved (continued)
•Connect with National Initiatives
This campus model as well as all six campus SOTL models described later in this tutorial draw direction and resources from
AAHE and Carnegie national initiatives … and frequently publicize the national connections to the local community. Other
connections to related national initiatives include Peer Review, Problem-Based Learning, and Preparing Future Faculty.
•Exhibit Administrative Support
It’s important not only to have administrative support but to exhibit it. One way is by regular participation of key
administrators in SOTL events. Serving food exhibits institutional backing with funds as well as increasing participation,
especially for events held at lunchtime or the end of the working day. Examples of SOTL events
•Provide Scaffolding for Faculty
If a SOTL initiative is to achieve it’s objectives, getting faculty interested or involved is not enough. Many need
“scaffolding” … various forms of support … on the pilgrimage from initial interest to completion of scholarly work. Planning
for this scaffolding and building it into the campus model is essential.
Changing the Institutional Culture
• Change the faculty member’s view of what faculty members do
Under the aegis of SOTL, employ administrative resources to acknowledge and reward a fuller range of faculty work.
• Expand the range of activities that get rewarded
Many campuses have faculty who have done scholarship of teaching for years. However, nobody necessarily knows who
they are if such scholarship does not count in reward systems. In this campus example, small grants ($1000) were offered at
the start of the SOTL initiative for scholars of teaching to present work already completed or well underway. These unusual
“presentation” grants for work that may have already been done made existing scholars and their projects visible campuswide as models. Later conventional “research” grants to undertake new investigative projects were offered.
Many campuses already have small grant programs. These can be tweaked to foster SOTL. A spin-off benefit can be greater
productivity for a given institutional investment in grants; e.g. a grant that was previously to implement a teaching
innovation will, as a SOTL project, also require data-gathering and analysis to assess the effect of the innovation as well as
dissemination of results to peers.
• Expand the range of rewards
Opportunities to present their work to large interdisciplinary audiences in high profile campus-wide events; to have their
abstracts, biographical sketches, and photographs widely disseminated; and to experience the support and confidence of the
central administration proved enough reward to some faculty members for them to offer themselves as SOTL
presenters(without grants). For some faculty members, finding a community that valued their work to improve teaching and
that helped them find opportunities for professional growth and recognition beyond the campus and beyond opportunities
available in their departmental and disciplinary cultures also proved rewarding.
Changing Institutional Culture: One Example
IU FACULTY SUMMARY REPORT PRIOR TO ACADEMIC YEAR
1999-2000
TEACHING ACTIVITIES
A. Courses taught (weekly contact hours reported by course number in
tabular form).
B. Development or major revision of course(s) during the year.
C. Dissertation, Research and Field Work Committees
D. Teaching awards and honors, including those of your students.
Indiana University’s summary report form for individual teaching activities required of each faculty member annually was as
shown above prior to Fall 1999. It is similar to that used in many institutions. There are also sections (not shown) for
Research & Creative Activities and Service.
Look at the categories of reportable teaching activities. Category B is generally taken to mean curriculum development or
major revision of curriculum. Activities associated with improvement of pedagogy or with demonstrable evidence of
improved student learning are not encouraged on such a report. Indeed, the lack of a place to report pedagogical activities on
the form is a tacit statement about institutional values and rewards.
Changing Institutional Culture: One Example (cont.)
IU FACULTY SUMMARY REPORT (REVISED) IN ACADEMIC YEAR
1999-2000 AND THEREAFTER
TEACHING ACTIVITIES
A. Courses taught (weekly contact hours reported by course number in tabular
form).
B. Activities directed at improving instruction, learning, or course
administration. (Please describe rationale for/description of
innovations, methods/measures for assessing outcomes, and results.)
Please note: Scholarly activity related to teaching and learning (e.g.
investigation/research, dissemination/publication of results) should be
reported under the section on Research/Creative Activities.
In Academic Year 1999-2000, the Faculty Summary Report form for the Bloomington campus was revised to include the
new Category B shown in red. The former categories B, C, and D respectively become C, D, and E on the new form. This
revision was prompted by the SOTL initiative.
Changing Institutional Culture: Another Example
An initiative of Craig Nelson, a faculty
member committed to the campus SOTL
initiative and a Carnegie Scholar, led to an
institutional policy of partially funding travel
of faculty members to attend teaching-related
conferences. Candidates for support must
show demonstrable connection to their
scholarly interests in teaching and learning
and provide a quid pro quo to the campus in
return.
Craig Nelson is a co-founder of the campus
SOTL initiative. His contributions are many.
It’s very helpful to have a teacher-scholar of
his prominence involved!
Craig Nelson
Biology
E-mail to Craig
Changing Institutional Culture
Further Examples
Input from the Faculty Advisory
Council and interested faculty
researchers influenced the
human subjects process to
facilitate SOTL activities.
Input from the Faculty Advisory
Council and interested faculty
researchers facilitated access to
institutional data (registrar, etc.)
for SOTL researchers.
These achievements were important steps in “developmental scaffolding” for faculty embarking on SOTL projects. An
aversive institutional review board decision regarding students as human subjects or inability to gain access to essential
background data regarding the students in the study can be lethal to a SOTL project.
Assessing SOTL Program Impact:
Participation Statistics 1999-2001
Campus Participation in SOTL Program of Events
David Pace provides an overview of SOTL program impact.
14 main events in 1999-2000 program; 11 in 2000-2001 – about one every two
weeks in fall and spring terms
108 campus units represented ( 74 academic departments/ programs)*
Participants:
496 individual faculty members
286 tenure/tenure track (22% participation rate for T/TT)
51 individual staff
237 graduate students
Total: 784 individuals (63% faculty, 30% graduate students, 7% staff)*
Total attendance: 1598 (53/event in 1st year and 77/event in 2nd)*
*Greatly exceeding expectations of many observers
Selected Faculty Comments about
the Campus SOTL Initiative
We need SOTL
The events I attended were terrific. Keep it up. And
keep me in the loop.
I was not able to participate directly in most SOTL
events, due to numerous personal and professional
obligations. But I was impressed by the degree to
which SOTL spurred conversation on teaching
among my colleagues. I also met with one SOTL
participant to consult on his research project, and I
was impressed with the enthusiasm he was
bringing to his teaching and the related project.
SOTL has been great for this campus (and, more
importantly, our students), and I plan to participate
enthusiastically next year. Thanks!
SOTL is an outstanding source of intellectual
support for the graduate school's efforts in
"preparing future faculty."
I realize that the program is in some sense intended
to support and provide a forum for people producing
the scholarship of teaching and learning. However,
please do keep in mind how useful it can be to
those of us simply working on teaching.
I think you are doing a wonderful job, and I fully
support your work
SOTL is a very valuable asset to the university and
the School of HPER. Please continue with your
excellent efforts.
I think you are beginning to make substantive
improvements to the learning environment and are
slowly raising awareness in the general faculty about
the importance of teaching and how this may be
approached through scholarship.
I am afraid I have a rather dim view of "scholarship"
about teaching or learning. Teachers should teach,
learners should learn. Scholarship should be
reserved for inquiry in genuine subjects.
Scarce resources would be better devoted directly to
teaching (e.g. smaller class sizes) or traditional
substantive areas of research
The ratio of positive to negative comments is actually
much higher than in the sample here.
Assessing SOTL Program Impact: Scaffolding
Support for SOTL by research establishment provided on
same basis as research in the disciplines (matching funds,
summer fellowships, etc.)
Campus colleagues with specialized expertise voluntarily
contribute time and effort to consult with scholars of
teaching.
Growing constellation of linked efforts affording opportunities
and support (course portfolios, PFF, etc.)
Large informal “SOTL community” on campus Samuel
explains why multidisciplinary community is important to
SOTL (beginning at 2 minutes: 10 seconds into the audio
clip)
Scholarship given birth under SOTL initiative begins to gain
recognition (and invitations) beyond campus
A visiting scholar of teaching and
learning, Professor E. F. Redish,
Department of Physics, University
of Maryland, remarked that he had
not previously experienced such a
multidisciplinary community of
scholars investigating issues of
teaching and learning.
After meeting members of this
community, Lee Shulman,
President of the Carnegie
Foundation for the Advancement
of Teaching, echoed similar
sentiments ... that this SOTL
community has a truly
extraordinary robustness and
interdisciplinary character.
TASK D: Refining Your Own Campus SOTL
Program or Plan
Think for a minute about your own campus. What administrative priorities or
structures might help in your campus level SOTL program? What elements of
the preceding example, if any, may be applicable? What hurdles might need
to be overcome on your campus?
Pause here to reflect on administrative features and obstacles on your own campus in the context of the Bloomington
example. Press the button and share your thoughts. Your comments may prove useful in unforeseen ways. Besides, we’re
REALLY interested!
Other Campus Examples
•
•
•
•
•
•
•
Elon College
The Citadel
Rockhurst University
Abilene Christian University
Notre Dame University
Middlesex Community College
Task E – Applicable Features
Campus examples of SOTL initiatives from six different institutions of widely varying types are exhibited in the following
frames. All of these campuses are participants in the AAHE Campus Program
After surveying these examples, you’ll be asked to identify features of them that might be applicable to your campus program
(Task E)
Elon College
• Multidisciplinary, Multiyear, $72,000 Investment
– $6000 projects in each of 3 years
– Projects directed by faculty-student research teams
– Learning for BOTH student and teacher
– Eight projects selected in years 1 and 2
• Create intellectual engagement
– New thinking in diverse fields
– Application of learning to life
– Opening spaces for reflective integration
The magnitude of the Elon investment in the
SOTL projects is notable for a school with about
4000 students. The projects focus on conditions of
learning and the individual course is the unit of
study. The use of student-teacher research teams
to carry out each project is a distinctive feature.
Elon jointly with Western Washington University won
the first AAHE “Going Public” grant. John G. Sullivan,
Powell Professor of Philosophy, kindly provided the
information used in this tutorial. For more information
The Citadel
•
•
Mission
– Increased campus awareness of and participation in SOTL
Focus
– Communication, Resources and Continuing Education
– Self-selected research projects
• Highlights
– Biweekly, participatory meetings with assignments
• Effectiveness
– 15% of full-time, tenure-track faculty at bi-weekly meetings
– 12% of full-time, tenure track faculty in classroom research
• Administrative Support
– Attendance at functions
– Financial support
The Citadel SOTL initiative got a big boost from a
Mimi Steadman workshop on classroom research.
The Citadel SOTL website
is rich in resources
and ideas. It’s well worth a browse. One distinctive
feature is the schedule of “assignments” for the year
given to participating faculty.
Professors Alix Darden and Suzanne Mabrouk
spearhead work in this initiative. Suzanne can be emailed through the website. Alix was selected as a
Carnegie Scholar for 2001-2002. E-mail toAlix
Rockhurst University
• Beginnings
• Products
– (Fall 1998) All University Symposium
– (Spring 1999) Follow-up Symposium
Year-long Carnegie faculty seminar
– (2000-2001) Carnegie faculty seminar •
continued
• The Rockhurst “Carnegie Seminar”
–
–
–
–
Central Questions
Seminar Members
Discussions
Methods
Rockhurst, a small (3000 students) independent university
in Kansas City, conducted a year-long Carnegie seminar
which resulted in a formal letter addressing reward for
scholarly teaching and scholarship of teaching in tenure
and promotion.
– Formal Letter on SOTL
– Faculty SOTL Projects
Selected Key Issues and Observations
– Obstacles to Discussion
– Interdisciplinary/Collaborative
Approaches
– “Where's the beef?”
– “Scholarly Teaching” as best first path
– To be a good consumer of the SOTL
An observation of the Rockhurst Carnegie Seminar … which
is entirely consistent with observations at other institutions
… is that a focus on scholarly teaching is a good first step to
scholarship of teaching.
Several faculty research projects are underway at Rockhurst.
One researcher is Professor Anita Salem (mathematics), also
a Carnegie Scholar of Teaching. Anita provided the
information for this tutorial. E-mail to Anita
Abilene Christian University
•
19 Faculty Engaged in SOTL Projects
•
Strong Institutional Support
– Stipends for Materials & Resources
•
Travel to Teaching-Related Conferences
-Ongoing Peer Meetings,Videoconferences
Abilene Christian provides an example of an effective
SOTL initiative at very modest cost in a private
institution with about 4500 students. Faculty
participants make a year-long commitment to get their
projects to the stage of publication. They are given
small stipends for materials and resources. An outside
consultant provides a day long workshop to help shape
the projects. Ongoing support is provided through peer
meetings and videoconferences with the consultant.
Visit the ACU SOTL website
or request a SOTL
program brochure from Nancy Shankle, English
Department Chair. Nancy steers the SOTL initiative at
ACU. E-mail to Nancy
University of Notre Dame
Initial campus conversations with 90 campus leaders
– SOTL needed support
– RFP resulted in 9 funded SOTL projects
Sample research question: Do new teaching methods in introductory engineering
affect students’ learning?
Support for SOTL teams includes these elements:
–
–
–
–
$5,000 per team for student time, equipment, supplies, faculty time
Consulting with methodology experts
Group meetings 2x/semester for mutual support
Help in dissemination of results
The Notre Dame SOTL initiative
features faculty
research projects, campus conversations, and a Carnegie
Scholar. Barbara Walvoord and her team of “experts”
provide support to investigators in regional institutions as
well as at Notre Dame E-mail to Barbara
Notre Dame’s approach – to develop a community of
scholars and define SOTL by example – is similar to the
approach in Bloomington. The impediments are similar too
… like faculty reluctance to engage in a scholarship for
which they feel they have no training.
Middlesex Community College
Interdisciplinary group of 9-10 faculty
Receive 1 course release for minimum 2-year commitment
Meet biweekly for two-hour seminars
Study SOTL - and how to create environs that foster intrinsic motivation
Undertake individual faculty projects
A goal at Middlesex is to link faculty projects to
intrinsic motivation in students. Professor Donna Duffy
(psychology), also a Carnegie Scholar, provided
information about the Middlesex SOTL initiative.
Donna’s own project at Middlesex is ‘Resilience as a
Path to Integration in Abnormal Psychology.” She
chose resilience, a positive attribute, as a theme because
abnormal psychology is generally about negatives …
the problems people face. E-mail to Donna
What’s common to all of the seven campus examples (including Indiana University)?
-they involve institutional investment and support for the SOTL program
-they are connected to AAHE/Carnegie national initiatives.
-they involve faculty investigative projects into issues of teaching and learning
Task E: Campus Examples
of SOTL Initiatives
List features of the examples you have just seen that might be most
applicable to your campus.
Please reflect on the applicability of the campus examples just discussed to your own institution. Construct a
“mock-up” of a SOTL program on your campus, drawing on the features of programs at other campuses.
Press the button on the frame and send your model to us.
Unit 2B Faculty SOTL Projects
Donetta Cothran – Kinesiology
Valerie O’Loughlin – Medical Sciences
David Pace – History
William Becker - Economics
Rita Naremore – Speech and Hearing Sciences
Dennis Jacobs – Chemistry
Leah Savion – Philosophy
Other Examples – Carnegie Scholars
Task F: Minute Paper
After viewing these projects, you’ll be asked to identify the most important points for you in the first four modules of the
tutorial (Task F)
Expectations and Effects of Graded Writing Assignments
For Donetta Cothran, a junior
faculty member, interest in
improving her students’ sense of
fairness in grading of their papers
led naturally into a scholarly project.
Donetta gained insights
about
scholarship of teaching from her
participation in a campus SOTL
initiative and involvement in her
own scholarly project.
E-mail to Donetta
Donetta Cothran - Kinesiology
A Neophyte’s Adventures in the Scholarship of Teaching
Fostering Interactive Learning in a
Large Science Course and
Methodically Measuring the Effects
Valerie’s innate interest in teaching found form and expression
in the campus SOTL initiative. Her project is a “Child of
SOTL”, conceived, born, and nurtured to scholarly product
within the campus initiative.
Valerie Dean
O’Loughlin
Medical Sciences
“Scaffolding” facilitated her acquisition of the concepts and
resources needed to carry out her study. Valerie’s initial
presentation of her work to peers was of interest to many both
for the pedagogical techniques she employed and for her means
of assessing their effects.
Valerie describes the launch of her project
Powerpoint presentation of the complete study
E-mail to Valerie
Details of Valerie O’Loughlin’s study are considered later in the tutorial in Questions, Designs, and
Resources.
What do I want my Students to be Able to Do?
David Pace, a Carnegie Scholar of
Teaching, describes his odyssey to the
question at the heart of his scholarship;
“What are students supposed to be
doing (or learning) in my course?” He
shares his realization that the above
question led to understanding of his
discipline in ways he never understood
it before.
E-mail to David
David Pace–History
What Does the Quantitative Research Literature
Really Show about Teaching Methods?
Bill Becker’s project is an example of a study that is
not centered on one’s own classes. Rather by studying
selected published papers of others, he seeks to
separate empirical results from conjecture about the
student outcomes associated with classroom
assessment techniques and other teaching strategies
designed to engage students actively in the learning
process.
In this project, Bill also seeks to advance the
scholarship of teaching and learning by establishing an
eleven-point set of criteria which faculty members can
use to evaluate inferential studies.
William Becker
Economics
Bill’s study culminated in a paper: What Does the
Quantitative Research Literature Really Show about
Teaching Methods?
E-mail to Bill
A Departmental Level SOTL Project
The problem: a group of faculty members concerned that
teaching excellence was getting short shrift in the faculty
evaluation process
The initial question: What can (and do) student evaluations
really tell us?
Rita Naremore
Speech and Hearing Sciences
The department had a culture that valued excellent teaching,
and in some ways took it for granted. Salary increases were
based on data provided using the campus faculty annual
report form It had become abundantly clear that those
faculty members engaging in innovative or particularly
effective pedagogical change were not being rewarded
because nobody knew what they were doing or how
effective it might be.
Rita introduces the project
E-mail to Rita
Powerpoint presentation of complete study
A Departmental Level SOTL Project
Issue # 2: Peer Evaluation
Once the question regarding student evaluations seemed answered, attention turned to peer
evaluation.
The faculty consensus: traditional classroom visitation is not very revealing
of anything useful. What we really need is evaluation of course structure
and materials used in the class for teaching and evaluation. We’d like to
have periodic external peer review of these materials.
Peer evaluation presented the group with a much larger problem. After much heated discussion, it became
clear that the occasional visit to a classroom by one’s colleagues did not seem an adequate form of peer
evaluation. This faculty agreed that what was missing was a systematic review of course organization and
materials (including exams, visual support for class presentations, etc.) conducted by people outside the
department who knew the content.
A Departmental Level SOTL Project:
Ongoing Concerns with Peer Evaluation
• The biggest concern: IT
TAKES TOO LONG TO PUT
ALL THIS STUFF
TOGETHER!
• What are we doing? We’re
working toward the
development of course
portfolios, to be done over time
and turned in by faculty
members every second or third
year, not every year.
Although agreement in principle was quickly attained, agreement in practice was not. Several faculty members remained
convinced that they did not have time to put together their teaching materials in a way that made them accessible to an
outside reviewer, especially if they had to explain what they were trying to do. Nonetheless, the department is continuing to
work toward a system of course portfolios, along with a checklist that can be used by outside reviewers who are looking at
the portfolios.
A Departmental Level SOTL Project:
The Campus and National Ties
The departmental course portfolio effort should eventually tie in with a campuswide effort now underway, conducted by an interdisciplinary team in
conjunction with a national project funded by Pew, based at the University of
Nebraska, and involving four other research universities.
Review the Relationship of Peer Review of Teaching,
Course Portfolios, and SOTL
Visit the Indiana Campus Course Portfolio Initiative
More Information About the Consortium Project Led by
Nebraska
Simon Brassell
Geological Sciences
Chair of the Campus
Course Portfolio
Initiative
E-mail to Simon
The connection to course portfolio projects exemplifies the way SOTL may be
synergistically connected to and integrated with other teaching-related
initiatives.
An Exemplary Course Portfolio
And
A Superb Model of SOTL
An Alternative Approach to General
Chemistry
Assessing the Needs of ‘At-Risk’ Students with Cooperative
Learning Strategies
Dennis Jacobs
Professor of Chemistry
University of Notre Dame
E-mail to Dennis
The next 7 frames contain details of this project. The project
exemplifies further the tie between SOTL and course portfolios as
well as providing an excellent example of each! Dennis Jacobs is a
Carnegie Scholar of Teaching and Learning.
The project in complete detail (log-in required)
Skip to the next SOTL project.
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students
with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTi me™ and a
GIF decompressor
are needed to see thi s pi ct ure.
Rationale:
• Recognized Problems:
• ‘At-risk’ students (Math SAT ≤630):
• dropped out of General Chemistry.
• didn’t take any advanced science.
• frustrated by large lecture format.
• Alternate Course Design:
• Similar requirements and lectures.
• Comparable exams.
• Various activities involving structured cooperative learning.
• Initial Comments on Proposal
• “Only delaying inevitable failure.”
• “Efforts should be focused on the best not the ‘at-risk’ students.”
Comments that colleagues initially made to Dennis are shown in the
last bullets. Dennis proved them wrong. He demonstrated
convincingly that ‘at risk’ students could perform at the same
standards as other students in general chemistry and go on to
- 4 - in organic chemistry (a sequel course) at the same rate.
succeed
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students
with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTi me™ and a
GIF decompressor
are needed to see thi s pi ct ure.
Documentation:
• ‘At-risk’ students markedly more likely to drop or fail the
course in the traditional class format.
As you view this project,, note Dennis’ methodologies and the
thoroughness of assessment. This is a well designed deed of scholarship.
Ask yourself at the end, whether you feel convinced from this brief slide
- 5 of
- the impact of his alternative course for ‘at risk’ students.
series
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
Implementation:
QuickTi me™ and a
GIF decompressor
are needed to see thi s pi ct ure.
• Aim:
• Provide improved learning opportunities for ‘at-risk’
students.
• Develop more effective teaching in large lecture format.
• Alternate Course Design:
• Introduced opportunities for structured cooperative
learning including:
• discussion of concepts in pairs.
• small group in recitation sections.
• work as pairs in laboratory.
• Mandatory recitation sections:
• more time committed to class.
• direct contact with instructors.
-6-
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTi me™ and a
GIF decompressor
are needed to see thi s pi ct ure.
Traditional vs. Alternative Classes:
• Similarities:
•
•
•
•
Size (250 students), text, chapters.
Lecture time (3 hr), lab time (2.5 hr).
Lecture format (Powerpoint slides and demonstrations).
Exam format and many exam questions.
• Differences in Alternative Section:
• Mandatory recitations (1 hr/wk, 20 students); attendance 95%
vs. 10%.
• Weekly homework (10 vs. 30, graded).
• On-line quizzes (www chapter reviews).
• Weekly feedback from homework, group problems, on-line quiz,
in-class questions.
• Personal contact with instructor and follow-up if performance
declined.
-7-
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTi me™ and a
GIF decompressor
are needed to see thi s pi ct ure.
Impact of Alternative Section:
• Assessment Strategies:
• Effects on conceptual understanding, problem-solving and selfconfidence:
• feedback from students.
• evaluation of individual elements of the cooperative
learning activities.
• Immediate and long-term benefits:
• retention of ‘at-risk’ students.
• success in advanced science classes.
• Data Collection:
• Recording in-class learning activities.
• Tracking individual grades and progress.
• Longitudinal study of ‘at-risk’ students:
• progress in subsequent classes.
-8-
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
Measurement of Impact:
• Success of ‘At-risk’ Students:
QuickTi me™ and a
GIF decompressor
are needed to see thi s pi ct ure.
• Better grades in General Chemistry.
• Improved retention in class.
• Higher success rate in subsequent classes.
-9-
An Alternative Approach to General Chemistry
Assessing the Needs of At-Risk Students with
Cooperative Learning Strategies
Dennis Jacobs (University of Notre Dame)
QuickTi me™ and a
GIF decompressor
are needed to see thi s pi ct ure.
Longitudinal Impact (follow-on course):
Intervention introduced between 1996 and 1997
Organic chemistry is a follow–on course to general chemistry. In 1997, a
higher percentage of ‘at risk’ students who had taken Dennis’
alternative general chemistry course completed organic chemistry than
- 11 the percentage of students in the traditional course.
Investigating Pet Theories and Naïve Misconceptions
Leah Savion’s personal interest in learners’ cognition and interference
effects of naïve misconceptions caught on with faculty colleagues (the
issue crosses most disciplines) and led to a cross-disciplinary study.
Leah explains misconceptions and their effects
A brief overview may be gained from a one page summary of the project
Add student misconceptions from your own experience to Leah’s
Incomplete Inventory of Misconceptions
Further detail may be gained from her working paper
Leah Savion
Philosophy
Leah warmly invites your peer review and comment (E-mail to Leah)
“What force acts on a coin that has been tossed up in the air”? The large majority of students who completed a course in
mechanics in MIT gave the same wrong answer as the totally untrained students, citing the “original upward force of the
hand”. The pet theory of motion at work here is that there’s no motion without force.
Pet theories are involuntary explanatory constructs that we all build from a very early age, in an attempt to understand the
world around us, to build causal connections between events, and to enhance our sense of control over the environment.
These theories are about every aspect of life. They are amazingly universal, are based on surface features of the relevant
data, fragmented, inconsistent, contain principles that emerge spontaneously, complex, intricate, serviceable, and
seemingly well organized. These robust theories are not normally tested against scientific, social or logical facts. A
specialized psychological architecture that contains principles of quick reasoning, rapid accumulation of information, and
economy of cognitive operations, may underlie and guide their construction and their perseverance in face of contradictory
evidence.
Pet theories inevitably imply what I termed here “naïve misconceptions”, that prove extremely resistant to change. When
the naïve meets the academically acceptable theories we deliver in class, the better students either attempt to alter the new
information, or combine the incompatible principles, or, most commonly, adapt the new theory as a “school bound”
Scholarly Projects of Faculty (Continued)
Where you can learn about additional
scholarly projects. Opening Lines contains
8 case studies of approaches to individual
projects by Carnegie Scholars. The seven
bullets below form the outline of the narrative
in each case study. Style and tone are similar
to those in the video vignettes used in this
tutorial.
Opening Lines may be a useful source for identifying
issues of teaching and learning in one’s practice and
framing them. Many faculty will be able to identify with
the thoughts and experiences of the authors of these case
studies. For faculty members who feel that they do their
“thinking about teaching” in isolation within their
departments or institutions, this publication may feel like a
conversation with colleagues.
-Framing the Question
-- Context of the Project
-- Gathering the Evidence
-- Emergent Findings
-- Conditions for doing SOTL
-- Benefits of the Work
-- Lessons Learned
Opening Lines comes with a CD
containing a variety of tools and resources
for scholarly projects. These include
interviews (some as video clips), survey
instruments, protocols, and reference
sources. The publication showcases recent
work in CASTL
Task F: Looking Back Over Travel Along the
Continuum Thus Far
"MINUTE PAPER"
1. What were the 3 most important points presented in Units 1A, 1B, 2A
and 2B of this tutorial?
2. What 3 things would you most like to learn in the remaining two
modules, Units 3A and 3B, of the tutorial?
.
This Classroom Assessment Technique (CAT) denotes the end of Unit 2B. The formative feedback offered by you to us here
will help make this a better tutorial for everyone. Please press the button and send your feedback. If you’ve worked through
the tutorial continuously to this point, you might want to take a break before entering Unit 3 - the last stretch of the
continuum from alpha to omega (and it’s more than a third)!
Unit 3A
Bridges to Productivity
How Could I Do Scholarship of Teaching and Learning (Genres of SOTL)
Task G: Reflecting on Genres
Approaches to Scholarship Via Classroom Research Projects
Where to Publish and Present
Sources of External Funding
Task H: Reflecting on Classroom Research and Key Reference Books
Unit 3 may be thought of as a rudimentary “tool kit” for embarking on scholarly projects in issues of teaching
and learning. This is the longest unit. 3A surveys some general considerations. 3B focuses on project design.
Still Another Look at What We Mean by the
Scholarship of Teaching and Learning
Before venturing into a mélange of categorization schemes, it may be well
to pause for another overview of this scholarship.
Do the following situations represent scholarship of teaching and learning?
1.
A faculty member reforms a course, building into the reform all three criteria of scholarly teaching.
She puts the course on-line not only for students but for all faculty in the discipline, and furthermore
invites faculty to peer review and critique it on the web. The course catches the attention of faculty at
another institution. They are interested in it and invite the course author to come there and give a
colloquium on the course. The author does so and reflects on the attributes of the course for her
students. Subsequently, the department in the second institution adopts large elements of the author’s
course to improve their own undergraduate program.
2.
An assistant professor in quest of tenure develops a course portfolio as evidence of his effectiveness
in improving student learning. The promotion and tenure committee routinely sends candidate
packages to outside reviewers. External reviewers comment on the excellence of this portfolio as
convincing evidence of the candidate’s teaching effectiveness. The candidate is granted tenure. The
course portfolio is subsequently disseminated through the state university system as a model for other
faculty.
Still Another Look at What We Mean by the
Scholarship of Teaching and Learning (Cont.)
If you accept Shulman’s quasi-definition , you will probably conclude that both situations in the previous frame
DO represent scholarship because they satisfy all three of his conditions. As key reformers urged, the concept
of scholarship is broadened to include useful contributions that do not necessarily conform to conventional
scholarly forms in the disciplines. To be sure, scholarship of teaching and learning includes publication in
journals, but it also includes other forms, as in the two previous examples, that contribute to the advancement of
teaching and learning both directly and indirectly.
To survey possibilities for this scholarship, we exhibit several categorization schemes. First, we’ll consider
Craig Nelson’s “Genres of SOTL.” Then we’ll look at “traditional and classroom research” and “qualitative and
quantitative methods.” All of these schemes are messy and inconsistent. The genres, for example, sometimes
seem separated by unit of analysis at other times by type of research design. And as we see from the two
examples in the preceding frame, they are not necessarily inclusive of all possible forms of scholarship.
Similarly, much good scholarship combines elements of traditional and classroom research as well as qualitative
and quantitative methods.
Well, why do we even bother with the classification schemes if they are inconsistent? Because they give us
valuable “points of possibility.” They may be thought of as topographical features that map the territory. The
actual point at which one locates one’s individual deed of scholarship is usually somewhere between the “points
of possibility.”
How Could I do Scholarship of Teaching and Learning?
Craig Nelson introduces the genres of SOTL
Different Genres of SOTL
– Reports on Particular Classes
– Reflections on Years of Teaching
– Larger Contexts—Comparisons
– Formal Research
– Meta-Analyses
Genres of SOTL overlap and can be combined or subdivided variously. The particular examples of scholarship for each genre
are chosen to illustrate importance for improving teaching and learning and largely excerpted from Craig Nelson’s paper on
genres: "How could I do scholarship of teaching and learning?"
Two opening points:
1. Learning and teaching are complex activities where approximate, suggestive knowledge can be very helpful, and, indeed,
may often be the only kind that is practical or possible
2. Much important expertise on teaching resides in the day to day practices of good faculty. Typically, this knowledge
remains private and is totally lost when its possessor retires. A key task in this field is systematically making much more of
this expertise public.
How Could I do Scholarship of Teaching and Learning?
•Different Genres of SOTL
–Reports on Particular Classes
–Reflections on Years of Teaching
–Larger Contexts—Comparisons
–Formal Research
–Meta-Analyses
Reports on Particular Classes
•It Worked! Important pieces of our expert knowledge as experienced practitioners can be preserved by writing
up exemplary approaches to content or pedagogy that work especially well in our own classes. In this genre, the
teacher's own impressions of the effectiveness frequently serve as sufficient assessment. The trend now is to try to
document the effectiveness a bit more formally using CATs and CR.
•Before & After: Assessments of Changes in Practice
-Qualitative The many examples of this genre in Angelo and Cross
include a calculus class (pp. 69-72) in
which the professor wanted to help students improve their problem solving skills. This example illustrates the
process of refining the pedagogical questions and the successive modifications that are often necessary to make
new pedagogical approaches work successfully. In this case, the new pedagogy improved student success
sufficiently that no student made an F, despite the maintenance of high academic standards
-Quantitative Group study approaches to calculus decreased the D/F/W rate for African Americans from
60% to 4%. See R. E. Fullilove & P. U. Treisman. 1990. Mathematics Achievement Among African American
Undergraduates at the University of California, Berkeley: An Evaluation of the Mathematics Workshop
Program. Journal of Negro Education 59: 463-478.
How Could I do Scholarship of Teaching and Learning?
•Different Genres of SOTL
–Reports on Particular Classes
–Reflections on Years of Teaching
–Larger Contexts—Comparisons
–Formal Research
–Meta-Analyses
Reflections on Years of Teaching
•Essays Developing Good Ideas An example providing good articulation of the rationale for SOTL
is L. S.
Shulman. 1993. Teaching as community property: Putting an end to pedagogical solitude. Change 25: 6-7.
•Summaries of Expert Knowledge Gained by Self-Reflection and Experimentation
Examples are P. Frederick. 1981 The Dreaded Discussion: Ten Ways To Start. Improving College & University Teaching
29:109-114. P. J. Frederick. 1986. The Lively Lecture--Eight Variations. College Teaching 34:43-50 Many course
portfolios fit here too. Path breaking examples can be accessed through links at Implementing Entities: Carnegie and
Pew. Another example by Randy Bass is (http://www.georgetown.edu/bassr/portfolio/amlit/ .
•Integration of Larger Frameworks with Classroom and Curriculum Practice
J. D.
Herron. 1975. Piaget for Chemists: Explaining What "Good" Students Cannot Understand. Journal Chemical Education
52:146-150. One factor that explains why bright, hard-working students can do poorly and how we can help them. Easily
applicable in all quantitative fields. R. J. Kloss. 1994. A nudge is best: Helping students through the Perry scheme of
intellectual development. College Teaching 42:151-158. Another factor that explains why bright, hard-working students
can do poorly and how we can help them. Easily applicable across the curriculum.
How Could I do Scholarship of Teaching and Learning?
•Different Genres of SOTL
–Reports on Particular Classes
–Reflections on Years of Teaching
–Larger Contexts—Comparisons
–Formal Research
–Meta-Analyses
Larger Contexts: Comparisons
Across Courses and Student
Change Over Time
•Qualitative Studies to Explore a key Issue
Wm. G. Perry, Jr. [1970] 1998. Forms of Intellectual and
Ethical Development in the College Years, A Scheme. New introduction by Lee Knefelkamp. Jossey-Bass. The impetus
here was the observation that students could flunk out of Harvard despite working quite hard at learning the course
material. The longitudinal design used extensive interviews with students at the end of each of their four undergraduate
years. Patterns of intellectual development were inferred and checked for inter-judge reliability. A very influential study.
•Quantitative comparisons of Different Courses or Sections
M. D. Sundberg & M. L. Dini.
1993. Science majors vs nonmajors: Is there a difference? Journal of College Science Teaching. Mar/Apr 1993:299-304.
Question: Does covering more teach more?. Both courses taught with traditional pedagogy and by multiple instructors,
but with different intensities of 'coverage.' Learning assessed with the ACT exam for AP Biology "The most surprising,
in fact shocking, result of our study was that the majors completing their course did not perform significantly better than
the corresponding cohort of nonmajors."
•Comparisons over an Array of Courses with a Common Assessment Instrument R. R.
Hake. 1998. Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for
introductory physics courses. American Journal of Physics 66: 64-74.
(http://carini.physics.indiana.edu/SDI/welcome.html#z44).
Uses a common test of Newtonian physics to compare
increases in understanding achieved by a wide array of pedagogies in introductory physics courses at institutions ranging
from high-schools to Harvard. Found that "interactive engagement" approximately doubles the amount of physics earned.
An especially important model for emulation in other disciplines.
How Could I do Scholarship of Teaching and Learning?
•Different Genres of SOTL
–Reports on Particular Classes
–Reflections on Years of Teaching
–Larger Contexts—Comparisons
–Formal Research
–Meta-Analyses
Formal Research
What constitutes “formal”? Does “formal” reside in the lower right
quadrant of Rice’s diagram? Not necessarily. We use “formal research” to
mean forms of scholarship in which the principal goal is research. Usually
such scholarship will incorporate one or more features of conventional
research paradigms. Formal research can still be classroom-based and
related to active practice (see the last three examples below).
Experimental Analyses C. M. Steele. 1997. A threat in the air: How stereotypes shape intellectual identity and
performance. American Psychologist 52:613-629.
Classroom Incivilities R. Boice. 1996. Seminal study on incivilities. Identifies and counts perceived incivilities
committed by faculty as well as students and investigates relation between them. Journal of Research in Higher
Education 37: 453-485.
The Impact of One Minute Papers on Learning in an Introductory Accounting Course E. D. Almer, K. Jones, & C.
Moeckel. 1998. Four hypotheses investigated simultaneously in one class using a fractional factorial design. Issues in
Accounting Education 13:485-497.
The One-Minute Paper: Some Empirical Findings J.F. Chizmar & A. L. Ostrosky. 1998. Four teachers each taught
control and experimental sections. Data were analyzed using a multiple regression model. Journal of Economic
Education 29:3-10
This study like the one above on incivilities could also be categorized under “Larger Contexts”
The last two studies of one-minute papers are good examples of formal research at the classroom teacher level. They are
also important contributions to the structure of evidence that one-minute papers improve student learning.
How Could I do Scholarship of Teaching and Learning?
•Different Genres of SOTL
–Reports on Particular Classes
–Reflections on Years of Teaching
–Larger Contexts—Comparisons
–Formal Research
–Meta-Analyses
Meta-Analyses
•Annotated Bibliographies
R. N. Johnson, D. M. Enerson & K. M. Plank. 1996. Diversity: A
Selected and Annotated Bibliography
Center for Excellence in Learning and Teaching.
Pennsylvania State University.
•Brief Annotated Summaries of Key Research Findings T. A. Angelo. 1997. The
campus as learning community: Seven promising shifts and seven powerful levers. AAHE Bulletin 49:3-6.
R. B. Barr & J. Tagg. 1995. From teaching to learning: A new paradigm for undergraduate education.
Change 27:13-25.
•Formal (Quantitative) Meta-Analyses
L. Springer, M.E. Stanne & S.S. Donovan. 1997.
Effects Of Small-Group Learning On Undergraduates In Science, Mathematics, Engineering And
Technology, A Meta-Analysis. National Institute for Science Education, University of Wisconsin. 608/2634214 [average effect size "would move a student from the 50th percentile to the 70th..."]
Task G: Reflecting on Genres
Where does the course portfolio fit in the genres described?
Reflecting on the genres, which seems most appropriate for a scholarly
project of your own? Which seems easiest for you to do?
Are there forms of SOTL for which the genres described do not apply?
If so, how would you name one or more additional genres?
RESPONDING TO THIS TASK IS NOT JUST BUSYWORK! Reflection on the “points of possibility”
described in the genres, may be an important step in conceptualizing your own work in SOTL. You may want to
read one or more of the example papers before attempting this task. Your conception of your own SOTL work
need not coincide with one of the genres, of course!
Approaches to Scholarship Via Classroom Research
•Classroom Research
Distinguishing classroom research from traditional research
A “grand schematic” of classroom research
Classroom Assessment Techniques
Effective Grading
The Course Portfolio
Task H: Reflecting on Classroom Research
Classroom research is described. A schematic diagram is exhibited and used to show strengths of various useful
books. Finally, in Task H you will be asked to identify the step in classroom research that seems easiest and the
reference book that seems most useful.
Classroom Research
“Classroom Research is not traditional research conducted in or on classrooms. It is a
specific methodology designed for discipline oriented teachers without training or
experience in the methods of educational research. Classroom Research is ongoing and
cumulative intellectual inquiry by classroom teachers into the nature of teaching and
learning in their own classrooms. Inquiry into a question about how students learn typically
leads to new questions and thus to continual investigations through classroom research.”
Cross & Steadman
“Finding that almost no relationship existed between research on learning and
collegiate teaching practices (professors were either oblivious to such research
or ignored it), K. Patricia Cross concluded that the research itself was at fault for
failure to pay attention to actual classrooms. She argued that teaching and
learning reforms could only occur if they were based on concrete classroom
situations. Cross made “how to do it” her calling card.” M. Lazerson, U
Wagener, and N. Shumanis. Teaching and Learning in Higher Education. p. 17
Change 32: 3 2000
Many journals that publish articles about teaching and learning, both within
disciplines and across disciplines, invite articles based on classroom research.
The term “action research” is sometimes used synonymously with classroom
research. The former dates from the work of Kurt Lewin in the 1940’s and may
be defined as “Research carried out by practitioners with a view to improving
their professional practice and understanding it better.” Borg, Gall & Gall.
1993 Applying Educational Research 3rd ed. Longman,: 390
Cross, K.P. and Steadman, M. 1996. Classroom Research:
Implementing the Scholarship of Teaching, Jossey-Bass
“Traditional” and “Classroom” Research
Traditional and Classroom research are compared for the sake of characterizing two “points of possibility.” One is not
advocated over the other and it’s an oversimplification to view them as independent. Traditional research is often
performed in classrooms. Classroom research may be enhanced by traditional methods and measures, especially if the
results are for publication in national journals. For example, use of an “off the shelf” assessment instrument with widely
accepted validity may be preferable to a “home grown” test or survey with unknown validity. Using someone other than
the person who determines grades to collect potentially sensitive data from students is also a traditional method that may
enhance a classroom research effort (and make it compliant with the local human subjects policy!)
Traditional
Classroom
Origin
State of present educational literature
Professor’s teaching practice
Purpose
Contribute to existing state of
educational theory
Obtain practical knowledge
applicable in limited circumstances
Requirement
Specialized training / broad grasp of
pertinent literature
Specialized training not essential but
developmental scaffolding may be
Benefit
Field and researcher
Students and professor
A key point is that classroom research is a good point of embarkation
for the typical faculty member using materials at hand and without
specialized training in educational research methods. Classroom
research is usually directed at local teaching issues or questions of
specific interest to the teacher.
“Classroom research is not
about finding universal truth
but rather contextualized
truth.”
Tom Angelo
Improving Teaching and Learning via Classroom
Research
IMPETUS
Goal
Issue
We argue that if the scholarly
project is done well, teaching and
learning will inevitably improve
regardless of the results of the study.
Why so? Because the classroom
research by its very nature will at
least improve attention to students
and information they provide,
thereby improving understanding
and communication. Student
performance is also positively
motivated by evidence that teachers
care enough to study them and the
efficacy of their learning.
INFORMATION
GATHERING
Formative
Assessment
Summative
Assessment
Recorded
Observation
Existing
Scholarship
Institutional
Information Base
APPLICATION
Improved
Teaching and
Learning
Our grand schematic of classroom research!
Schema with similar features are common; We
envision three major components to a research project;
a motivating goal or issue, acquisition of information
bearing on achievement of the goal or resolution of the
issue, and effects identifiable from the juxtaposition of
impetus and information. Sometimes the project
involves an intervention, but not necessarily.
EFFECT
Analysis
SCHOLARLY
PRODUCTIVITY
Reflection
Synthesis
After the third major component in the
above schema (Effect) the possibility of a
scholarly product may emerge (curriculum
materials, article, presentation, etc.) Such
a product may enhance the institution’s
information base as well as the existing
body of scholarship.
Classroom Assessment
Classroom assessment is systematic and formative
• Class is the unit of measurement rather than the individual
• Conditions of learning may be assessed rather than student performance.
• Correct and incorrect are not the emphasis.
• Unexpected rather than expected responses are often most useful.
• Review Unit I material about classroom assessment/research
IMPETUS
Goal
Issue
INFORMATION
GATHERING
Formative
Assessment
Summative
Assessment
Recorded
Observation
Existing
Scholarship
Institutional
Information Base
Defining features of classroom
assessment are shown in the
bullets.They are quite different from
the features of summative assessment
associated with traditional grading.
The book belowhelps faculty clarify
their teaching goals, explains
formative assessment to improve
achievement of goals and provides a
catalog of 50 classroom assessment
techniques (CATs).
EFFECT
Analysis
SCHOLARLY
PRODUCTIVITY
Reflection
Synthesis
APPLICATION
Improved
Teaching and
Learning
Angelo, T. & Cross, K.P. (1993) Classroom Assessment Techniques:
A Handbook for College Teachers (2nd Ed) Jossey-Bass
Effective Grading
Primary Trait Analysis (PTA)… building scales that make performance criteria
explicit in order to:
The book below:
-Categorize/classify student work.
Is a useful resource for classroom
researchers
-Benchmark student learning and document changes.
Aims at documenting how well students
-Improve validity of grading.
are achieving cognitive objectives and
at providing evidence for change in
achievement over time
IMPETUS
Goal
Issue
INFORMATION
GATHERING
Formative
Assessment
Summative
Assessment
Recorded
Observation
Existing
Scholarship
Institutional
Information Base
Treats almost any conventional
objection or concern about grading
EFFECT
Analysis
SCHOLARLY
PRODUCTIVITY
Reflection
Synthesis
APPLICATION
Improved
Teaching and
Learning
Walvoord, B. and Anderson, V. (1998) Effective Grading: A
Tool for Learning and Assessment, Jossey-Bass .
The Course Portfolio
“I was familiar with teaching portfolios … but thinking about teaching as scholarly inquiry began to lead me
in the direction of something I had not seen anyone else doing: a portfolio that focused on the course rather
than on all of one’s teaching. Being a social scientist, I began to think of each course … as a kind of
laboratory - not a truly controlled experiment of course but as a setting in which you start out with goals for
student learning, then you adopt teaching practices that you think will accomplish these and along the way
you can watch and see if your practices are helping to accomplish your goals, collecting evidence about
effects and impact.”
Review Unit I material on course portfolios
W. Cerbin quoted in Hutchings, P. (Ed.)
1998 The Course Portfolio, AAHE
Review a course portfolio example.
As a kind of laboratory notebook for recording data
and observations over time, the course portfolio
keeps the teacher’s experience from being lost with
the teacher’s memory. It’s a possible means of
capturing “wisdom of practice.”
Where to Publish and Present
Venues for publication and presentation of work in SOTL abound. Journals and conferences
may be discipline-based or inclusive of all disciplines, refereed or non-refereed, and local,
regional, national, or international. Publications may be printed or on-line and conferences
may be physical or virtual (teleconference).
Many sources of publication are evident in references given throughout the tutorial.
Additionally, over 100 journals publishing SOTL are accessible through a list of journals
publishing scholarship of teaching and learning.
Opportunities to present in cross-disciplinary venues (calls for proposals and conference
announcements) are provided by the Professional and Organizational Development (POD)
Network
in Higher Education and the American Educational Research Association
(Division J- Postsecondary Education).
Both POD and AERA maintain electronic mail
lists (listservers) which include calls for proposals and conference announcements from all
over the world. More information about these lists are available at the above links.
Potential Sources of External Funding
Sources of grant funding for specific topics and categories in higher education are generated by
search engines operating on SPIN and COS databases. Most institutions subscribe to at least
one of these databases. See your local sponsored research services or institutional research
services for more information. Some sources, but by no means an exhaustive list, are shown
below.
Pew Charitable Trusts
Fund for Improvement of Postsecondary Education
National Science Foundation
Woodrow Wilson National Fellowship Foundation
Alfred P. Sloan foundation
Department of Education
Spencer Foundation
Andrew W. Mellon Foundation
Eli Lilly Foundation
Task H: Reflecting on Classroom Research
Which major step in the classroom research schematic (determining the goal
or issue, gathering essential information, or determining effect) seems the
easiest for you to carry out? Why?
Which of the texts described in this section seems most immediately useful to
you?
Unit 3B
Questions, Designs, and Methods
•Framing Questions
•Task J: Examining Valerie’s Questions
•Task K: Framing Your Own Question
•Designs for Studies
•Choosing Measures
•Guiding Questions in Choosing Methodology
•Task L: Designing Your Own Project
•Summary of Standards
•Task M: Evaluation of the Tutorial
We begin the final module of the tutorial with a disclaimer; the possibilities are too many for adequate
survey here. The topics in this module could easily fill a textbook (and, in fact, do fill many
textbooks!) We limit coverage to a few possibilities.
Framing the Question
Scholarship of teaching and learning does
not necessarily begin with a question. To
record “what worked,” the “wisdom of
experience” or to describe observed
phenomena without attempting
intervention can constitute scholarship.
To integrate the work of others, as in
constructing a continuum of SOTL
experience from Alpha to Omega, or as in
editing a collection of papers into a book
might also constitute scholarship.
Nevertheless, much scholarship is rooted
in inquiry because research is rooted in
inquiry.
Faculty members do not necessarily receive formal
preparation for teaching, much less scholarship of
teaching. Therefore, to expect faculty members to
know how to transform their issues and goals of
teaching and learning into researchable questions
without a little tutoring does not seem quite
reasonable.
As many dissertation advisors and doctoral students
will attest, the most difficult step in the whole
research process is often framing the question to be
addressed. A well framed question is one for which
procedures can be devised that offer the possibility of
arriving at an answer.
The next series of frames are rather narrowly focused
on this particular challenge.
Framing Questions (Goal Approach)
1.
Define a goal
2.
Ask and answer questions to refine the goal (Colleagues are usually
helpful.)
3.
Create a succinct summary of a specified goal
4.
Ask “what evidence would reveal that the goal is achieved?”
5.
Frame possible researchable questions
Teachers commonly think of goals in terms of improved learning outcomes for students. Often, a study is built upon
one or more interventions to achieve a goal and the question becomes “Does the intervention lead to achievement of
the goal?” The goal approach also includes inquiries which Pat Hutchings might characterize as “What works?”
However, a study based on a goal approach need not involve an intervention. A well-framed question could simply
address whether an intended goal is presently being achieved or whether a goal is even feasible to achieve.
Framing Questions (Issue Approach)
Criteria for selection of issues
•
•
•
Investigable (not necessarily empirical)
Bounded and well-defined
Significant (not necessarily statistically)
•
•
•
•
Considerations for investigation of issues
Length of time needed
Complexity of procedures
Availability of subjects
Availability of support (resources, personnel, funds)
Questions can involve investigation of issues rather than achievement of goals; e.g. “How do students who do not
meet prerequisites fare compared to those who do?
” In Opening Lines, Mills Kelly describes how his department chair framed Mills’ question by asking, “How do you
know that using the Web as opposed to depending on paper (the way most of us have taught history) is transforming
student learning – and, if so, whether for good or for ill?” Other types of questions like “What does the current
learning environment for students in my course look like to them?” , “How do they perceive my instruction?” or
“How do students who yield evidence of deep understanding in my course, gain that level of understanding?” can
lead to investigative studies.
Making Vague Questions Answerable – Using
Operational Definitions
Vague questions are made answerable by using operational definitions of terms to improve conciseness.
Operational definitions sharpen the question and afford possibility of clear answers. The price paid is usually
a sacrifice in generality of answers obtained.
Less framed
More framed
Do students who help others learn an
academic discipline learn it better
themselves?
Do students in CMSC 250 who tutor students in CMSC 150 perform
better on the CMSC 250 final exam than students who do not tutor but
have similar grades in CMSC 150?
Do students learn more in small classes?
Do students in sections of Phys 118 enrolling fewer than 50 students
perform better on the departmental final exam than students from
sections enrolling more than 75 students?
Considering small classes to be less than 50 students and “learning” to be characterized by final exam performance as in the
above example are arguable definitions, of course. Nevertheless these operational definitions afford obvious and necessary
clarity to the structure of the study.
What is the optimum number of homework
assignments to give in a beginning math
class?
Do students enrolled in M036 who are given a homework assignment
every week perform differently on the departmental final exam than
students enrolled in M036 who are given homework every class period?
A Neophyte’s Adventures in the Scholarship of Teaching
Task J: Examining Valerie’s Questions
Review the introduction of Valerie’s project
Valerie’s Questions
1.
2.
Valerie Dean
O’Loughlin Medical
Sciences
3.
Will the use of interactive learning activities in A215 lecture improve
exam performance?
Will these activities improve lecture attendance and create a more
active student engagement in lecture?
Will these activities improve overall “merit success rate” in A215?
Valerie’s questions need further specification. What terms , in particular, need
improved operational definition?
After submitting your answers, compare them with the answers in the
following frame.
A Neophyte’s Adventures in the Scholarship of Teaching
Task J (Answers)
Terms particularly needing improved operational definition:
“interactive learning activities”
Primary intervention: use of one or more interactive learning activities per lecture session
Examples of Learning Activities*
•Memory Matrices
•Learning Exercises
•Sample Exam Questions
* Angelo, T and KP Cross (1993). Classroom Assessment Techniques: A
•“Muddiest Point”
nd
Handbook for College Teachers. 2 Ed. San Francisco: Jossey-Bass
“active student engagement”
Valerie defines this term orally (beginning at 1 minute: 40 seconds into the video clip)
“merit success rate”
merit success rate = total A,B,C’s
initial enrollment
Framing Questions as Hypotheses
H1: Students who write one-minute papers
will perform better on a subsequent
a. essay b. multiple choice
quiz than students who do not write oneminute papers.
H3: Students whose one-minute papers are
graded will perform better on a subsequent
quiz than students whose one-minute
papers are not graded.
H1 actually contains two hypotheses. Why?
H2: Students who address their oneminute papers to a novice audience will
perform better on a subsequent quiz than
students who address their papers to the
instructor.
H4: The difference in quiz performance
between students who do and do not write
one minute papers will be less for higher
ability students than for lower ability
students.
In some studies, questions are most usefully framed as hypotheses to be tested. All of these hypotheses were
successfully tested in the referenced study.
Almer, E.D., Jones, K., & Moeckel, C.L. (1998) “The Impact of OneMinute Papers on Learning in an Introductory Accounting Course.”
Issues in Accounting Education (13) 3
Task K: Framing Your Question
1.
Write a tentative question to be addressed in one of the courses you teach.
2.
Discuss your question with one or two colleagues for the purpose of framing it in the
clearest and most answerable way. Encourage your colleagues to challenge your
framing.
3.
Write your well-framed question at the end of this process.
THIS TASK MAY TAKE YOU AWHILE. You might want to stop here, think about your question, take it down the hall for
discussion with colleagues, and bring back the results after lunch. You might also want to compare and contrast it to the
questions in Faculty SOTL Projects before writing it in well-framed form and pressing the button to send it.
What is a Design for a Study?
• A plan or protocol for carrying out the study
• An underlying scheme that governs functioning, developing, or unfolding
A good design always promotes efficient
and successful gathering and analysis of
the needed information.
Tthe design makes clear what information is being
gathered (Example: student perception by means
of fixed interview procedure in focus groups) or is
being measured (Example: student learning by
means of 3 exams). The design also makes clear
what, if anything, is being manipulated.
(Example: the study is designed so that some
students attend review sessions before each exam
while others do not.)
The nature and frequency of measurement and
manipulation are made clear along with factors
that the investigator is attempting to take into
account because they may explain the result even
though they are not the focus of the study.
(Examples of such factors are students’ class
attendance or backgrounds.)
The design should include a timetable for significant
milestones, specification of measures to be used in
acquiring information, and provision for obtaining needed
resources (Example of milestone: institutional approval to
use your students as subjects. Two examples of needed
resources: grade distributions of students in the target
course over the last several years; a colleague to conduct
interviews of your students.)
It may be necessary to modify the design as work
progresses. In general, the greater the degree of
qualitative information gathering, the more likely is
modification in the design as the study progresses. It’s
important that any design modification be the result of
careful consideration rather than something that just
happens in the course of events without the implications
for the goals of the project having been thought through.
A Qualitative or a Quantitative Study?
Danger: This may not be the best question to ask!
Why may the above question not be the best one to ask? Because it often tends to polarize faculty
along their disciplinary lines and this may not be in the best interest of excellent scholarship of
teaching and learning!
No factor is more influential in shaping our teaching and
research perspectives than our individual disciplines. For
most of us, almost our every word and thought about
either teaching or research is grounded in our experience in
the discipline. The research methods traditional in a
discipline tend to influence the approaches faculty in that
discipline take toward SOTL (and what journals in that
discipline will accept for publication as SOTL).
An interesting experiment is to pose an issue for
investigation to a cross-disciplinary group of faculty
members. If encouraged to talk about approaches to the
investigation, their approaches might be very different,
reflecting their different disciplinary homes. Often each
approach has potential to shed light on the issue. The
scholarship that is most effective in advancing teaching
and learning may be that which combines a variety of
approaches.
But there is a danger. Rather than achieving synergy
between different approaches, we sometimes tend to
discredit or reject approaches which are not familiar to us.
I felt I was moving between two groups … who had
almost ceased to communicate at all, who in
intellectual, moral and psychological climate had so
little in common … literary intellectuals at one pole – at
the other scientists …Between the two a gulf of
mutual incomprehension – sometimes hostility and
dislike, but most of all lack of understanding. They
have a curious distorted image of each other. Their
attitudes are so different that even on the level of
emotion, they can’t find much common ground.
C. P. Snow*
*Snow, C. P. (1959) The Two Cultures and the Scientific
Revolution. Cambridge University Press pp.2-4
Quantitative and Qualitative Methods as
“Points of Possibility”
Feature
Quantitative
Qualitative
nature
empirical, statistical, comparative
naturalistic, fieldwork, constructivist
goal
hypothesis testing, confirmatory
descriptive, generative, finding meaning
design
predetermined, fixed
flexible, evolving
sample
large, representative
small, purposeful
measures
scores, percentages, counts, rates
interviews, observations, writings
investigator
outsider, non-perturbing
insider, perturbing
analysis method
deductive
inductive
findings
summative, precise, reliable
formative, rich, expansive
As in previous comparisons, we do not “go to the wall” in defense of the absolute correctness or comprehensiveness of
the above table entries. Nevertheless, they are offered as generally useful characterizations.
Typical Measures Associated with
Quantitative and Qualitative Methods
Quantitative
•course exam, project, paper scores
•survey scores (Likert)
•frequencies of multiple choice test item
responses
•scores on standardized scales and tests
•counts (participation, web requests, office
visits)
•measures of time use
•institutional research data (GPAs, grades,
admissions scores, frequency distributions
of demographics)
Qualitative
•performances (possibly recorded on
tape)
•interviews (possibly recorded on tape)
•focus groups
•student projects, term papers, essay
items or exams
•reflective statements
•journals
•reports of others (counselors, etc.)
In general, the more ways of gathering information and
the more frequent the events in which information is
gathered, the greater the validity of the study.
A Neophyte’s Adventures in the Scholarship of Teaching
Examples of Measures in a Particular SOTL Project
Review the introduction to Valerie O’Loughlin’s project
Review Valerie’s research questions
Valerie discusses content of the box below
Assessment Measures
•
•
•
“Affective” measures
– muddiest point, mid-course survey
“Process” measures
– lecture attendance, Web hits/requests on learning activities
“Performance” measures (for both the Class and the Instructor)
– Fall 2000 compared to Fall 99, 98, 97 semesters
In the next 6 frames of the tutorial, some specific measures in Valerie’s study and the
information gained from them is surveyed.
Skip over the examples of measures from Valerie’s study.
A Neophyte’s Adventures in the Scholarship of Teaching
Examples of Measures in a Particular SOTL
Project: Mid-Semester Evaluation
The mid-semester evaluation was a questionnaire designed by Valerie that students completed
individually or in groups during class time and submitted anonymously. Key findings were:
–
–
–
–
–
Lecture notes on Web very helpful
More learning activities requested
Extra lecture review sessions requested
Review sheets requested
Wanted more exams that cover less material
The second finding confirmed that students liked the learning activities (which were the primary
intervention in the study) and wanted more of them. Valerie granted the requests in 2, 3, and 4,
during the semester of her study and is considering 5 for future semesters.
A Neophyte’s Adventures in the Scholarship of Teaching
Examples of Measures in a Particular SOTL
Project: Count of Web Hits on Learning Activities
Are students really accessing the learning activities?
Lecture Requests:
-Integument: 315
-Myology: 312
-Muscles of Upper Limb: 311
-Nervous System: 315
Learning Activity Requests:
-Epithelium: 237
-Myology 1: 159
-Muscles of Upper Limb: 155
-Neurons: 136
Web hits were counted as a measure of student use of learning activities outside of class. The left and right
columns above respectively show number of requests for lecture notes and the associated learning activity
during a particular month (September). The data below show that overall web hits associated with the target
course increased sharply in the semester in which the learning activities were implemented.
Average Number of web requests per
student:
Fall 1998: 116
Fall 2000: 176
16000
14000
12000
10000
# of page
8000
requests
6000
4000
2000
0
Sept.
Oct.
2000
1999
Nov.
1998
Dec.
A Neophyte’s Adventures in the Scholarship of Teaching
Examples of Measures in a Particular SOTL
Project: Pre-course data comparisons
SAT-V
SAT-M
GPA
Credit Hours
1997
524
534
2.98
47
1998
533
545
3.04
48
1999
518
534
2.95
46
2000
516
536
2.95
49
Grand
Avg.
524
537
2.99
47
The purpose of background measures is to investigate whether students across semesters are
“fairly similar.” This controls for the possibility that any improvement shown on outcome
measures in the study is because the class using the intervention consisted of higher achievers
than the comparison group. In this case, the higher achievers appeared to be in the 1998 class
rather than the class involved in the intervention (2000).
A Neophyte’s Adventures in the Scholarship of Teaching
Examples of Measures in a Particular SOTL
Project: Common Exam Item
•
Motor neurons are examples of
______ neurons.
a.
Bipolar
b.
Afferent
c.
Association
d.
Pseudounipolar
e.
Multipolar*
80
70
60
50
%
40
correct
30
20
10
0
2000 1999 1998 1997
A common exam item can serve as a measure of learning. Except for minor changes in distractors, this item
was the same over four years of tests in the target course. In the SOTL project (2000), a learning activity
about types of neurons was done in class.
A Neophyte’s Adventures in the Scholarship of Teaching
Examples of Measures in a Particular SOTL
Project: Mean Exam Performance
Exam 1
Exam 2
Exam 3
Exam 4
2000
80.3
75.3
79.6
81.2
1999
74.4
72.5
76.5
76.9
1998
71.6
71.8
78
82.4
1997
74.7
69.8
75
78.3
Exam scores are frequently used as measures of learning. For a comparison of exam scores across
years to have any meaning, the comparability of the exams across the years must be demonstrable.
The comparability of exams was demonstrable in this study and students scored higher on 3 of 4
exams during the semester (Fall 2000) in which the learning activities were employed.
A Neophyte’s Adventures in the Scholarship of Teaching
Examples of Measures in a Particular SOTL
Project: Instructor Evaluations
Fall 2000 notables:
-highest overall mean evaluation
-highest score for question “my instructor clears
up points of confusion for me”
-highest score for question “my instructor
facilitates discussion among students”
4.8
4.7
4.6
4.5
4.4
4.3
4.2
4.1
4
3.9
3.8
3.7
1997
•
•
•
•
1998
1999
2000
“The format of lectures, notes and learning exercises appealed to my style of learning”
“She made learning easier by using different teaching techniques.”
“She gave more personal attention in a class of 250+ than many do with much smaller classes. She consistently made us
feel that she wanted us to succeed and that she would go the extra distance to make that happen.”
“Dr. O’Loughlin is an excellent teacher. I speak as a humanities student who might turn to science if every science
instructor were like her.”
The instructor evaluation is commonly used as both a quantitative and a qualitative measure. An endof-course evaluation alone may not make a very compelling body of evidence in a study. However,
when the evaluation is used in conjunction with a variety of quantitative and qualitative measures as
in this SOTL project, a clear and compelling work of scholarship may result.
Guiding Questions in Choosing Methodology
What approach fits your research problem?
• Qualitative case study
• Quantitative study enhanced by qualitative data
• Qualitative study enhanced by quantitative data
Do you have the skills/resources to carry out the methods?
The skills/resources needed to employ qualitative and quantitative methods appropriately tend to be very different.
Pooling of skills and resources is a good reason for community and collaboration in a campus SOTL initiative.
Will your audience find these approaches acceptable?
Editors or audiences within a discipline may be disinclined to accept work that does not include methods familiar to
that discipline.
A good primer: User Friendly Handbook for Mixed-Method Evaluations (NSF)
Why such a publication from the premier organization representing the sciences? “ Because of the recognition that by
focusing primarily on quantitative techniques, evaluators may miss important parts of a story.” Similarly by focusing
exclusively on qualitative methods, researchers may miss opportunities to back their findings with the kind of
objective evidence that makes the findings credible to a broader audience.
Guiding questions provided by Samuel Guskin, Professor Emeritus, School of Education, Indiana University
Task L: Designing Your SOTL Project
1.
Individually design a study to address the question you framed in Task K.
2.
Discuss your tentative design with with colleagues (perhaps the same ones who
helped refine your question.) Encourage your colleagues to question you and
comment.
3.
Sketch any revisions to your design at the end of this process.
Like Task K, This task may take awhile to complete. Your design may involve sketches, or other graphics
and tables or charts. Completing this task is an important step in processing the content of this unit. It is also
a critical step if you plan to undertake a SOTL project!
Summary of Standards
The six general standards for scholarship below seem an excellent wrap-up of our work in the tutorial.
George Walker comments on standards for SOTL
Clear Goals
Does the scholar state the basic purpose of his
or her work clearly? Does the scholar define
objectives that are realistic and achievable?
Does the scholar identify important questions in
the field?
Significant Results
Does the scholar achieve the goals? Does the
scholar’s work add consequentially to the field?
Does the scholar’s work open additional areas for
further exploration?
Adequate Preparation
Does the scholar show an understanding of
existing scholarship in the field? Does the
scholar bring the necessary skills to his or her
work? Does the scholar bring together the
resources necessary to move the project
forward?
Effective Presentation
Does the scholar use a suitable style and effective
organization to present his or her work? Does the
scholar use appropriate forums for communicating
work to its intended audiences? Does the scholar
present his or her message with clarity and integrity?
Appropriate Methods
Does the scholar use methods appropriate to the
goals? Does the scholar apply effectively the
methods selected? Does the scholar modify
procedures in response to changing
circumstances?
Reflective Critique
Does the scholar critically evaluate his or her own
work? Does the scholar bring an appropriate breadth
of evidence to his or her critique? Does the scholar
use evaluation to improve the quality of future work?
Glassick, C.E., Huber, M.T., Maeroff, G.I,. 1997 Scholarship Assessed:
Evaluation of the Professoriate. Jossey-Bass.
Task M: Closing Evaluation of Tutorial
1.
My grasp of SOTL from Alpha to Omega is improved.
2.
The tutorial had fresh significant perspectives of value.
3.
I gained valuable resources for future use.
4.
The tutorial was worth the time I spent working through it.
SD
D
U
A
SA
Please complete the above evaluation and press the button to e-mail responses. For example, if you strongly
disagree with item 1, please reply 1SD, etc. Your comments are also most earnestly solicited. Thanks very
much.