Transcript 955 Design based research

In theory, there is no difference
between theory and practice, in
practice there is — Jan L.A. Van de
Snepscheut
Design based research, understand
the world by trying to change it —
Chris Hoadly
In theory, there is no difference
between theory and practice, in
practice there is — Jan L.A. Van de
Snepscheut
Design based research, understand
the world by trying to change it —
Chris Hoadly
In theory, there is no difference
between theory and practice, in
practice there is — Jan L.A. Van de
Snepscheut
Design based research, understand
the world by trying to change it —
Chris Hoadly
Design Based Research
November 2007
Defining research
• “. . . the systematic study of materials and
sources in order to establish facts and reach
new conclusions . . .”
• adjectives describing research as
•
•
•
•
•
•
disciplined
transparent
public
scientific
diligent
and
accessible
•
•
•
•
•
organized
problem orientated
creative
systematic
laborious
Research Paradigms
• Quantitative - discovery of the laws that
govern behavior
• Qualitative - understandings from the inside
• Critical - Investigate and expose the power
relationships
• Design-based - interventions, interactions
and their effect
Design-Based
Research
• Related to engineering and architectural research
• Focuses on the design, construction,
implementation, and adoption of a learning
initiative in a real context
• Related to ‘Development Research’
• Closest educators have to a “homegrown” research
methodology
Design based research
• “Engineering” particular forms or learning
and systematically studying those forms of
learning within the context defined by the
means of supporting them.
• Subject to test and revision
• Successive iterations play a role similar to
systematic variation in experiment
• To develop theories not just to determine
“what works”
• To develop a greater understanding of “learning
ecology”
Design Studies are
•
•
•
•
•
•
•
Iterative
Process focused
Interventionist
Collaborative
Multi-leveled
Utility oriented
Theory driven and generative
Design based research
• Theory vs. Practice
Principles of Design Based Research
• Designing learning environments intertwined with developing
theories (proto-theories)
• Cycles of design, enactment, analysis & redesign. Respond
to emergent features
• Must lead to sharable theories (plausible causal accounts)
• Must account for how designs function in authentic settings
(document success, failure and interactions)
• Relies on methods that can document and connect
processes of enactments to outcomes of interest
DBR…
• Interventionist - in that a purposeful change
is made in a functioning educational context.
• Participatory - in that researchers work with
practitioners.
• Theoretical - in that theory is used to design
the intervention.
• Generative - in that it generates theory
through studied application in the original
and subsequent educational contexts in
which the innovation is used.
DBR
• The 5 fold way
•
•
•
•
Develop theories - about process and means
Interventionist: test beds for innovation
Place theories in harm’s way: Prospective & reflective
Iterative - to develop explanatory framework that
specifies expectations, become focus of next round
• Theories are humble: concerned with domain-specific
learning & accountable to the activity of design
• Design experiments are extended (iterative),
interventionist (innovative and design-based), and
theory-oriented enterprises whose “theories” do real
work in practical educational contexts.
DBR: A CASE STUDY
Koehler & Mishra (or should I say Mishra & Koehler)
TPCK
• Over the years, we have developed a
framework for thinking about what
teachers need to know about
technology
• Over this period, we have articulated a
theoretical framework called
technological pedagogical content
knowledge (TPCK).
• Before going into what that framework
is, let talk about how it began
Ancient History (1999-2000)
• Traditionally, if you want to teachers to
use technology …
• Teach them technology skills (assuming
this is the barrier to their use of
technology)
• Hold a few workshops or training
seminars, and teach them to make webpages, use microsoft word, maybe iMovie,
etc.
• Assume that armed w/ this knowledge,
they will put it to good educational use in
the classroom.
Ancient History (1999-2000)
• In 1999 or 2000 Matt met Punya (and
vice versa), and had a few
conversations about this idea
• We agreed that this approach doesn’t
really work
• And roughly, had an idea why (ie.,
learning skills out of context)
• And we both favored a more hands on
approach / learning by doing approach /
learning in context approach
Ancient History (1999-2000)
• So we began a few “design experiments”
(Brown, 1992; Cobb, Confrey, diSessa,
Lehrer, & Schauble, 2003; Design Based
Research Collective, 2003).
• Our teaching of ed tech masters students
(and college faculty) became oriented
around an approach that we called learning
technology by design
• People would learn about educational technology
by designing educational technology
• We have been developing theory; developing
teachers; and developing a curricular
approach (learning technology by design).
Learning Technology by
Ancient History (1999-2000)
• Learning Technololgy design projects included:
• Designing online courses that would be taught in the
college of education (ed tech students and college
faculty working in groups)
• Designing a website or powerpoint to teach ______
• We thought the advantages were:
• Collaboration allowed for sharing expertise
• Learning about technology was always contextualized
• Rich opportunities to learn about multiple technologies
in one single design project
• Engaging learning environment
History (2002)
• We collected data to see what people
were learning:
• Surveys
• The artifacts that teams were designing
• Transcripts of what people talked about
• Copies of writings and emails
• Interviews
History (2002)
• We were sure that it was working
• But… why? It surely had something to
do about the learning technology in
context, but what does that mean?
What is the educational technology
context anyway?
• We were struck by an very good
description of teacher knowledge by
Shulman (1986).
History (2002)
• Shulman argued that to date, there are roughly two
things that scholars had identified that teachers need
to know:
• Content knowledge : Expertise about the topic they were
teaching
• Pedagogical knowledge: Knowledge about teaching
techniques, assessment, etc.
• But what was missing was:
• Pedagogical Content Knowledge: Knowledge about how to
teach a particular subject matter, knowing about common
student trajectories (and misconceptions), and ways to
assess that knowledge for a particular subject matter
• This idea by Shulman is widely accepted (and cited)
now by everyone
History (2002)
• We thought that when you threw technology into the
mix, that traditional workshop approaches that teach
technology in isolation were doing:
History (2002)
• Instead, we thought learning technology in
context meant learning this:
•C : Content
•P: Pedagogy
•T: Technology
•R: Representations
Why?
Diagrammatic
representations
• The tetrahedral model…
Diagrammatic
representations
• The tetrahedral model…
History (2002)
• Ultimately, R (Representations) went away from our
way of thinking:
• Representations are usually one of the three categories
• New representations afforded by technology
• Representations of content or pedagogy (e.g., a formula to
represent math or physics ideas)
• Shulman often wrote about representations for a subject
matter to be part of pedagogical content knowledge
• It was really hard to code for
• It wasn’t doing much for it
• Which leads us to 2003 and the present
Theoretical Framework
• The current framework has three,
interconnected components
• CONTENT
• PEDAGOGY
• TECHNOLOGY
An intermediate step
Still with lines… but with no way to represent
TPCK
Theoretical Framework
• These three components, each
represent knowledge bases that ideally
overlap.
Theoretical Framework
• In our framework, there are multiple things
that teachers need to know
• Content (C) – The subject matter of what they are
teaching
• Pedagogy (P) – The methods of teaching
• Technology (T) – Tools, software, and hardware
(commonplace technologies like blackboards, as
well as digital computers)
Theoretical Framework
• But not just the three in isolation, but their
intersection as well
• Pedagogical Content Knowledge (PC) – How to teach a
particular subject matter, common student understandings
and misconceptions, etc. (Shulman, 1986, 1987)
• Technological Content Knowledge (TC) – how a subject
matter is transformed by the application of technology
• Technological Pedagogical Knowledge (TP) – how
technology can support pedagogical goals (e.g., fostering
collaboration).
Theoretical Framework
• But not just the three in isolation, but their
intersection as well
• Technological Pedagogical Content Knowledge (TPC) –
understanding and negotiating the relationships between
these three components of knowledge (Bruce & Levin, 1997;
Dewey & Bentley, 1949; Rosenblatt, 1978)
• Similar to ideas presented by Hughes, in press; Keating &
Evans, 2001; Lundeberg, Bergland, Klyczek, & Hoffman,
2003; Margerum-Leys, & Marx, 2002
What is such a framework
good for?
• Building theory
• Describing phenomena
• Telling you where to look for
evidence
• Informing practice
• Others?
2003 - Present
• Recall that our framework evolved
over a period of 4 years,
• The result of design experiments
where we constantly designed and
redesigned a learning environment
• Closely connected to our philosophy
of how to teach this stuff: Learning
technology by design
Funded by
• College funds to conduct research on
Learning Technology by Design
seminars
• Preparing Tomorrow’s Teachers to use
Technology (PT3 grant 1 and 2)
• 1999 - 2007
Learning Technology by
Design
• Our design seminars have focused on teams
working together to build online or hybrid
courses as a means of developing their
knowledge of educational technology.
• Design teams typically consist of one tenure
stream faculty, and several teachers seeking
their masters degree in educational
technology.
• The teams work over the course of a
semester to develop one online course.
Learning Technology by
Design
• The “official” site for learning is a course that
we teach.
• We do very little formal lecturing instead we
do more technology “demos”
• There is some reading, dealing with current
issues in design or issues of online
pedagogy.
• Groups tend to go in different directions,
choosing different technologies and
pedagogies.
• Accordingly, our role as instructors is to
guide and facilitate a potentially vast field of
possible scenarios (technical and
Learning Technology by
Design
• Designing an online course requires inquiry
research to design tangible, meaningful
products (e.g., syllabus, websites, online
communities, lessons, etc.).
• Teachers use technology to build something
that is sensitive to the subject matter, and
specific instructional goals.
• As such, every act is a weaving together of
content, pedagogy, and technology
Learning Technology by
Design
• So in short, our claim has been that unlike some
approaches to teacher training, that develop
technology skills in isolation. Learning technology by
design develops skills in an integrated fashion.
Testing our Theory
†
• In Koehler et al, 2004 we presented a case study of a college
faculty member (Dr. Shaker) as she worked with her design
team to create an online course.
• Our analysis revealed important changes in Dr. Shaker’s
technological literacy and her thinking about her personal
relationship with technology.
• In accounting for these changes, we found that the learning by
design approach afforded rich opportunities for Dr. Shaker (and
her other team members) to deeply consider the relationships
between content, pedagogy, and technology.
†
Koehler, M.J., Mishra, P., Hershey, K., & Peruski, L. (2004). With a little help from your students: A new
model for faculty development and online course design. Journal of Technology and Teacher Education, 12(1),
25-55.
An early case study
Koehler, Mishra, Hershey & Peruski (2004)
Testing our Theory II
†
• In Koehler et al, 2004 we followed a different design
group in detail, tracking their conversations over a
course of a semester.
• A mixture of quantitative and qualitative analyses
showed that:
• Initially participants talked about content, pedagogy, and
technology as independent constructs
• Over time, their conversations revealed more and more
connections between these bodies of knowledge
• This suggests that the learning by design approach
develops knowledge consistent with the integrated
TPCK theory.
†
Koehler, M.J., Mishra, P., Yahya, K., & Yadav, A. (2004). Successful teaching with technology: The complex interplay of
content, pedagogy, and technology. Proceedings from the Annual Meeting of the Society for Information Technology & Teacher
Education, Atlanta, GA. Charlottesville, VA: Association for the Advancement of Computing in Education
Studying Design Talk
Koehler, Mishra, & Yahya (2007)
The survey study
• Research on course taught by Koehler
during 2003
• Design teams: 1 college faculty + teachers
(ed-tech masters students) worked to design
online courses
• 4 college of education faculty, 14 students
• Had to design everything: syllabus, readings,
student assignment, assessment rubrics, the
nature of student interactions, etc.
• Also had to figure out how technology would
be used to accomplish all of the course goals
Measuring Outcomes by a
Survey
• We designed a survey that measured
participants attitudes and opinions
about:
• The learning environment (e.g., the
learning technology by design approach)
• How they thought about designing for a
face to face course vs. designing for an
online audience
• How participants were developing
knowledge as suggested by the TPCK
framework (Technology, Pedagogy,
Content).
A Survey of TPCK Thinking
Koehler & Mishra (2005)
“Designing an online course requires
changes in what we teach and how we
teach”
“Our group has chosen technologies that
fit the course content and the instructors
teaching philosophy.”
Conclusions
• Both at the individual, and group level
participants reported significant
changes in how much they thought,
talked about, and learned about
Content, Pedagogy, and Technology.
• Not completely unexpected given the
attention to each of these topics
• However, it does agree with our
earlier, more in-depth studies of single
design groups
Conclusions
• Our theoretical framework, however,
stresses the importance of developing
knowledge at the intersection of the three
components.
• It has been our hypothesis that the learning
technology by design approach was ideally
suited to developing this type of knowledge.
• The results of this study confirm previous
results that suggest the approach does
develop these integrated forms of knowledge
as well.
Book based on faculty dev
courses
• Faculty Development by Design: Integrating
Technology in Higher Education edited by
Punya Mishra, Matthew, J Koehler, Yong
Zhao
Update…
• AACTE Book, goal to “situate Mishra & Koehler’s
concept of TPCK in the realm of teacher education”
Back to DBR
• Long term project
• Can be built around smaller more
manageable studies
• Don’t do this for your dissertation
• Or better still think of your dissertation
in these terms (just don’t tell your
committee right away)