Transcript The Higher Education Academy

The Academy Research Observatory:
development and community engagement
Martin Oliver
London Knowledge Lab
& Higher Education Academy
Overview
Context: what’s motivated this?
Evidence, practice, policy and e-learning
Projects at the IoE
The Academy’s Research Observatory
The landscaping study
Early pilots
Current developments
Future plans
Discussions
Your uses of evidence; the Observatory; etc
Some background
Familiar account of the rise of New Managerialism
Monitoring, accountability and performance indicators
Private, professional judgements (e.g. around
pedagogy) made into strategic, manageable issues
“What gets measured gets done”
Rational and pragmatic
Evidence-based practice (a caricature)
Draws from a positivist tradition of enquiry
Evolution in medicine
The hierarchy of evidence – reliability as key
Summarise and aggregate studies appropriately (and
often, mathematically)
However…
Medical model doesn’t necessarily work in e-learning
Qualitative research acceptable or favoured
Education as a theory-building field (whilst evidence-based
practice movement eschews theory for pragmatics)
Practical (and political) issue of de-centred field (no Cochrane
database… what fields should we cover?)
Evidence of benefits from involving practitioners in research,
rather than differentiating researchers from practitioners (the
studied)
What alternatives might there be…?
And more generally…
Clegg’s critique of systematic review
Appeal to evidence as part of wider erosion of professional
judgement
‘Consumer’ represented in the data, but only as constituted and
spoken for by the researchers
Reviews undertaken to shore up policy, not inform it
‘Black box’ of favoured methods fails to explain what works –
which is important given complexity of teaching and learning
Consequent gap between reviews and practical application
A theorisation
Wenger’s communities of practice model
Practice and Reification as necessary but
fundamentally different (experience and abstraction)
Reifications received by communities and meaning
negotiated in relation to practice
A social account of acceptable (policed) interpretation
This has implications for how we make sense of
research
An aside about legitimation
Lyotard’s account of knowledge in post-modernity
Separation of ‘knowledge’ from the knower
Knowledge as a commodity, valorised through use
(consumption)
Value to researchers? To practitioners? To policy makers?
Scepticism about grand narratives
Language games as the means of legitimising claims
(…somehow, this should all connect…)
So…
More general interest in ‘evidence informed’ practice
and policy
Problems perceived to exist at various stages:
• Finding research
• Making sense of it
• Acting on it
Not strongly visible in UK policies
• cf. No Child Left Behind: “scientifically-based research”
…but evident in policy discourse
“We need social scientists to help determine what
worked and why, and what types of policy initiative
are likely to be most effective”
Blunkett, 2000
Harnessing Technology
A practice-based research environment
64. Our best understanding of how to improve practice
comes through learning from others – from
experimentation, collaboration, and dissemination.
We need an active, practice-oriented R&D forum to
bring together teachers, lecturers researchers, and
industry to develop the leading edge applications. It
would combine the practitioner knowledge of
teachers and lecturers with the specialist knowledge
of learning technologists and industry suppliers. It
should help to build the evidence-base for the value
and impact of e-learning.
An interesting start
• A social account of knowledge-building
• Explicitly linked to practice
Although…
• Political imperative: unifying practice, support and the market
• Evidence “for” (not to change?) value and impact
Research into the ‘teaching-research nexus’
This research indicates that staff who see their
research as tentative and as part of a wider debate in
the discipline, and see their teaching as supporting
student conceptual change, are more likely to bring
their teaching and research together.
By contrast staff who see their research as atomistic
investigations and their teaching as concentrated on
teacher-focused transmission of information are less
likely to experience strong connections between
teaching and research.
At departmental level teaching and research are now
often organised separately, and in many cases
limited thought is given to, and few explicit policies
determine, how they might be linked. Indeed in some
cases it may well be that the pressures for research
selectivity, such as those of the RAE, are causing
increased fissures between teaching and research.
The ‘nexus’ isn’t one thing
Differentiation between:
• Research tutored
•
•
•
• Research resources central to curiculum
Research based
• Inquiry-based learning
Research led
• Oriented to subject content
Research oriented
• Focus on process of knowledge construction
Meanwhile, back at the ranch…
IoE participation in Academy-funded e-Learning
Benchmarking work (Mellar et al)
• A structured programme of self-assessment
• But…
“Whilst the changes to strategy are important, it is clear that
these in themselves will not be able to bring about the
changes needed. The rationale for participation thus became
more one of what we could learn generally from other
participants’ experiences: about the meaning of e-learning
for them, and the way in which they organised and managed
e-learning in their institutions.”
(Benchmarking blog entry)
PREEL
From Pedagogic Research to Embedded e-learning
Building on the Benchmarking project
Funded through the Academy Pathfinder funding
Within our institution (and we suspect in many others) there
are pockets of excellent practice in e-learning, and several
strong research communities (as well as specific work
undertaken by individuals) but there is a lack of coordination,
and a recognition that this wealth of experience is not
coming together to provide a unified e-learning experience
for all our students. This project sets out to bridge this gap
between e-learning research and e-learning practice within
the IoE, and to demonstrate how this was done in such a
way as to be of value to other HE institutions.
PREEL
Led from the LTU
Projects to redevelop curricula, leading to
publications
A network built around this work
Workshops to share existing research
Report summarising research, tools, resources
A real person to talk to! (A key element)
A focus on involving people in research
PREEL2
Building on the work in Quality Assurance at a
national level
e-Learning QA/QE Special Interest Group
A community-led initiative, involving QAA
London Pedagogy Planner
Laurillard et al – building on existing research into
curricula and technology
Development of a “prototype for a collaborative online
planning and design tool that supports lecturers in
developing, analysing and sharing learning designs”
A formal, computable representation of courserelated decisions
Developing strategies for translating elearning research into practice
Building on PREEL, LPP, etc
Mellar, Laurillard & Hadjithoma-Garstka
PREEL process: “practitioners could not see the links between
the research and their particular needs”
LDSE (Learning Design Support Environment): “It will be a
space for teachers to get support in designing their lessons
with integrated technology, share their practices of
technology enhanced learning and make it easier for other to
embed TEL in their practices.”
Action research as a way forwards?
Meanwhile, somewhere in York…
The Observatory
A service promoting and exploring the use of
practice- and research-based evidence to influence
policy and practice in teaching and learning in Higher
Education
Tools to enable access to evidence and syntheses of this
evidence (e.g. prototype repository and syndicated search,
wiki)
Spaces (real and virtual) to help communities explore
evidence-based practice and its implications for students'
learning
A work in progress
Initial proposal to HEFCE for an e-Learning Research
Observatory
Landscaping study
https://mw.brookes.ac.uk/display/hearoc
Judged to have wider relevance
A research observatory for Higher Education
e-Learning, Widening Participation, Employer Engagement
Other ‘strands’ may be added - although the strands may not be
explicit represented in the final structure
Overview of development
•
August 07 – July 08
•
•
•
•
•
Scoping the Observatory
Landscaping report for e-learning (exemplar area)
Generation of pilot resources and services
Proof of concept piloted at Academy conference
July 08 – July 09
•
•
•
•
Development phase
Pilots focusing on community engagement
Wider consultation
Promoted at Academy conference July 09
Landscaping evidence use in e-learning
Series of exploratory studies within e-learning
Beetham, Sharpe & Benfield
‘Landscaping’ consultation
Interviews with key informants
Survey (116 responses)
Follow-up interviews
https://mw.brookes.ac.uk/display/hearoc/
Yes, but…
Keen on single point of access; research reviewed,
evaluated and synthesised
After that, great variability
“The impossibility of categorising respondents as
users, producers, policy makers or intermediaries for
research is in itself is an important outcome.”
HB: What kind of research or evidence should a research
observatory focus on?
I: Evidence that e-learning really works.
HB: What would that evidence look like?
I: It would need to show real improvements to learning outcomes,
across a large number of students. It would have to have
credibility and rigour.
HB: Can you think of an example of research evidence of that kind?
I: Not off the top of my head, no.
HB: So does this research really exist?
I: No, the observatory would have to fund it. OR Yes, it is out there,
the observatory will have to work really hard to find it.
HB: Can you think of a situation when evidence like this has really
changed people’s practice or understanding, in your
experience?
I: Well, the cynics always ask for evidence that e-learning really
works.
HB: Do you think that evidence, if you had it, would lead them to
change their minds?
I: No, they would find arguments against it from their own discipline
perspective.
HB: So what about people who are actually open to change?
I: They never ask for evidence. They ask for examples, especially
from their own subject area, and practical ideas. They are really
responsive to other people in their discipline who have tried
something and made it work.
Identified strategic choices, approaches and risks
Strong persuader or neutral observer? (Setting agendas)
Funder?
Push or pull communications?
Quality assured, selective research or evidence and examples?
Audience: researchers, intermediaries, practitioners or policy
makers?
Building authority or democratic knowledge? (Who can write?)
Central or local knowledge management?
Face-to-face or technology supported networks?
Building from this: e-Learning pilot
How do communities produce, share and use evidence?
QA/QE in e-learning SIG
Expert review about e-portfolios
Discussion about professional role (M25 Learning Technologists
group)
Open peer commentary on the national development programmes
Review processes, to identify approaches that may have
wider value
Feed these back to inform the development of the whole
observatory
Echoed conclusions from IoE projects
Resources provided for people were not necessarily
(ever?) taken up
Communities rise and fall
Expert review uninviting to others (useful, but not an
invitation to contribute)
Open peer commentary engaged invitees then
stopped
Pilot site for preliminary consultation
Building from this pt2: wider consultation
Inviting contributions from wider groups
Employee learning and widening participation communities
TLRP conference workshops
ELESIG meeting (Thursday)
Case studies with communities (Heads of e-Learning Forum)
Input into specifying the Observatory; documented
cases of evidence generation and use
Building from this pt3: e-Learning projects
Small grants for research
Projects awarded, visited once, conclude by
generating reports
New model
Projects awarded, brought together, given technical
infrastructure (wiki, Ning)
Will be visited, encouraged to use Web2.0, brought
together mid-project and again at end
Wiki for Academy-funded projects
However…
Continued visibility of central resource
Yes, it’s a repository… but it’s not just a repository
If it’s just a website it’ll be pointless
Invitation to engage does not guarantee engagement
And what next?
New round of projects (provisionally)
Smaller-scale reviews
Special Interest Groups
Possibly projects about technology to support evidenceinformed practice
Emphasis on engagement (I hope)
Wiki-based reviews for open engagement
Social networks – finding people, not just research outputs
Conclusions?
Scepticism about centralised determination of evidence
and method
Authoritative or authoritarian?
Interested in community production and use of evidence
Community-owned tools and spaces as a worthwhile
experiment
Some possible starting points for
discussion
A personal bias towards a social account of negotiating
the meaning of encounters with research evidence
How credible is the approach represented here?
Is this approach to evidence the right one?
How should the choices outlined earlier be
responded to?
Will this be able to help change practice?
How does this relate to you and your practice?
What kinds of decisions do you draw on explicit
evidence for?
How could these be supported?
How could you share your conclusions?
What kinds of synthesis or structuring would be most
useful and credible?
Emails
- [email protected]
- [email protected]
- [email protected]
More information at:
http://www.heacademy.ac.uk/ourwork/research/observatory
Piloting site:
http://academy-research-observatory.pbwiki.com/