Transcript Slide 1

Connecting Research,
Policy and Practice
First Impressions:
How to Win Grants and Influence
Your Research Career
Meredith Larson, Ph.D.
National Center for Education Research
and
Amy Sussman, Ph.D.
National Center for Special Education Research
ies.ed.gov
Goals
1. Identify two things you should have with you at all
times.
2. Describe a theory of change and discuss why it’s
essential grant writing.
3. Have the tools necessary for refining your research
statement of purpose.
4. Have a sense of what program officers and reviewers
want from you and your applications.
All of this is to help prepare you to write winning
proposals and articulate your research agenda to the
broader (research) community.
ies.ed.gov
Our Assumptions of
You and What You Want
• We assume that you want
– to help improve education
– to carve out your own research agenda
– to get funding to conduct your research
• We assume that you have
– expertise in a wide range of topic areas and
research methods/analyses
– varying levels of familiarity with grant writing
ies.ed.gov
What You’re Facing
Whenever you ask someone to support your
research, remember that it all comes down to
convincing them that they should care about
your topic and that you have the necessary
competencies to reach your goals.
…so how do you do it?
ies.ed.gov
What You’re Facing
Audience awareness
1. Seek common ground.
• Highlight core values or concerns that your research
addresses.
• Look for ways to build off of things that are familiar.
2. Be helpful and thorough.
• Make things easy to understand, remember, and find.
• Be engaging and helpful.
ies.ed.gov
What You Need to Do
Sell your research idea, promote yourself as the
best person to do the research, and build
goodwill and trust.
How?
By demonstrating that you know what the
problem is and have a way to address it.
ies.ed.gov
Explaining Your Research
You should have two things ready at all times:
1. Statement of Purpose (preferably an “elevator
speech” or “research pitch”)
2. Theory of Change
ies.ed.gov
Statement of Purpose
We will come back to this later, but in general a
ready-made Statement of Purpose should
– Be short and attention-getting and
– Contain the problem statement and your contribution
to solving it.
Your fellow researchers, friends, and family members
should be able to understand it and see its relevance.
NOTE: It’s not as easy to do as you may think. You have to know a
lot about what it is you want to do before you can describe it
succinctly.
ies.ed.gov
Theory of Change (ToC)
• This is the model underlying your research.
• It is a roadmap to your grant narrative.
• It is a source for generating research
questions.
• It is constantly evolving.
NOTE: Some fields and scholars use terms like Logic Model or Logical
Framework to mean things similar to what we are calling a Theory of
Change here.
ies.ed.gov
ToC: What is It?
Theories of Change help define how and why an
intervention (e.g., curriculum, policy, practice)
should lead to particular outcomes.
– Makes assumptions explicit
– Suggests the causal relations
ies.ed.gov
ToC: What is it?
• Some associate them with program evaluation
studies, but they are also useful for
exploration work, work on cognitive or
behavioral processes, etc.
• They are frequently represented visually, but
they can be simply text.
ies.ed.gov
ToC: Why You Should Care
Reviewers will evaluate your research on how clear
your theory and model are. A strong ToC will help
make your case for you.
Four Criteria Used to Judge ToCs (Connell & Klem, 2000):
1.
2.
3.
4.
ies.ed.gov
Plausible
Doable
Testable
Meaningful
ToC: How to Make One
• Start with the long-term outcome and work
backwards.
– What preconditions need to exist to lead to the
outcome?
– What is occurring in the context that could hinder or
support the outcome?
– What assumptions are you making?
• Draw it out.
• Write it up.
• Get feedback. Revise. Repeat.
ies.ed.gov
Create the General Framework
Come up with the broad framework.
Initial State
Context
Strategies/
Activities
Intermediate
Outcomes
Long-term
Outcomes
Generally, you should start with the outcomes.
You can then jump all the way back to thinking
about the initial state or hop around.
ies.ed.gov
The Basic Components
Assumptions
Initial State
(resources)
Strategies/
Activities
Outcomes
(proximal &
distal)
Context
INITIAL STATE: What’s there before the intervention including things such as resources
or student characteristics, etc. (sometimes called input)
STRATEGIES/ACTIVITIES: The things that happen (e.g., activities, events, curriculum) to
the people who participate or who are the targets of the intervention (sometimes
called output)
OUTCOMES: The proximal and distal changes for people involved in or that
are the targets of the intervention
ASSUMPTIONS: What you believe to be true of the resources, the people
involved, the people targeted, and the intervention in general
CONTEXT: The environment (internal or external) in which the intervention takes place
(e.g., other policies or practices occurring simultaneously)
ies.ed.gov
Example
4-year-old
pre-K children
Demographics
Eligibility/need
Exposed to
intervention
1-hour lesson
Wrap-around
services
Positive supports
Focus on early
literacy
PD for providers
Positive
attitudes to
school
Improved
pre-literacy
skills
Increased
school
readiness
Learn
appropriate
school
behavior
Setting/context
Personal and family characteristics
Prior experience
Teacher/provider experience
ies.ed.gov
Greater gains
in K literacy
Detailed Components
Now that you have the general aspects, you should
specify the details. These will lead into your research
design, measures, etc. in your narrative.
Indicators: Operationalizing the outcomes (e.g., what indicates
success?)
Populations: Identifying who should show which change (could be
multiple groups)
Thresholds: Setting your expectations for change for each outcome
(e.g., how much is good enough?)
Timeline: Determining when we should meet the threshold for
each outcome
(Connel & Klem, 2000)
ies.ed.gov
Example
OUTCOME = Improved pre-literacy skills
Indicator: Correctly identifies letters and sounds
Population: 4-year-olds, especially those with
little print exposure
Threshold: 1.5 effect greater than comparison
Timeline: After full intervention, 16-weeks
ies.ed.gov
Another Example
(Slightly different layout )
Instructional Context
Text and task
characteristics
Nature of
instructional support
Reader
Characteristics
Decoding and fluency
proficiency
Verbal knowledge
Attention and
behavior
ies.ed.gov
Motivation to Read
Decision to engage in reading and task
persistence
Expectancies for
success
Self-efficacy
Perceived control
Value
Achievement goals
Intrinsic motivation
Usefulness
Social motivation
Text Comprehension
Reading Engagement
Physical
Cognitive: Building and
maintaining coherence
Adapted from a model
presented by Dr. Sharon
Vaughn (R305F100013).
Don't Do This
1. Be overly simplistic
2. Overwhelm the reader
3. Use color as a key (audience awareness!)
ies.ed.gov
Example: Too Simple
Students
with
Disabilities
ies.ed.gov
The
Increasing
School
Success
Program
Increased
Academic
Achievement
[Processes 3-17 repeat
twice in Year 2]
16. interview
coaches, SLTs,
and PLTs about
PD and WL
processes
13-15b.
Observe SLTs,
document
implementation
15a.Coaches
assist SLT in
implementing PD
with faculty
14a. SLT and
Coaches
create PD
unit
8a2. Coaches
assist PLTs in
weekly WLs and
Debriefing
through mid-year
WL
Debrief
WL
Debrief
PLT
8a1. Coaches
assist PLTs in
using all data
PLT
PLT
WL
Debrief
SLT 1
Implement
Prof Devt
PLT
PLT
WL
Debrief
Begins
weekly
meetings
PLT
6-8b. observe
PLTs,
documents
implementation
7a. Coaches
teach PLT to
conduct WL
6a. Coaches
facilitate PLT
identification of
annual goal
PLT
WL
Debrief
approx. 3 wks
after school
begins
WL
Debrief
PLT
13a. researchers
train SLT to
research bestpractices in PD
area
PLT
Profile Analysis
Set instructional
goals & WL focus
PLT
[PLT appoints
SLT1 to address
PD Topic 1]
= Begin Process
= Feedback for next process
= Delegation of PD
PLT
= Primary Leadership team
SLT
= Secondary Leadership team
PLT
PLT
WL
Debrief
PLT
[Processes 4-11 WL
repeat to mid-year]
Debrief
12. Coaches & PLTs
choose 2-4 teachers
(based on WLs) to
become Dynamic
Leadership Team 1]
WL
Debrief
11. Coaches
share/
implement
revisions
with PLTs
10.
reviews/revise
s model based
on findings
approx. 9 wks
after school
begins
22
4. Coaches
work with
PLT
WL
Debrief
PLT
WL
Debrief
5. guide Coaches
and PLTs in
analyzing data
collected
PLT
[4 weeks at end of prior
year or beg of current
year]
WL
Debrief
SLT 1
WL
Debrief
PLT
WL
Debrief
Research
ies.ed.gov
18a. submit
Annual Report
to schools
17.
reviews/revise
model based
on findings
9. interviews
coaches and
PLTs about
WL process
1a. PI
Recruits
and Trains
Coaches
1b.
develops
data
collection
tools
2a. PI
matches
Coaches
to schools
(n=5)
3b.
develops
electronic
tool
3a. Coaches
collect 3 yrs.
stud. ach. &
demo. data
per school
2b. trains
coaches
to use all
data tools
Draw your Theory of change
Take a few minutes and write a couple words
about the following:
1. What are the initial states (inputs)?
2. What are the strategies/activities (outputs)?
3. What are the outcomes? Proximal, Intermediate,
Distal…
4. What is in the context?
ies.ed.gov
So Now What?
Your Theory of Change acts like a roadmap that
can help lead you to research questions, a
proposal, and so much more.
ies.ed.gov
ToC & Research Questions
Questions about initial states or inputs
– Were resources provided and used?
Questions about strategies/activities or outputs
– Did the activities occur as planned?
Questions about outcomes
– Were there changes in the proximal/distal outcome relative
to the control/comparison?
Questions about context
– Did the context suppress or reinforce effects?
ies.ed.gov
(Milanowski & Kimball, 2009)
ToC & Your Application(s)
In IES applications, there are four major parts of
the application: Significance, Research Plan,
Personnel, and Resources. The first two of them
can draw heavily from the ToC.
Significance – Draws heavily from the outcomes,
context, and assumptions
Research Plan – Draws heavily from the
strategies/activities and the outcomes
ies.ed.gov
ToC & Your Research Plan
In your research plan, you need to specify exactly what
it is you’re exploring, creating, validating, or testing. You
also need to specify how you will do these things.
Strategies/Activities
What are the pieces that you’ll be exploring, creating, testing, etc.?
Outcomes
Indicators: What will you measure, and how you will measure it?
Populations: Who and where (both in treatment and
control/comparison)?
Thresholds: What effect (size) should you expect?
Timeline: When should you be collecting what data?
ies.ed.gov
ToC & Your Application(s)
Other components of your narrative and your
application can also be informed by the ToC:
Personnel – For example, you want someone with
experience in each of the outcome areas and with knowledge
of how to assess the strategies/activities.
Budget – For example, how many observations will you need
to take (# of researchers and % effort) and when (during what
grant year) and what assessments you will need (how many
of which one should you buy).
ies.ed.gov
ToC & Your Statement of Purpose
Your Theory of Change also contains the core
elements of a good statement of purpose,
mission statement, elevator speech, etc.
You have the pieces.
Put them in a short, digestible format that
conveys the problem’s significance.
And now you have your statement.
(It actually requires a bit more than that.)
ies.ed.gov
An Example
Now we’re going to walk through an example of
how you might take a ToC and generate an
“elevator speech”, a statement of purpose, even
an opening paragraph.
Note: Sometimes people have their statement and then create their ToC. You
need both, and they inform one another. You do not need to have a ToC to
write a statement.
ies.ed.gov
From ToC to Statement of Purpose
Distal Outcome: Increase student achievement in science
Proximal Outcome: Young children (preK – K) with greater ability to
think scientifically
Strategies/Activities:





Professional development curricula
Four 1-hour long workshops for teachers
Training on science content and early childhood pedagogy
Current, practicing educators
PreK students
Initial State: Early childhood educators who lack science content and
pedagogical knowledge, lack of curricula to help teachers
Assumption: Teachers are essential to student outcomes. Teachers
need both content and pedagogical knowledge and expertise. Teaching
teachers will improve student outcomes.
ies.ed.gov
…continued
Speech:
Being able to think scientifically at a young age predicts longterm academic success in science. However, many children
coming into elementary school lack this ability. Even if they
are exposed to science in PreK programs, they are still not
acquiring the necessary skills. One reason PreK students may
be struggling is that early childhood educators often do not
have the background knowledge of science content or
instruction to help their students. Because teachers are so
important to the learning process, we need to help better
prepare them for their role. So I am developing an
intervention that will increase early childhood educators’
content and pedagogical knowledge so that they can better
prepare their students.
ies.ed.gov
Components of a Good Statement
• Story! Characters, goals, drama, resolution.
• Facts – but not too many. This will vary based
on audience and use.
• Short – shoot for a paragraph or < 90 seconds.
Make them care.
ies.ed.gov
Now you try
• What is the problem? The tension? The
drama?
• What evidence do you have that it’s
important?
• How can you help solve it?
ies.ed.gov
Uses of Your Statement
• Opening paragraph(s) of your applications or
your purpose paragraph
– Note: This is true for all documents, such as dissertation proposals, not
just IES applications.
• Job talks
• Communicating with the general public (e.g.,
when you give your NPR interviews)
ies.ed.gov
Importance of Opening Paragraph
• Opening paragraph sets the scene for
readers.
– Identifies the significance of the work to be
done and what actually will be done
– Readers use it to organize information in rest of
the application
– You can lose your readers right off with an
unclear opening
ies.ed.gov
NCER/NCSER Abstracts
As an example of an opening/purpose
paragraph, we’d like to use examples from some
of the abstracts we have posted online:
http://ies.ed.gov/funding/grantsearch
ies.ed.gov
Example of Opening/Purpose
Purpose: Teaching children how to think scientifically in the
preschool years has the potential to address an existing
achievement gap in early science and provide children with the
skills necessary to continue learning and thinking critically
throughout the school years.
Background
Problem
Unfortunately, early childhood teachers typically lack content and
pedagogical knowledge in science and are not prepared to provide
developmentally appropriate experiences that support children's
early science learning and readiness.
In order to address these challenges, the researchers will develop
Goal:
the Cultivating Young Scientists (CYS) intervention, which includes How will you
a professional development program for early childhood
address this
educators, science curricular content, and a set of formative
problem?
assessment tools. The intervention is intended to lead to an
increase in teachers' use of science instructional content and
practices in preschool settings and improvements in young children
science content knowledge and scientific thinking skills.
ies.ed.gov
Example of Opening/Purpose
Core diagnostic features of autism include deficits in social-communicative
functioning. Two pivotal skills for young children with autism include joint
attention and pretend play, which constitute early foundations upon which later
social-communicative skills are built. Joint attention (characterized by behaviors
such as pointing, showing, and coordinated looking to share attention toward
objects or events with another person) and symbolic play (characterized by the
ability to pretend), play important roles in language development and social
engagement with peers.
Children with autism show deficits in these skills, and as a consequence may lag
behind their peers academically and socially. Advancing Social-Communication
and Play (ASAP) is an intervention that has recently been developed to help
preschool children with autism learn and practice these important skills.
However, we do not know how effective it is.
The purpose of this research is to conduct a cluster randomized trial to
evaluate the efficacy of ASAP. The major goals of the project include
investigating whether children experiencing the intervention demonstrate
greater gains in the proximal child outcomes of social-communication and play
skills as well as the more distal outcomes of language development and
engagement with classroom objects and peers. The study will also examine
whether child-level and teacher-level (i.e., teacher burnout, general classroom
quality) characteristics moderate the impact of the intervention.
ies.ed.gov
Background
Problem
Goal:
How will you
address this
problem?
Final Comments:
Helpfulness of the Program Officer
• Share your framework and statement of
purpose with the Program Officer
– Ensure you are submitting to the correct
competition/topic
– Springboard for further discussion
ies.ed.gov
Final Comments:
Importance of Clarity of Writing
• Readers (e.g., application reviewers) often
complain about lack of clarity.
– Significance too general
– Lack of detail regarding intervention,
development cycle, or data analysis
– Use of jargon and assumptions of knowledge
– Poor writing (e.g., grammar), awkward
constructions, etc.
ies.ed.gov
Final Comments:
Importance of Practicing
All forms of communication require practice.
Things only other people can tell you:
– Is your description of your theory too long or
complex?
– Is your statement of purpose too long or complex?
– Is your idea engaging, and does it invite questions
that build off of your ideas (rather than those that
try to figure out what your idea really is)?
ies.ed.gov
Pulling It All Together
Recall Why You’re Here:
You want to write winning grants and build a
solid research career.
But How?
By getting people interested in your work and
the questions you feel are important and by
building trust that you have a plan for
addressing an important issue.
ies.ed.gov
Pulling It All Together
• Have a theory of change
– Should be as fully explicated as possible
– Is always being refined
– Should be made with and informed by others
• Have a clear statement of purpose at all times
– Your “elevator speech”
– Simple, elegant (hard to do, actually)
ies.ed.gov
Connecting Research,
Policy and Practice
Thank you.
[email protected]
[email protected]
ies.ed.gov
More info
Remember: Some scholars distinguish between a Theory of Change and a Logic
Model. Others do not. We use the term “Theory of Change” for the purposes of this
presentation as a way to discuss how people can present the causal relations among
elements. Please use the term most appropriate for your field. (Note that the IES RFAs
use “Theory of Change.”)
This presentation was informed in part by the following resources:
Connell, J., & Klem, A. (2000). You can get there from here: Using a theory of change approach to plan urban
education reform. Journal of Educational and Psychological Consultation, 11(1), 93-120.
Center for Theory of Change: http://www.theoryofchange.org/. Retrieved May 1, 2013.
Harris, E. (2005). An Introduction to Theory of Change (Issue Topic: Evaluation Methodology, vol. 11). Retrieved
May 1, 2013, from http://www.hfrp.org/evaluation/the-evaluation-exchange/issue-archive/evaluationmethodology/an-introduction-to-theory-of-change.
Kimball , S. & Tony Milanowski, T. (2009). Establishing a Theory of Action and Logic Model for Your Project
Session 1 [PowerPoint slides]. Retrieved on May 1, 2013 from Center for Educator Compensation and Reform
website: http://www.cecr.ed.gov/pdfs/september2009meeting/TheoryofAction.pdf.
Reisman, J., Gienapp, A., Langley, K., & Stachowiak, S. (2004). Theory of change: A practical tool for action,
results and learning. Organizational Research Services for Annie E. Casey Foundation.
ies.ed.gov