Transcript University of Maryland Dissemination and Implementation

Barbara Resnick, PhD, APRN, BC, FAANP
University of Maryland, School of Nursing
Ann Gruber-Baldini, PhD
University of Maryland, School of Medicine
1



Participants- thorough description, including
demographic information, injury or disorder
characteristics, mean time since onset
Measures- include a detailed description of
all measures used, including their reliability
and validity information
Procedure- Include all procedures in detail,
so that someone else could replicate your
study exactly; include recruitment, research
design, and types of statistical analyses




There is no specific page limit, but a key
concept is to keep this section as concise as
you possibly can.
People will want to read this material
selectively.
The reader may only be interested in one
formula or part of a procedure.
Materials and methods may be reported
under separate subheadings within this
section or can be incorporated together.

The objective is to document all specialized
materials and general procedures, so that
another individual may use some or all of the
methods in another study or judge the
scientific merit of your work.
◦ It is not a detailed recipe.
◦ it is not a long winded story.




It is awkward or impossible to use active
voice when documenting methods without
using first person.
First person writing would focus the reader's
attention on the investigator rather than the
work.
Use third person passive voice.
Use normal prose with complete sentences –
avoid informal lists.




Materials and methods are not a set of
instructions.
Omit all explanatory information and
background - save it for the discussion.
Omit information that is irrelevant to a third
party, such as what color ice bucket you
used, or which individual logged in the data.
NO RESULTS should be included in methods



The purpose of the section is to make it
possible for interested readers to repeat the
author’s experiment …to reproduce results.
Explain exactly what was done.
Think of bench research:
◦ What experiments were run and how they were run,
what equipment and materials were used and how
they were used, how much, how often, what, where,
when, and why.

Design
◦
◦
◦
◦
◦
◦
RCT
Quasi exp
Repeated measures
Single group
Randomization process
Who if anyone was blinded?
 Double blinded-interventionist doesn’t know (ie
provider in drug trials) and participant doesn’t know
 In social sciences what methods were used to preserve
blinding

Sample
◦ Recruitment process-who did the recruitment?
◦ Number available; number contacted; number
approached; number consented; number refused;
reason for refusal
◦ Eligibility criteria
◦ Determination of sample size
◦ Stopping rules if relevant
◦ Randomization-who performed this?


Declaration that an institutional review board
governing research has determined that the
study protocol adheres to ethical principles.
Without such approval, no research project
can be conducted nor can it be published in a
reputable, peer reviewed journal.

Consolidated
Standards for
Reporting Trials
(CONSORT)
Diagram

Location of the study
◦ For example: If in institutions as relevant a brief
description of site is helpful
 Inner city
 Rural
 1,000 bed or 25 bed

Measures
◦
◦
◦
◦
The what aspect of measures!
Brief description of what data was collected
Brief evidence of reliability and validity
Acknowledgement of no evidence of reliability and
validity but rational why the measure was used

Protocol for data collection
◦ Who: Unit nurses, research assistant, family
◦ When: baseline, 2, 4 and 6 months post
implementation of the intervention
◦ Where: location such as primary care office; home
setting; nursing home room
◦ How: Face to face interview; paper and pencil test;
internet survey etc.




Describe your intervention in sufficient detail
to conceptualize a replication.
Reference prior work or refer to the web for
further detail (as appropriate)
Write this in the past tense and third person
….The exercise program was implemented
every second Thursday of the month and
lasted for 2 hours.
….The specimen was centrifuged for 10
minutes.


Gory details can be placed in a table to save
room.
What are the gory details?
◦
◦
◦
◦
◦
Intervention dosage
Strength of dose
Time intervals of dosing
Who is implementing the intervention
See Conn article WJNR
http://wjn.sagepub.com/content/34/4/427
The online version of this article can be found at:
DOI: 10.1177/0193945911434627

Gory details
◦
◦
◦
◦
◦
◦
◦
◦
◦
◦
◦
Reference if previously developed
Conceptual framework
Intervention components
Timing of delivery
Dose
Mode(s) of delivery (e.g., face to face; internet)
Intervention target and recipient (e.g., patient or family)
Delivery setting
Culturally relevant
Intervention variations: men/women ; tailoring rules
TREATMENT FIDELITY –plan can be presented here







Content: What was the content and how was it
delivered
Provider: who delivered it?
Format: What method was used (telephone,
individual)
Setting: Where was it done
Recipient: To whom was the intervention
delivered?
Intensity: How often/how long for each touch?
Duration: Over what period of time were there
intervention contacts?
Component
Description of the Intervention
Component I
Environmental
and Policy
Assessments
The identified facility champion worked with the research function focused care nurse (FFCN) to complete
environment and policy assessments using Environment and Policy/Procedures for Function and Physical Activity
Evaluation Forms. The findings were used to identify and recommend appropriate and affordable interventions to
alter the environment and policies/procedures so that they would optimize function and physical activity of
residents (e.g., make pleasant walking areas inside and outside sites).
Component II Education of nursing staff, other members of the interdisciplinary team (e.g., social work, physical therapy),
Education
residents and families was done by the FFCN using previously established materials and adult learning techniques.
All staff in the communities were invited to attend the 30 minute educational session. For those who are unable to
attend a face-to-face session, a printed powerpoint was available. Hard copy educational materials were given to
all residents and available family/proxies.
Component III The FFCN worked with the champion and staff DCWs to complete Physical Capability Assessments and Goal
Establishing
Attainment Forms with each resident. Goals were established based on capability assessments, communication
FFC Goals
with other members of the team and input from residents. In addition, the FFCN identified intrapersonal factors
(e.g., cognitive status, pain, fear of falling, drug side effects such as somnolence) and worked with members of the
health care team to implement interventions to optimize patient participation in functional and physical activity.
The FFCN also worked with the champion to make sure the goals were written into the required Service Plans on
each resident.
Component IV The FFCN worked to motivate all caregivers to provide function focused care throughout the 12 month period by:
Mentoring and (a) observing performance of caregivers and providing one-on-one mentoring to incorporate FFC into routine
Motivating
care; (b) providing caregivers with positive reinforcement for providing FFC; (b) meeting with caregivers to
address their beliefs about physical activity and feelings and experiences associated with providing FFC; (c)
reinforcing benefits of FFC and strengthen outcome expectations; (d) highlighting role models (other caregivers
who successfully provide FFC); (e) identifying change-aides and positive opinion leaders to disseminate and
implement FFC and eliminate the influence of negative opinion leaders.

Can include a diagram to demonstrate flow
thru the protocol
◦ Include any deviations or decisions made at
particular points through the process.



How was nesting/clustering addressed?
Randomization by setting or individual?
Adjustments done and how done?
Sufficient detail for replication
◦ Reference statistical methods for more detail




State if an intention-to-treat analysis was
done
State if any participants were excluded for
any reason from the analyses
State if subgroup or additional analyses were
performed
State level of significance used and be
prepared to address inflated p for multiple
tests (often addressed through method of
analysis).







Statistical methods should be described in sufficient detail to
enable a knowledgeable reader with access to the original data
to verify the reported results.
References for statistical methods should be to standard works
when possible.
Any computer programs used should be identified.
Statistical terms, abbreviations, and symbols should be defined.
Details about randomization, if used, should be given, as well as
concealment of allocation to treatment groups, and the method
of masking (blinding).
Losses to observation (such as dropouts from a clinical trial)
should be reported.
It is recommended to include the word “considered” in
descriptions of statistical significance, such as “a P value of less
than 0.05 was considered statistically significant”, since the
choice of this cut-off point is arbitrary.

Hitting just the right level of detail is difficult
in these sections.
◦ Enough detail for a reader to reconstruct his/her
study, but not so much that the relevant points get
buried.
◦ Ask yourself at each place: “Would I need to know
this to reproduce this experiment?”
◦ This is tooo much!...
 “We rolled the patient over and wiped the skin clean
prior to applying the dressing. We did this with every
dressing change”.…assume they know something.

Not Enough Information is more commonly
the problem
◦ Make sure it is replicable!


DO NOT include results in the methods
DO NOT include discussion in the methods

If a plan is in place this can be explained in
the methods section
◦ Procedures or Measures as relevant

Same information is needed in methods
◦
◦
◦
◦
Sample
Intervention-not usually relevant
Measures
Data Analysis





It is probably the case that convenience
sampling is the most frequently used in
qualitative studies.
State the size and type of sample used in the
reported study.
If an unusual variant of sampling is used, it is
useful to acknowledge the nature of it.
Other comments about the sampling process
may be helpful-snowballing
SIZE and whether saturation was achieved




It is important to explain why your data set is
the most illustrative and useful to answer the
question you are posing.
Be careful to describe how you picked your
sample. What criteria did you use?
Can you compare the data set to other
alternatives and why did you choose this one?
Describe the important variations within the
data set (for instance age and gender
distributions) so that the reader gets a good
picture of it.


Descriptive data included?
Most qualitative studies (but not all) the data
collection method is usually the interview method.
◦ How the interviews were carried out
◦ Location/timing/by whom/ questions asked

Example of write up:
◦ All students were interviewed by the researcher on two
occasions, for between 30 and 45 min. All interviews were
recorded, with the permission of the students being
interviewed. After the interviews, the recordings were
transcribed into computer files. Care was taken by the
researcher to assure the respondents that they and the
place of their work would not be identifiable in any
subsequent report. Once the final research report was
written, the tapes from the interviews were destroyed.

Describe how the researcher handled the data
◦ ‘The interviews were recorded and transcribed. The
researcher then sorted those data into a range of
categories and these are reported below’….is a bit
too brief!

Care should be taken with very general terms
such as ‘content analysis’, when reporting
data analysis. The term is probably so broad
as to have little meaning.



Describe carefully each step in the analysis to
make it possible for the reader to believe that
your conclusions are correct -- or argue
against them.
A good rule is to present the analysis of one
observation/item/response in detail.
Describe your interpretations during the
analysis in a systematic way, in small
identifiable steps.

For example might describe “in vivo” coding
which uses the participants own words for the
codes.

Data analysis was done using basic content analysis(Crabtree B & Miller
W, 1992) and started with the first interview. The analysis began with “in
vivo” coding(Strauss & Corbin, 1998), or “grounded” coding (Glaser&
Strauss, 1967), which involves using the informants' own words to
capture a particular idea. The following is an example of “in vivo” coding:
The code identified was independence…"They feel more independent,
you know, because it seems like most of their ability to do things
independently goes away when they come here”. The codes identified
were grouped based on similarities and differences. For example, a
number of codes arose from the data that focused on facilitators of
restorative care such as encouragement of the resident, cueing the
resident, or asking them to “help you out”. These were combined under
the theme of “Facilitators " of restorative care. Coding was completed
initially by the principal investigator and a code book established. The
second nurse investigator with experience in implementing restorative
care programs review the coded data and revised the codes and added
new codes based on her review. This two coders then reviewed the data
and codes together until consensus was achieved between the two
reviewers.
◦ All of the interview transcripts were read by the researcher
and coded in the style of a grounded theory approach to
data analysis (refs). Eight category headings were generated
from the data and under these all of the data were
accounted for. Two independent researchers were asked to
verify the seeming accuracy of the category system and
after discussion with them, minor modifications were made
to it. In the grounded theory literature, a good category
system is said to have ‘emerged’ from the data (refs). Other
commentators have noted that, in the end, it is always the
researcher who finds and generates that system (refs).

Carefully describe the reliability and
validity/Confirmability of your data analysis
process
◦ Reliability-recognizability of the
findings/transfirmability
◦ Validity-consensus among a group


Credibility of the data refers to the believability, fit, and applicability of
the findings to the phenomena under study (Denzin & Lincoln, 2000;
Lincoln & Guba, 1985). The focus groups were done when the 12 month
intervention was completed in each of the treatment facilities. Since
completion of the intervention by site occurred at different times, this
allowed the investigators to use the findings from the first focus groups
to confirm or refute codes and emerging themes in the subsequent
groups.
Confirmability or auditability of the data refers to the objectivity of the
factual aspects of the data (Lincoln & Guba, 1985). Confirmability of the
data was considered by having other members of the research team
review the findings and provide feedback as to whether these findings
logically fit with other settings and experiences. Specifically the findings
were reviewed by three different RCNs that had worked in the treatment
facilities, as well as the co-investigators on the study including three
epidemiologists with experience in long term care research.
This guide for observational studies may be
helpful:
von Elm E, Altman DC, Egger M, Pocock SJ, et
al. (2007) The Strengthening the Reporting of
Observational Studies in Epidemiology
(STROBE) Statement: Guidelines for Reporting
Observational Studies. PLoS Med 4(10): e296.
http://www.strobe-statement.org/


Methods Section
◦
◦
◦
◦
◦
Design
Sample
Intervention as appropriate
Measures
Data analysis


Homework due Thursday, June 13
Please send to: [email protected]

We’ll see you back here on Thursday, July 25
at 4:30 p.m.