Transcript WritingtheMethodSect.. - Usc - University of Southern California

General Suggestions for the Method
Section
•
The general goals of the Method section are threefold:
•
•
•
•
•
•
To lay out your methods so that readers can understand them
To describe your methods in sufficient detail so that they can be
replicated by other researchers
To demonstrate that you have addressed relevant threats to internal
and external validity
Go here and here for examples of complete methods sections.
Not every category described on the next several slides is present.
Journals allow flexibility in what headings and subheadings you use
as long as the information is present and sufficiently complete for
readers to understand what you did and how you did it. Some
Method sections are very short, others are more extended
To obtain a model for how to write your Method section, consult an
article from your literature review list of references that is most
like the study you propose to do and see how they wrote their
Method section
Your method section should probably be about one-three pages
long, depending on the complexity of your study
Writing the Method Section
•
•
In writing up your method section, you probably will want
to put everything in the past tense, as if you had already
done the study. Then when it comes time at the end of the
semester to put all the sections together, you won’t have to
change it from future to past tense
Following are some of the categories that are conventionally
found in a Method section. Not all of them will apply to
your study
•
•
•
•
Design
Subjects
Materials
•
•
Apparatus
Measures or
•
•
Independent Variables
Dependent Variables
Procedures
•
Development of Coding Scheme
Design
•
Briefly describe the study design, particularly if it is an
experiment, and the independent and dependent
variables.
•
•
For example, you might say that the study was a single
factor between-subjects design examining the impact of
job category classification (custodial, clerical, and
managerial) on attitudes towards job outsourcing by
American businesses
Another example would be: the study was a two-factor
repeated measures design examining the impact of two
independent variables, number of days of instruction (1,
2, 3) and number of practices per day (1, 5, 10) on
student learning of SPSS data entry
Design, cont’d
•
•
For a correlational study, you might say: “The study
was a cross-sectional analysis of the strength of
association between beliefs about nuclear proliferation
and attitudes toward global warming.” Presumably
you are not proposing any particular causal ordering
so it doesn’t matter what order you put them in
For a content analytic study, you might say: “This
study examined gender display in online personal
advertisements, using a new content-analytic scheme
based on gender-stereotypic categories of facial
gesture and posture present in advertisers’
photographs”
Participants (cases or units of
analysis)
•
•
•
•
If data are collected from human subjects, state the
number of participants, how they were recruited, and
provide any available information about their demographic
characteristics (age range, mean age, ethnicity, gender, etc)
Indicate if they were randomly sampled, and if so, describe
your sampling frame, if possible (for example, respondents
were a randomly drawn sample of all Presbyterian choir
directors in the greater Pasadena area (and define that).
Describe briefly how you compiled your sampling frame if
applicable. Indicate what your response rate was
If you did an experiment, indicate if participants were
randomly assigned to levels of the independent variable
If they were compensated for participation, describe the
type of compensation provided (extra class credit, gift
certificate, etc.)
Participants (cases or units of
analysis), cont’d
•
•
When submitting for publication or conference paper,
leave out any specific mention of the location of
recruitment (for example, say “a large private
Western university” instead of University of Southern
California)
If your data are collected from human subjects in the
form of information from their public or posted selfrepresentations (photographs, videos, messages on
SNS, blogs, IM utterances, etc.) provide what
information you have about the characteristics that
define this subject population and indicate how you
selected them, including a sampling frame if you have
one
Materials: Apparatus
• Apparatus: if you used any sort of physical
device as part of your study, such as
laboratory equipment, you can report it
under this heading
•
For example, “respondents’ hand movements
were captured with a Flock of Birds hand
tracker.” If you recorded participants’ speech for
later transcription and coding, you can report
the technical details under this heading such as
AV apparatus
Materials: Independent Variables
•
•
Under this heading you should provide a full account
of any manipulations that you conducted during the
study. For example suppose you were doing a study
about the impact of Interviewer Status on interviewee
self-disclosure, where you varied the status of the
interviewer by using variations in attire. It would be
appropriate in such an instance, for example, to
include figures in which you show still photos of the
“confederate” in each of the three dress conditions
(high, medium, and low status).
Also here you can include a description of any nonmanipulated factors that were treated as independent
variables with several “levels,” such as ethnicity or
marital status
Materials: Independent Variables,
cont’d
•
Report the categories of the independent
variable(s) you used and how you determined how
people should be assigned to categories
• For example, how did you decide if a prospective
subject was Hispanic? (self-report; member of
some existing group known to have Hispanic
members? Language in which the respondent
requested to complete the
questionnaire/interview?)
• If you use measures in more than one language
be sure to describe your translation procedures
including backtranslation if any
Materials: Independent Variables,
cont’d
•
•
Also under the heading Independent Variables you should
include descriptions of any scales or surveys which you used in
a correlational design as an independent variable (i.e., you are
arguing that it is causally prior), including questionnaires that
were used to classify people into levels of the independent
variable. Report previously established alpha reliability and
validity coefficients where available. Go here for quick and
easy reliability analysis-if you do your own you will report this
in the Results section rather than in Method
Also under this heading list and describe any measures that
you took to exclude alternative explanations, such as testing
people for covariates. If you thought that demographic
characteristics such as age might partially explain variation in
your dependent variable, loneliness, then you may have
collected demographic data on your subjects to use as a
covariate, as we did with the “educational attainment” variable
earlier in the semester
Materials: Dependent Measures
•
•
•
•
Under this heading you would provide details about how you
measured the variables you were treating as dependent variables
(effects, outcomes) in your study
Suppose for example that “loneliness” was your dependent
measure. Tell the reader if you measured it using the UCLA
Loneliness scale, and if so with which version (short, 21-item) and
which languages; if you measured loneliness by self-reports of
number of friends or frequency of social interaction; if you used a
diary method, or some other procedure. Refer the reader to an
Appendix where they will find copies of the items, unless it is a
very well-known scale. Provide a brief reference to the place
where you found the scale, preferably the original source, and list
that source in your references. (this also applies to measures of
independent variables)
If you used Likert-type items (strongly agree to strongly disagree)
or semantic differential scales, you can say so and then just
provide a table of the statements (“I often feel lonely”) or bipolar
adjectives that you used (“happy/sad”, etc.)
If you used interviews to obtain data, describe the interview setting
and protocol.
Materials: Dependent Variables,
cont’d
•
•
For scales like Likert-type or semantic differential indicate how
many “steps” were used (5, 7, etc). Where possible, report
the known validity and reliability of measure, any procedures
that you undertook to establish these can be reported under
Results
• You can say, for example, “Loneliness was measured with
the 21-item version of the UCLA Loneliness Scale, which
has a published validity coefficient of .xx with measures X
and Y and a known reliability of .xx. Then in your Results
section you can say “Reliability of the UCLA Loneliness
Scale obtained in the present study was high (alpha =
.90)”
Make it clear which measures are your independent variables,
dependent variables, and which ones you measured because
you thought they were potential confounding variables or
covariates
Materials, cont’d
• Generic “Measures” or “Variables”
category
• You would use this category to describe
all of your measures as above if you
were not treating them as IVs and DVs
but simply doing a cross-sectional
analysis and you were not suggesting a
causal order. Then you would eliminate
the separate IV and DV sections
Procedures
•
•
Describe in chronological order the procedures used
For a content analysis, describe your procedures for
capturing or logging the data and transcribing it
•
•
Describe your procedures for “unitizing” it (segmenting
audio, video, or text into “units” such as time units,
conversational units like the turn or utterance, frames,
etc) for applying your coding scheme and report the
unitizing reliability (Guetzkow’s U)
For an experiment, describe exactly what happened
from the time they were chosen for the experiment to
the time that they left the experiment
•
•
•
How they were contacted
The circumstances under which they arrived at the
testing location
How informed consent was obtained if applicable
Procedures, cont’d
•
Also include in the Procedures section
•
•
•
•
What instructions were given to subjects (sometimes you
might want to include these, in a block quote, italicized)
How the manipulations if any were introduced
How questionnaires were organized and administered (for
example, did you rotate any of the items, or any of the
questions in terms of order or direction of the scales to reduce
order effects)
Any physical steps you took to maintain control, such as
•
•
•
•
Ruling out extraneous variables (for example, in a learning
experiment excluding subjects who upon reporting to the test
site seemed not to be able to understand the consent form
and/or comprehend directions
Excluding or accommodating subjects who were unable to read
Excluding cases of posted messages from your corpus of
messages to analyze because they appeared to be spam or
were likely to be fictitious or designed to attract “flames”
Your debriefing procedures; how you gave and received poststudy information from the subjects
Procedures, Coding Scheme
Development
•
•
If you did a content analytic study you will want under
Procedures to include a section on “Development of the
Coding Scheme”
Go here for an exemplary Method section (p. 49) describing
the development, refinement, and establishment of intercoder reliability of a new coding scheme
•
•
This is a model of how to develop a coding scheme. If the
hyperlink doesn’t work (you need to be browsing from a .usc
domain), the reference is: Golish, T. D. (2003) Stepfamily
Communication Strengths: Understanding the Ties That Bind,
Human Communication Research, 29, 41-80. We read this
for Larry’s class.
Presumably you will have explained how you obtained,
transcribed and unitized your data corpus earlier under
Subjects and Procedures
Procedures, Coding Scheme
Development, cont’d
•
In the “Development of the Coding Scheme” you
should describe
• How you immersed yourself in a subset of the data
corpus and relevant literature to come up with a
preliminary set of categories,
• How you tested those categories against as yet
unexamined samples, and refined your categories
through an iterative process.
• How you created a coding manual for your coders.
• Your training process for coders, how you determined
coder reliability, and how inconsistencies were
resolved (see this excellent review article)
• What your obtained values of kappa (the inter-coder
reliability coefficient) were; go here for quick and
easy kappa computation