Transcript Lecture #1 Overview of the Research Process

Research
Methods in
SpeechLanguage
Pathology
The Research Process: An Overview

Research is
◦
◦

a systematic investigation designed to develop or contribute
to generalizable knowledge
a planned investigation of gathering and analyzing data
from a sample to draw conclusions about a research question
that can be applied to a larger population.
From a discipline perspective, research is
◦
◦
like diagnosis, particularly differential diagnosis, a processes
of asking and answering questions
a process of formulating “answerable” questions or forming
“testable” hypotheses, and then making the observations
needed to answer them.
The Research Plan

Most research plans evolve from the following factors:
o
o
o
o
o

Observation of a problem that leads to a question;
Development of a problem statement in the form of a testable
hypothesis;
Use of appropriate methods for testing the hypothesis;
Statement and interpretation of results; and
Discussion and evaluation of the results.
Guiding the research plan are the rules of the scientific
method that
o
o
o
recognize that a problem can be studied objectively;
data can be collected through observation or experiment; and
conclusions can be drawn based on an analysis of the data that
have been gathered.
Clinical Research and Evidence

Like the research plan, the scientific method can be
used to answer clinical questions and gather clinical
evidence as well.
The challenge for all practitioners is to answer the question:
“How do you know that what you do works?”


Answering clinically relevant questions or testing
clinically relevant hypotheses for clinical purposes
should be no different than for research purposes.
ASHA’s Code of Ethics (2003) states that “individuals
shall evaluate services rendered to determine
effectiveness.”
Clinical Research and Evidence


In order for clinicians to function ethically, they must
systematically evaluate the impact of the services
they render.
They must move away from decisions based on
opinion, past practice, and past teaching towards
clinical decision-making that is guided by science,
research and evidence (Elliott, 2004).
Evidence-Based Practice



Evidence-based practice is more than clinical
problem-solving.
It requires SLPs to integrate their individual clinical
expertise with the best available evidence from
systematic (scientific) research to demonstrate that
what they do works.
Like the scientific method, evidenced-based practice
involves the following steps:
Evidence-Based Practice
 Converting
a clinical need into an answerable question;
 Searching for and finding the best evidence to answer
the question;
 Critically evaluating the evidence you find for its
validity, strength, and applicability to the individual
client;
 Applying the results of the search and appraisal to
clinical practice; and
 Evaluating or auditing your performance.
Empirical Research


Empirical research is based on observed and
measured phenomena.
It involves questioning, observing, experimenting, and
eliciting behavior to:
o
o
o

define relationships
demonstrate cause and effect and
spark our own minds to begin thinking of other possibilities
to be tested and studied.
Empirical research leads to knowledge from actual
experience rather than from theory or belief.
Empirical Research Designs: Purpose




Empirical research can be classified by its purpose or
by the type of knowledge that will be produced.
The three levels of classification are exploratory,
descriptive, and explanatory.
Exploratory (preliminary) research is typically
conducted in the field and/or in natural settings or in
non-natural settings with the purpose of discovering
phenomena and/or theory.
Exploratory studies require no active manipulation or
alteration of the research context/condition by the
researcher.
Exploratory Research



It is typically conducted early in a research problem
to uncover basic viewpoints, perceptions, behaviors,
attitudes.
It can be quite informal, relying on secondary
research, such as literature or data reviews, or
qualitative approaches, such as informal discussions
with clients, employees, management or competitors.
More formal approaches would include in-depth
interviews, focus groups, case studies or pilot studies.
Exploratory Research
 Liebow
(1967) conducted exploratory research
through extensive observation and interview
techniques to obtain stories of men’s lives on Tally’s
Corner, an urban street corner in a black
neighborhood to explore the meanings of their
experiences.
 Liebow wanted to illuminate the richness of the lives of
these men as well as struggles that each man
encountered when attempting to fill varied life roles.
 His purpose in conducting an exploratory study was to
reveal theory, not to support or test existing theory.
Exploratory Research



The results of exploratory research are not usually
useful for decision-making by themselves.
The results can not be generalized because they are
not representative of the whole population being
studied.
But they can provide significant insight into the best
research design, data collection method or subject
selection procedure.
Descriptive/Ethnographic Research



Descriptive/ethnographic research involves the study
of human behavior through the description or
reconstruction of events that are observed as they
naturally occur.
The ethnographer collects naturalistic data which is
then arranged to render complete descriptions of
activities and interactions.
Ethnography is an appropriate methodology for
studying the customs, social patterns, speechlanguage behaviors, and rule-governed interactions
of a culture or group of individuals.
Descriptive/Ethnographic Research



With descriptive research, data are merely described
and information is explained in relationship to the
individual studied.
Categories of analysis are generated and the
relationship between the categories are explained.
No inferences to a larger group or population can be
derived from the analysis.
Descriptive/Ethnographic Research



A classic example of naturalistic research is embodied
in the work of Piaget (1932).
Piaget studied language development by primarily
observing and recording children's questions,
reflections, and conversations.
From such qualitative methods, he categorized various
stages of what he termed “egocentric” and
“sociocentric” speech development that, in turn,
stimulated much additional research.
Explanatory (Experimental) Research



Explanatory or experimental research is designed to
reveal causal relationships and to predict outcomes.
Such studies are founded on an accepted theoretical
framework of reference and seek to support theory
through hypothesis testing and prediction.
It essentially involves comparisons of groups or
individuals and relies on measurement and statistical
analysis of quantitative data.
Explanatory (Experimental) Research



Let’s say for example you wanted to know if a
hospital-based group socialization program was
effective in reducing depression in young adults with
a spinal cord injury.
You might structure the study so that these patients
were randomly assigned to either an experimental
group, in which the socialization program was
conducted, or to a control group, which received
individual counseling.
You would use a standardized depression scale that
fit your theoretical framework.
Explanatory (Experimental) Research


All participants would be tested on the scale before
their participation in the experimental and control
group programs and after the conclusion of the
programs.
A statistical comparison of the changes in depression
scores between the two groups would be used to
determine the extent to which there is a causal link
between the socialization program and lowered
depression.
Empirical Research Designs: Structure



Empirical research designs can also be classified by
the way in which data gathering and analysis is
organized and structured.
The first classification system is used to examine and
analyze the structure of experimental-type designs.
There are three levels: experimental, quasiexperimental, and non-experimental.
Empirical Research Designs: Structure





The second classification system is based on the
nature of the data collected and the type of analysis
conducted.
Using this scheme, studies are classified as either
“qualitative” or “quantitative” in their structure.
Those that use numerical data and statistical analysis
are quantitative.
Those that rely on narrative and categorical analysis
are qualitative.
Let’s start with looking at the structure of
experimental-type designs.
Experimental Designs


Experimental research is the appropriate method for
investigating cause-and-effect relationships among
variables, such as the effects of treatment on speech
or language behavior.
True experimental designs have the following
characteristics:
o Subjects
are randomly assigned to at least two or more
groups;
o Some type of active manipulation is performed; and
o One group of subjects (experimental group) is then
compared with another non-manipulated group (control
group) (Maxwell & Satake,1997).
Experimental Designs



Typically, quantitative measurements and methods of
data analysis are used to compare the experimental
and control groups.
The results are presented as quantities or numbers
(e.g., statistics).
Based on the results, the researcher is able to support
or refute the original hypothesis.
Experimental Designs


Additionally, because experimental designs control
for extraneous variables, the research is able to
generalize findings to the larger population from
which the sample is drawn.
Nonetheless, for many practical and ethical reasons,
it is sometimes impossible for an investigator to assign
subjects randomly to treatment groups or to
indiscriminately apply a particular treatment to one
group while withholding it from another.
Experimental Designs


Moreover, it could be argued that withholding a
treatment from a target population or administering
an alternative treatment with unknown effect rather
than one with established benefits is unfair or
perhaps illegal.
This is often the case in many clinical studies in which
an insufficient number of appropriate subjects may
preclude the use of randomization procedures.
Quasi-Experimental Designs



Quasi-experimental research designs are generally
selected when true experimentation is impractical or
impossible to perform.
Typically, subjects are assigned to groups on the
basis of preexisting conditions or circumstances.
Suppose you work in a hospital clinic where you treat
many adult patients for hoarseness accompanied by
vocal nodules.
Quasi-Experimental Designs


Following the diagnosis, the availability of therapy is
on a “first-come, first-served” basis so that many
patients are on a waiting list for three months or
more.
Although the use of randomization procedures may
not be possible, you still wish to draw some
conclusions about the efficacy of your treatment
program.
Quasi-Experimental Designs



To estimate treatment effectiveness, a constructed
control group must be established for comparison
with a treated group of subjects.
The two groups would be matched on a number of
variables prior to treatment.
Matching is done to control as many extraneous
variables as possible so that any positive betweengroup differences can be attributed to your
treatment.
Quasi-Experimental Designs


These variables would possibly include factors such
as the degree of hoarseness, size of the nodules,
duration of illness, occupation, age, sex,
alcohol/tobacco consumption, etc.
Quasi-experimental methods often necessitate the
use of more control procedures.
Nonexperimental Designs




Nonexperimental designs are those in which the
three criteria for true experimentation do not exist.
In such research, there is no attempt to achieve
randomization, nor is any purposeful effort made to
manipulate the variables under study.
These designs examine naturally occurring
phenomena and the researcher sets out to test and
describe a concept or construct or the relationships
among constructs.
The researcher does not manipulate the independent
variable but instead examines it in relation to one or
more variables for descriptive or predictive
purposes.
Nonexperimental Designs



Any manipulation of variables is done post hoc
through statistical analysis.
Because random selection, manipulation, and control
are not present in these designs, investigators must
use caution when make causal claims from the
findings.
In health and human services, the three most common
types of nonexperimental research include surveys,
passive observation, and ex post facto designs.
Survey




Survey designs are used primarily to measure
characteristics of a population and are typically
conducted with large samples.
There is relatively minimal expenditure and numerous
variables can be measured.
Perhaps the most well-known survey is the U.S. Census
which seeks to develop a descriptive picture of the
characteristics of the population of the U.S.
Statistical manipulation can permit multiple uses of
the data set.
Passive Observation



Passive observation designs are used to examine
phenomena as they naturally occur and to discern
the relationship between two or more variables.
These designs are often referred to as
“correlational designs.”
Passive observation can be as simple as examining
the relationship between two variables, for
example, height and weight.
Passive Observation


They can be as complex as predicting scores on one
or more variables from knowledge of scores on other
variables.
Like surveys, variables are not manipulated but are
measured and then examined for patterns.
Ex Post Facto Studies




Ex post facto designs are a type of passive
observation literally performed “after the fact.”
The phenomena of interest have already occurred
and cannot be manipulated in any way.
For example, an ex post facto design could be used
to examine differing career patterns of male and
female social workers after graduation.
One could look at patterns of gender differences in
salary, career opportunity, geographical preference,
etc.
Quantitative Methods



Quantitative research methods are rooted in the
tradition of physical sciences.
With quantitative research, the researcher explores
relationships using numeric data.
Statistical methods are used to analyze the data and
results are summarized in impersonal, objective
reports.
Qualitative Methods




Qualitative research methods enable researchers to
study social and cultural phenomena using textual,
rather than, numerical data.
Case study, observation, and ethnography are
considered forms of qualitative research.
Results are used to understand and explain social
phenomena in interpretive reports that reflect the
researchers’ constructions of the data.
As such, results are not usually considered
generalizable, but are often transferable in the sense
that readers will form their own constructions of what
is reported.
Empirical Research Design: Time




Time is an important factor in the design and
execution of research along both the experimental
and naturalistic continua.
There are two basic design classifications by time:
retrospective and prospective research.
Retrospective research examines phenomena after
the phenomena have occurred.
Prospective research searches for cause and effect
relationships or examines change in the present as
the event unfolds over time.
Retrospective Research
 Retrospective
experimental-type designs include those
that use chart extraction and are classified as passive
observation or correlational.
 A retrospective approach is useful when a reasonable
data set, such as medical records, is available, if the
researcher has limited time and funding resources,
and if random assignment is not appropriate.
 Although there is no manipulation of a phenomenon or
random assignment or control, the purpose is to
describe the occurrence of phenomenon by
Retrospective Research
o Examining
the relationships among variables; and
o Examining possible causative relationships.
 For example, Davidoff et al. (1991) conducted a
retrospective study to determine the efficacy of
rehabilitation programs to facility recovery after
acute stroke.
 They reviewed the medical records of 139 acute
stroke patients at admission, discharge, and oneyear follow-up from hospitalization.
Retrospective Research
 They



examined such information as neurologic status,
functional status, and the quantity and reasons of
outpatient physical and occupational therapy.
Another retrospective strategy is the life history
approach.
This approach to qualitative research uses the
interview or informant technique to gather
information or reminiscences about personal
experiences, feelings, and events.
This retelling and recasting of life events, feelings,
and experiences can be used as a basis from which
to interpret the sense of self as people age.
Prospective Research




There are two basic types of prospective designs:
cross-sectional and longitudinal.
In a cross-sectional study, the researcher examines a
phenomenon at one point in time.
There is only one single time frame in which data are
collected.
Longitudinal studies, on the other hand, involve data
collection over extended periods of time.
Cross-sectional Study



Suppose you wanted to describe attitudes toward
aging and how individuals of varying ages perceive
the aging process.
You could design a cross-sectional study that sampled
individuals in different age groups by conducting a
survey or interview at one point in time.
The analysis would involve a comparison among the
age groups on the dependent variables, attitudes
toward aging.
Longitudinal Studies



Longitudinal studies can be used to examine such
phenomena as the long-term effects of health care
programs or interventions, the natural course of
human development and adaptation, the trajectory
of illness, or the sequelae of various diagnoses.
There are three types of longitudinal studies: trend,
cohort, and panel studies.
Trend studies involve examining a general population
over time to see changes or trends that emerge as a
consequence of time.
Longitudinal Studies




Suppose you wanted to see if there were any trends
in the way in which different age groups perceived
the aging process over time.
You would interview individuals who fit into the age
brackets you are interested at one point in time, as in
the cross-sectional design.
Five years later, let us say, you would interview other
individuals who fit the same age brackets.
In this way, you would be able to compare and
contrast attitudes toward aging of persons 41 to 50
years old in 2000 with person of the same age
range in 2005.
Longitudinal Studies




Cohort studies involve examining a specific group or
one particular generational grouping as they change
over time.
In a study on attitudes toward aging, we would
examine aging perceptions of one age group at one
point in time, such as 2000.
Then, at a later point in time, let us say, 2005, we
would sample different individuals but those who are
from the same cohort as in the first data collection
effort.
The individuals in the sample would be five years
older.
Longitudinal Studies




Although we would be sampling from the same cohort
of individuals we would not be interviewing the same
individuals at both time periods.
Panel studies are another longitudinal design
strategy.
Similar to a cohort design, the same set of people
are studied over time.
For example, the subjects we interviewed in 2000
who were 41 to 50 years of age would be the same
individuals we would attempt to interview five years
later in 2005.
Research Variables




When conducting empirical research, the notion of what
constitutes a variable is important to understand.
Variables are concepts or constructs that change under
different circumstances rather than remain constant.
Variables don’t have to be numerical or quantitative; they
can be categorical, like gender.
In the behavioral sciences, examples of variables would
include:
o
o
o
stimulus characteristics (e.g., tone, intensity or frequency)
environmental conditions (background noise level)
speech behavior (rate of speech or number of nonfluencies)
Research Variables
o
o
o




language performance (MLU in a language sample)
hearing ability (speech reception threshold)
gender (male, female, intergender)
Variables can be categorized as independent or
dependent.
The behavior that the experimenter controls is
typically referred to as the independent variable.
Independent variables can be thought of as
conditions that cause changes in behavior (B).
In other words, the independent variable is the
antecedent (A) in the equation ABC.
Independent vs. Dependent Variables




The behavior that the experimenter measures is
referred to as the dependent variable.
Dependent variables can be seen as the behavior
that is changed.
In other words, the dependent variable is the
consequent (C).
For example, delay auditory feedback (the
independent variable) may cause a change in speech
rate (the dependent variable).
Independent vs. Dependent Variables



In experimental research, the examiner manipulates
the independent variable (while holding other
potential independent variables constant) to examine
the effect the manipulation of the independent
variable has on the dependent variable.
For example, MLU is sometimes used instead of
chronological age to classify children into groups that
vary in degree of language development.
In this case, MLU is the independent variable.
Independent vs. Dependent Variables


If, however, a researcher is looking at the effect of
manipulating the number of communicative partners
on children’s MLU, MLU becomes the dependent
variable.
The distinction between independent and dependent
variables is really a distinction based on the use of
variables rather than some inherent property of a
variable.
Active vs. Attribute Variables



A variable that can be manipulated is an active
variable.
The independent variable is an active variable
when it can be manipulated in some way by the
experimenter to see what effect it has on a
dependent variable.
For example, an experimenter can change the
intensity of a tone presented to a listener by
manipulating the hearing-level dial on an
audiometer.
Active vs. Attribute Variables



A variable that cannot be manipulated by a
researcher is an attribute variable.
Subject characteristics, such as age, gender,
intelligence, type of speech disorder, degree of
hearing loss, or history, have been manipulated by
nature, but they cannot be changed by the
experimenter.
Some variables may be either active or attribute
variables depending upon the circumstances of the
research or on how the researcher uses the variable.
Active vs. Attribute Variables
o
o
o
Anxiety is an example of a variable that can be
active or attribute.
Anxiety can be an attribute of subjects.
Anxiety can also be manipulated by the
experimenter by varying degrees to see what effect
the manipulation of anxiety has on some dependent
variable.
Continuous and Categorical Variables



A continuous variable is one that may be measured
along some continuum or dimension that reflects at
least the rank ordering of variable values.
The intensity of a tone, for example, can be
measured along a numerical continuum from low to
high values of sound pressure level.
Stuttering frequency can vary from zero nonfluencies
to a high number of nonfluencies.
Continuous and Categorical Variables




A categorical variable cannot be measured and can
only be categorized or named.
Tones could be presented to a listener binaurally or
monaurally.
Subjects could be classified as “stutterers” or
“nonstutterers.”
Continuous and categorical variables are displayed
differently graphically.
Continuous and Categorical Variables

Change in a dependent
variable as a function
of changes in a
continuous variable are
commonly displayed in
a line graph.
Continuous and Categorical Variables

Change in a dependent
variable as a function
of change in a
categorical variable is
customarily displayed
in a bar graph.