Transcript Research Methods for Counselors

Research Methods for
Counselors
COUN 597
University of Saint Joseph
Class # 6
Copyright © 2015 by R. Halstead. All rights reserved.
Class Objectives
Salkind Chapter 8 – Probability
Salkind Chapter 9 – Statistical Significance
Probability
The analysis of data that is generated from an
experiment usually involves calculating the
resulting means of the experimental and control
groups and then test those means to establish
whether or not they are “significantly different.”
The term “significantly different” is used to
denote, with a specified degree of confidence,
that any difference between the means did not
occur by chance.
Probability and the
Normal Curve
Any time we make reference to chance, we are
automatically implying the application of
probability.
The basic tool that helps us work effectively with
probability is the normal curve.
The Normal Curve
Characteristics
Mean, Median, and Mode are equal
Perfectly symmetrical in shape
The tails of the curve approach the horizontal
axis but never touch it
How do we establish a sample that creates the
normal curve?
Randomly select a large enough sample (the
larger the better but 30 is thought a minimum)
Standard Scores
Standard Scores are simply scores that have been
standardized so as to make them consistent with
the Standard Deviation of a distribution.
The value of setting up a standard score is that it
allows us to compare scores from different test or
measure on the same scale.
One type of Standard Score is the Z Score.
The Z Score has a mean of zero and and a
standard deviation of one.
Converting a Test Score into a
Standard Scores
The formula for this conversion is as follows:
(X - X)
Z =
s
Z is the z score
X is the individual score
X is the mean of the distribution
s is the distribution standard deviation
The Z Score
Beside the z score allowing us to compare scores
from different tests against one another on the
same scale, we can also determine what percent
of scores we would expect to find above and
below that score.
Logic would tell us, given that the normal curve
is really about probability, that we can also
determine the probability of a certain score
occurring at or above (or below) any z score.
Z Scores and Where this is
Heading
To date, we have established three points.
First, we know that conducting research is
about establishing a reasonable empirical
conclusion.
Second, empirical knowing is established by a
process of examining the nature of differences.
Third, any differences (or lack there of) could
be the result of random chance.
Now lets move one more step forward.
Z Scores and Where this is
Heading
If we want to be sure that the difference between
two observations, say the average levels of
depression between a cognitive therapy treatment
group and a no treatment control group after 10
weeks of treatment, means something, we want to
be relatively certain that any difference is due to
the treatment and not due to chance.
The z score is the first step toward actually being
able to empirically determine that type of
reasonably acceptable conclusion.
The Concept of Significance
As we have discussed in previous classes, when
we engage in a process of conducting quantitative
research we use a specific method for
establishing a certain form of knowing.
This method (the scientific method) utilizes a
process of comparison.
Often what the researcher is comparing is one
group against another group. When a difference is
found the research draws some conclusion.
The Concept of Significance
Given that the researcher’s conclusions are going to
be based on differences regarding some variable of
interest, there are two major elements that must be
taken into account.
First, we want to make sure that the difference found
is due to the manipulation of the independent
variable and not some other factor or factors.
The researcher addresses this by trying to control
for confounding variables and minimizing
sampling error and sampling bias.
The Concept of Significance
Second, once we have controlled, as much as
possible, for confounding factors we want to make
sure that the difference found is not due to those
elements that we can not control for and are
introduced by random chance.
Let’s say that you find a difference between your
experimental and control group. How can you be
certain that the resulting difference is due to the
manipulation of the independent variable and not
due to random chance?
The Concept of Significance
The answer to that question leads us to a special
type of difference that a research seeks to find. That
is the “Significant Difference” between groups.
Another way to express this is to say that the
difference between the groups was found to be . . .
“Statistically Significant”
Statistical Significance:
Researchers as Chickens
Okay - that title is a bit of an over statement but
in essence dealing with levels of statistical
significance is about the risk of being wrong.
Remember you can never be 100% certain of any
research outcome so what the researcher says is,
“I am willing to state that my hypothesis holds
and differences are due to my manipulation of the
independent variable 95% of the time - but, to be
on the safe side, 5% of the time this result could
be due to other factors.”
Statistical Significance and
Research Hypotheses
So you set a certain level of risk with which you
are willing to live - p < .05 (less than 5%). This
says that the probability (p) that any difference
between the experimental and control group is
due to some factor other than the manipulation of
the independent variable is less than 5%.
In testing the null hypothesis, if your test results
are shown to fall within that p < .05 region of the
normal curve you conclude that the difference is
due to the I.V. and not due to other factors.
Statistical Significance and
Being Wrong
Step 1 - Establish the study you wish to conduct.
Step 2 - Write your null hypothesis
Step 3 - Conduct your study and collect data
Step 4 - Analysis your data by testing for
differences accepting the a certain level of risk
with which you are willing to live - say p < .05
Step 5 - Publish your results and become famous
in the field of counseling
Piece of cake - right? No so fast researcher!!
Statistical Significance and
Being Wrong
Remember that risk you were willing to live
with?
Well sometimes it is not so easy - sometimes
your results will really be due to something other
than your I.V. even though you have stated that
the your I.V. was responsible for the difference
between the experimental and control groups. In
the world of research this is called . . .
Being Wrong!!
Statistical Significance and
Being Wrong
When we deal with hypotheses we have to make a
decision. Either the hypothesis is true or it is false.
There are two ways we can get into trouble.
The first kind of trouble is when we say the
hypothesis is false (reject the null hypothesis)
when it is true - Type I Error.
The second kind of trouble is when we say the
hypothesis is true (accept the null hypothesis)
when it is false - Type II Error.
Decisions about the Null
Hypothesis and Type I & II Error
Accept
Reject
Accepted the
Null Hypothesis Null Hypothesis
is True
and it is true
Reject the
Null Hypothesis
and it is true Type I Error
Accepted the
Null Hypothesis Null Hypothesis
is False
and it is false Type II Error
Rejected the
Null Hypothesis
and it is false
Reducing the Likelihood of
Type I and Type II Error
You have some control over Type I Error in that it
is based on the p level that you set for your test.
Type II Error is a bit more difficult to deal with in
that it results most often from the samples not
being an accurate representation of the population
from which they were drawn.
Reducing Type II Error is best accomplished by
establishing samples that are large enough so as to
increase confidence that the sample represents the
population it is supposed to represent.
Significant but Meaningful?
A primary purpose of conducting research is to learn
something about the world. The value in conducting
research is the increase knowledge and sharing it
with others.
When the analysis of your data results in statistically
significant differences you must explain the
meaning of those results.
Correlation Example - you have 45 cases how large
of a correlation would you need for a one tailed test
at a p level of < .05? Significant but meaningful?
Significant but Meaningful?
Significant relationship? Yes. Meaningfully
different? Probably not.
Remember the idea behind conducting research is
not to show that you were right about the hypothesis
but rather to learn something useful for practice.
Useful Phrases for Reporting
Research - NOT
“It has long been known. . .”
“A definite trend is evident.”
“Correct within an order of
magnitude.”
“Three of the samples were
chosen for a detailed study.”
I didn’t look up
any sources
Data are practically
meaningless
Wrong
The results of the
others didn’t
make any sense
Inferential Statistics
Up to this point in the semester we have used
descriptive statistics to understand the
characteristics of a sample with which we were
working.
We are now about to go boldly into the
fascinating world of inferential statistics.
Inferential statistics are used to allows the
researcher to say something about the population
given what can be known about the sample.
The Logic of Inference
The researcher randomly selects a sample from a
population and assigned participants to two
groups that are found not to differ prior to the
start of the research study.
One group gets some form of treatment and one
group gets another form of treatment.
After ten weeks of treatment the researcher
administers a test to each group and computes
and compares the means for each group.
The Logic of Inference
A conclusion is reached about the effectiveness
of the two treatments relative to each other.
The researcher infers that the same conclusion
would also hold for the entire population that the
sample represents.
Thus the researcher can report these findings to
the community of professional counselors and
client’s lives are made better due to a discovery
of a more effective treatment. Pretty neet!!
Statistical Tests for Finding
Differences Between Groups
As the chart on page 152/183/173 of your Salkind
text suggests there are a variety of tests available
to the researcher.
That chart is a good one to keep handy in that it
offers a quick method for helping to determine
which statistical test is appropriate for a given
testing situation.
General Steps in Applying
Tests of Statistical Significance
1. A statement of the Null Hypothesis.
The assumption that, given no other
information, all will balance out equally.
2. Set some level of risk (significance) by which
you will be willing to accept or reject the
hypothesis.
Remember that the risk you are taking here is
that you will be wrong (Type I Error).
General Steps in Applying
Tests of Statistical Significance
3. Select the appropriate test statistic.
Each research situation requires a specific test
for the job. We will learn more about that in
the weeks to come.
4. Compute the test statistic to arrive at the
obtained value for the test.
5. Determine the value needed for rejecting the
null hypothesis using the appropriate table of
critical values for the statistic you are using.
General Steps in Applying
Tests of Statistical Significance
6. Compare the obtained value to the critical
value.
If the obtained value is more extreme than the
critical value - reject the null hypothesis.
If the obtained value is less extreme than the
critical value - accept the null hypothesis.