Transcript Slide 1

Tonight’s Quantitative Thinking
Presentation (Theme TBA Later)
“There are three kinds
of lies: Lies, d______ed
lies, and statistics.
”
—attributed to
Mark Twain
Michael Hansen
St. Albans School of Public Service
Washington, DC
June 21, 2007
My background . . .
• Math and music (piano, violin, viola)
• M.S., Applied Mathematics, UIUC
• 1986–1998: U.S. Government contractor,
primarily in the Pentagon
• 1998–present: STA math teacher
Icebreaker Question
• In what city was I born? (Hint: It is one of
the ten largest cities in the U.S.)
Your Backgrounds . . .
• Tell me something unusual that is not in
your capsule bio
Which of these subjects are you most
likely to use later in life?
Geometry
Algebra
Statistics
Calculus
67%
17%
11%
C
al
cu
lu
s
s
is
tic
St
at
br
a
lg
e
A
eo
m
et
ry
6%
G
1.
2.
3.
4.
Why are geometry, algebra, and
calculus taught in high school?
61%
22%
17%
i..
.
ct
s
pr
ov
gr
..
bj
e
er
s
Th
es
e
su
ch
M
at
h
te
a
es
oo
dn
ar
e
is
fe
Li
s
on
ly
kn
cr
ue
ow
s.
l.
0%
G
1. Goodness only knows.
2. Life is cruel.
3. Math teachers are
greedy and want full
employment.
4. These subjects provide
a chance to teach
abstract reasoning
skills.
mortar : brick :: glue :
stone
wood
water
oil
89%
l
0%
oi
at
er
w
oo
d
6%
w
on
e
6%
st
1.
2.
3.
4.
arachnid : spider :: marsupial :
lizard
kangaroo
fish
beetle
83%
17%
et
le
be
fis
ar
oo
ng
ka
h
0%
ar
d
0%
liz
1.
2.
3.
4.
What have we been doing here?
• Answer: Gathering statistics.
• What is a statistic?
• A number computed from data.
Which of these are examples of
statistics?
•
•
•
•
•
A batting average
A vote tally for a political candidate
A census (headcount) for a community
A person who is injured in a traffic accident
The statement, “Most Americans support our
troops in Iraq”
• The statement, “51% of the adult voters polled
stated that the war in Iraq has been a mistake”
• The statement, “51% of adult American voters
think the war in Iraq was a mistake”
•
•
•
•
•
Terms
Data = facts (plural); one fact is a datum
Statistic = a number computed from data
Parameter = a number that describes a population
Key idea: We use statistics to estimate parameters.
Bias = error in parameter estimation that is
systematic (i.e., tending to one side or the other)
• Sampling error = the inevitable result of trying to
estimate parameters with a sample that is smaller
than the entire population
• m.o.e. = an estimate of sampling error (m.o.e.
shrinks as sample size grows, assuming you have a
random sample)
• Note: Bias is avoidable, at least in theory. Sampling
error is not.
Fun Chart: Who wins this
election? (5.8 million votes cast)
Candidate A
Candidate B
0
1,000,000
2,000,000
3,000,000
Fun Chart: Who wins this
election? (5.8 million votes cast)
Candidate A
Candidate B
Statistics:
0
1,000,000
2,000,000
3,000,000
Because of errors in the tabulation process, the m.o.e. for each candidate is at
least several thousand votes, possibly as high as 60,000 votes.
The initial tally shows Candidate A with a lead of 537 votes over candidate B.
Who do you think won?
1. Candidate A, by 537
votes (no recount)
2. Candidate A,
maintaining a lead
even after the recount
3. Candidate B, pulling
ahead after the recount
4. Tie
44%
33%
17%
Ti
,m
ai
nt
di
ai
da
...
te
B
,p
ul
lin
g.
..
an
C
di
d
an
C
C
an
di
d
at
e
at
e
A
A
,b
y
53
7
...
e
6%
Tonight’s Theme
(Try to guess it from these examples.)
• Space aliens stare at Earth through powerful telescopes. They observe that most
of the people buying diet soda are a bit overweight, to put it mildly. Conclusion:
Diet soda makes people fat.
• A teacher enforces discipline in the classroom by deducting one point for each
minor infraction (burping, tardiness, etc.) and tells a student who has lost several
points, “Your actions have lowered your grade from A minus to B plus.”
• Terrorists hijack airplanes and crash them into buildings, killing several
thousand Americans. Spokesmen for the terrorists, as well as a number of
international commentators, say that U.S. government policies are to blame.
• A Roman army unit performs poorly in battle, with several of the men deserting
in fear. After the battle, the commander decimates the troops (i.e., kills every
tenth man present). The troops learn that if they desert, they will be sealing a
death warrant for their friends still in the unit.
• A village under Nazi occupation revolts. In reprisal, the Nazis murder nearly all
the villagers, but they also lay waste to several neighboring villages. The revolts
quickly cease.
• A sheriff running for re-election proudly states, “During my four years in office,
the violent crime rate has dropped by 28%. Vote for me to continue the
progress!”
What is the theme?
1. Space aliens are
not very bright
2. Correlation does
not imply
causation
3. Distortion of cause
and effect
4. How to lie with
statistics
65%
29%
6%
st
at
is
tic
a.
.
us
e
ith
ca
w
of
H
ow
to
lie
or
tio
n
D
is
t
at
io
or
re
l
C
Sp
ac
e
al
ie
n
ns
do
ar
e
es
no
t.
no
t..
..
.
s
0%
Cause and Effect
• A most interesting subject: Even babies are
fascinated by it!
• All public policy decisions hinge on cause and
effect
• Warning: Remember the “Law of Unintended
Consequences” (examples of refrigerator subsidies,
Australian rabbit invasion, AFDC)
• Difficulties:
– Mathematics has nothing to say on the subject.
– We must turn to the relatively new science of statistics.
– There is only one way to prove cause and effect, and the
conditions are rarely met.
– But . . . that does not stop politicians, journalists, and
activists from asserting cause-and-effect relationships.
Real-World Questions to Ponder
• Do CO2 and other greenhouse gases cause global
warming?
• What fraction of global warming is caused by human
activities (e.g., burning of fossil fuels)?
• Will changing our behaviors cause a reduction or
reversal of global warming?
• What government policies on current and alternative
fuels will cause changes in people’s behavior?
• To what degree do U.S. energy policies cause changes
in other countries’ economies or behaviors?
Making it Real—Cause and Effect in the News
• Do mercury-based preservatives in childhood immunizations
cause autism? Many parents whose children suddenly became
autistic shortly after having an immunization are convinced
that this is so.
• Did Vioxx cause heart attacks and strokes? If so, should Merck
be required to pay damages to patients who suffered a heart
attack or stroke after taking Vioxx?
• Does second-hand cigarette smoke cause lung cancer or other
diseases? If so, should smoking be banned in all indoor
locations?
• Do American high school classrooms cause boys to lag behind
girls academically? If so, should curricula and teaching styles
be revamped? What would be the effects of doing that?
• Do silicone implants cause lupus and other autoimmune
disorders in women?
• Did Zicam cause people to lose their senses of taste and smell?
• Do cell phones cause automobile accidents?
Some Common Fallacies
•
•
•
•
Anecdotal data
Emotional appeals or ad hominem attacks
COI, appearance of COI, or accusations of COI
“Post hoc, ergo propter hoc” (roughly translated:
“after this, therefore caused by this”)
• Similar: Correlation confused with causation
• Extrapolation (i.e., assuming trends will continue)
• Overanalyzing time series of uncontrolled
systems (e.g., trying to predict the stock market by
using “technical analysis”)
Are there any people who know
what they’re doing?
• Yes, a few.
• Everyone should take a course in statistics.
• A statistics course is one of the few courses
where you are unlikely to study any actual
statistics! (You will study the science and
practice of statistics.) Actual statistics are seen
in nearly all other courses: history, biology,
sociology, etc.
“Of the 1300 randomly chosen adult American voters who were
polled, 38% were satisfied with President George W. Bush’s
performance in office. The margin of error is plus or minus 3%.”
35%
ow
n
te
ve
Id
o
no
tk
no
no
o
Id
...
tt
h.
..
w
w
te
qu
i
w
no
kn
el
l
w
ha
w
ha
t..
ha
t
w
ct
ly
ex
a
Ik
no
Ik
...
6% 6%
w
1. I know exactly what the
statement means.
2. I know quite well what the
statement means.
3. I do not know what the
statement means.
4. I do not even know whether
or not I know what the
statement means.
53%
“Of the 1300 randomly chosen adult American voters who were
polled, 38% were satisfied with President George W. Bush’s
performance in office. The margin of error is plus or minus 3%.”
Exactly what does this statement mean?
Th
e
is
tic
tr
ue
st
at
is
ra
m
et
er
pa
tr
ue
Th
e
e
...
is
...
...
is
tic
st
at
is
tr
ue
pa
ra
m
et
er
is
...
0%
Th
e
Th
18%
12%
tr
ue
1. The true parameter is
between 35% and 41%.
2. The true statistic is between
35% and 41%.
3. The true parameter is
probably between 35% and
41%.
4. The true statistic is probably
between 35% and 41%.
71%
On the previous slide, why did the correct answer
involve the word “probably”?
82%
to
o.
..
si
ze
nd
o
e
m
pl
sa
la
ck
of
ra
sa
w
as
m
ne
in
g
ss
bi
in
.
..
as
er
ro
r
6% 6% 6%
m
pl
1. sampling error
2. bias
3. lack of randomness in the
sample
4. sample size was too small
for a believable survey of
the entire nation
If you are making public policy decisions, why should you
take nearly all news reporting (as opposed to news analysis
or commentary) with a grain of salt?
65%
24%
12%
e
ec
do
ta
ld
at
a
bi
as
m
re
p.
..
ed
ia
an
cc
m
os
ta
sa
m
pl
ur
at
e
in
g
er
ro
r
0%
th
1. sampling error
2. the most accurate
reporting is usually
internal to the
government, not
coming from the
news media
3. media bias
4. anecdotal data
How to Talk Back to a Statistic
Source: Darrell Huff ’s classic 1950’s book, How to
Lie With Statistics
• Who says so? (Beware of COI.)
• How does he or she know? (What methods were used
to compute the statistic? Many numbers are simply
unknowable in a practical sense.)
• What’s missing? (Ask what the m.o.e. is, what
assumptions were used, what time period was used,
and whether there was a control group.)
• Did someone change the subject? (Beware of
“semiattached data,” “gee-whiz” graphs, and
extrapolation.)
• Does it make sense? Modern-day buzzterm: FACE
VALIDITY.
Regarding Face Validity . . .
• Use your common sense and read critically. Even reputable
sources contain errors.
• Compute ratios or per capita values. (For any national
budget number, simply take the number of billions and
multiply by 3 or 4. Example: A $50 billion program is
costing each adult in the country about $200.)
• Excerpt from The Washington Post Magazine on April 2, 2006
(posted on their website):
Adult education is thriving nationwide, with more than 92
million adults taking college classes. At the nearly 70 twoand four-year colleges in the Washington area, an estimated
175,000 adults are enrolled, 40 percent of them on a parttime basis.
• Which statistic in this excerpt lacks face validity?
What is a statistic?
...
th
e
ed
a
pe
rs
on
w
ho
is
pu
t
co
m
be
r
nu
m
vi
...
t
fa
c
a
cr
ib
..
td
es
a
be
r
nu
m
a
6%
0% 0%
th
a
1. a number that describes a
population
2. a fact
3. a number computed from
data
4. a person who is the
victim of crime or an
accident
94%
What is a parameter?
ns
ta
n
...
ed
co
pu
t
ab
l
e
co
m
an
ad
j
us
t
be
r
nu
m
..
t
fa
c
a
cr
ib
..
td
es
a
be
r
nu
m
a
6%
0% 0%
th
a
1. a number that describes a
population
2. a fact
3. a number computed from
data
4. an adjustable constant, or
a boundary condition for
a problem
94%
What is the only way to establish
cause and effect?
82%
1. a careful observational study
2. a careful observational study
with a sufficiently large
sample size and freedom
from bias
3. an experiment
4. a controlled experiment
18%
en
t
pe
d
ex
ex
p
co
nt
ro
lle
a
an
rim
er
im
en
t
...
tio
na
er
va
ob
s
re
fu
l
ca
a
a
ca
re
fu
l
ob
s
er
va
tio
na
...
0% 0%
On a scale of 1 to 7, please rate how you feel about this statement: “I
am planning to take a statistics course sometime within the next three
years.” (Mark 7 if you have already taken a statistics course in high
school, e.g., AP Statistics.)
Strongly Disagree
Disagree
Somewhat Disagree
Neutral
Somewhat Agree
Agree
Strongly Agree
24%
18%
12%
6%
6%
So
ly
Di
s
ag
re
e
m
D
is
ew
ag
ha
t D ree
is
ag
re
e
So
N
m
eu
ew
tra
ha
l
tA
gr
ee
St
A
ro
gr
ng
ee
ly
Ag
re
e
0%
St
ro
ng
1.
2.
3.
4.
5.
6.
7.
35%
Q and A
Thank you for your time and attention!
Michael Hansen
(e-mail: modd “at sign” modd.net)
St. Albans School of Public Service
Washington, DC
June 21, 2007