Critical Thinking 101 – Statistics and Deception

Gail P. Taylor, Ph.D.

MBRS-RISE Program, UTSA

References:

 Asking the Right Questions – A guide to critical thinking. N. Neil Browne and Stuart Keeley. Prentice Hall, 2000.

What is Critical Thinking?

 An awareness of interrelated critical questions that you employ at appropriate times  Results in systemic, active evaluation of what you read and hear  Influences how you react to information  Accept?

 Reject?

 Withhold judgment  Must also critically evaluate your OWN conclusions…

The Way of the World

 World- There will always be topics that are up for debate, with “reasonable” experts on both sides  Laboratory- There will often be more than one interpretation for experimental results observed  Some “Facts” are solid (measurements)  Some more open to interpretation  Social/Psych science is particular vulnerable

Approach to Information

 Sponge  Absorb everything…but no value judgment  Panning for Gold  Ask about speaker motives  Actively identify possible problems with what is being said?

 Evaluate identified problems?

 Form your own conclusions…

Beware Emotional Involvement with Opinions…  Everyone has pre-existing opinions  The older the opinion, the more likely you didn’t examine it  May have great emotional attachment to un-examined opinions  Do not “fall in love” with your ideas or opinions, so that they blind you to the truth…

In Science, Critical Thinking is used in your own Research…  The Scientific Method itself involves critical thinking        Observe problem/phenomenon/conceive ideas Make Predictions/Develop a hypothesis Devise a test/Formulate experiment Carry out experiments Draw conclusions from results Reject or support hypothesis Revamp experiment and continue onward…    VITAL for a scientist.

Impressive for a science student Need to take time to read and think

…And When You look at Others’ Research Results  Are the ideas supported by the literature?

 Are the techniques correct?

 Was data analyzed correctly?

 Are their conclusions supported by the data?

The “Right” Questions

           What is the issue/problem?

What are the conclusions?

What is the supporting evidence or reasons?

Are any significant words or phrases ambiguous?

How good is the evidence?

Are the statistics deceptive?

Are there hidden assumptions?

Are there any fallacies in the reasoning?

What significant information is omitted?

Are there rival causes/theories?

Are the conclusions reasonable?

How to Find Issues

What is the Issue?

 Given: They are trying to influence your beliefs  What question (or controversy) are they trying to influence your beliefs ABOUT?

 In the “World” often about personal agendas  In Science, can be to persuade you to the validity of their results and viewpoint/theory about a field.

Where and How to Find the Issue:  Often very obvious  Generally found at beginning  Sometimes only implied  It’s the question being addressed  Inferred through conclusion  Inferred through examination of author’s associations…  “They are trying to influence my view of…What??”

Explicit Issue:

 The question I’m raising is: Why must we have speed limits on our highways?

Implied Issue

By now, more than two years after voters overwhelmingly approved the lottery, it has been proven that the game is not a success; in fact, it can be considered a failure… What is the author reacting to?

 Issue: What is the impact of the Lottery?

Two Types of Issues

 Sometimes difficult to discern  Descriptive (how the world IS):  Usually see in science  What is the molecular basis for hibernation?

 Who decides to reduce the drinking age?

 Do families with pets argue less frequently?

 Prescriptive (How the world should BE)  Ethical/Moral  Should we reduce the drinking age?

 What ought to be done about welfare?

Conclusions

 Given: They are trying to influence your beliefs  “What are they trying to get me to conclude?”

Finding Conclusions

 Often at the end (but not always)  Often accompanied by specific phrases

Finding a Conclusion

          We conclude that Results indicate Consequently Hence Therefore Thus In short It follows that Shows that Indicates that          Suggests that It should be clear that We may deduce that Points to The point I’m trying to make is The most obvious explanation It is highly probably that Proves that The truth of the matter is

Conclusion?

By now, more than two years after voters overwhelmingly approved the lottery, it has been proven that the game is not a success; in fact, it can be considered a failure…  Conclusion: The lottery is not a success and is in fact a failure

Example Divorce is on the increase, and we’re worried. Finally, Psychologists have identified a key cause of divorce. Noticing that certain families have multiple divorces, they found that inherited genes play a major role in causing divorces. Although overall divorce rate is 20 percent, psychologists have discovered that a twin has a 45 percent rate of divorce if the identical twin has already experienced a divorce. Additional confirming evidence for genes as a primary cause stems from looking a the divorce rate of twin’s parents. If their parents have divorced, each twin has a 10 percent higher risk of divorce   Issue:  What is the cause of Divorce?

Conclusion:  Inherited genes are a major cause

Quick Talk about Stats! Statistics 101

 Statistics- Evidence expressed as numbers  Can and often DO lie…esp. in popular press  Can also seem counter-intuitive to people…

Humans, Risk and Stats…

 Statistics may seem counter intuitive…  Automobile Deaths:  1.7 deaths per 100 million veh. miles  Population risk per year: 1 in 6,300  Air Carrier Deaths:  0.7 deaths per 100 million aircraft miles  0.19 deaths per million aircraft departures  Population risk per year: 1 in 1,568,000 http://hazmat.dot.gov/riskcompare.htm

General Strategies to Detect Statistical Deception…

 How were the statistics obtained?

 What are the Motivations of quoter?

 What does “average” mean?

 Are the conclusions supported by the stats?

 Is all the information needed for evaluation there?

How Obtained – Unknowable Statistics

 There are obstacles to getting accurate statistics on some topics…  Physical barriers to obtaining information  Failure to report events  Improper sampling  Deception  “Educated Guesses”  Have to analyze what the person stating the stats has to gain….

Is there a truly a perfect way to find out the following?

 Number of Aids cases?

 Number of Abortions?

 How many people shoplift?

 Have affairs?

 Have a mental disorder?

 Use cocaine?

 Commit white-collar crimes?

 Are homeless?

 Beat their wives?

Biased Statistics – Bad Initial Sampling…

 We randomly polled 500 people from California, Florida, and Maine and have concluded that 34% of Americans prefer to live near the beach  We went to a fundamentalist Mosque to find out what Muslims think…

Averages:

 Americans are better off than ever; the average salary of an American worker is now $28,400.  The average pollution of air by factories is now well below the dangerous level.

Beware of “Average”

 Mean: Add all values and divide by n  Median: List all values, low to high, then take center (half above, half below)  Mode: Value that appears most frequently

Average salary?

 Mean- very high salaries of few skew the average, badly. Use if you want to inflate average salary  Median & Mode – not affected

Average Pollution

 What happens if only a few factories are very very horrible polluters, at above an extremely harmful level?

 Median & Mode – No danger  Mean? - Very bad, on average.

 Chrome Plating shops…

With Averages also Need Range and Distribution…

  Range – gap between smallest and largest Distribution – How frequently each value occurs.  Ex: Average mercury content in fish is very low…  What happens if 10 percent have dangerous levels?  On average, only 0.2 percent of landmines in the 25 fields were still active…

Averages, Range and Distribution….

America is not overcrowded. Nationally, we have fewer than 60 people per square mile, a population density lower than that of most other countries…  Some areas really ARE crowded.

 Some are mostly unpopulated…

When an average is presented….

 Check which type of Average you’re seeing…  See if the range and distribution might be important…

Prove One Thing, Conclude Another….

A car dealer considered a particular car a big success because only 5 out of 100 buyers who bought the car had complained to the dealership about its performance. “When 95 percent of buyers are pleased,” the salesman says, “then that’s a darn good car…” • Assumptions?

• All complaints came to dealer • Non-complainers were pleased

How would you get this conclusion?

 Conclusion: Car was a high quality success…  Was this supported?

 Blind yourself to communicators statistics  How would you actually test this?

 Random sample of many of this car’s buyers and ask them how pleased there are with the car.

Second Strategy, What Do Stats Say?

 Look at Stats alone…  Only 5 out of 100 buyers who bought the car had complained to the dealership about its performance.

 What can you conclude from this?

95% didn’t have a complaint that required a dealer visit. Might not like the car, but not enough to complain…

Example

 Almost one-fourth of children are arsonists. Children all over the country were surveyed. Of the 90 who replied to the survey, 24 percent said that they knew of instances in which children set fires….

 Problem??

Example II

 Women are better drivers than men, as proven by the fact that of men involved in accidents, 23 percent had been drinking compared to 9.6% of women…  Problem?

Deception by Omission…

 Ask yourself, “What further information do you need before you can judge the impact of the statistics?”

Evaluate:

 Fizz aspirin works 50 percent faster  Than what????

 Funding for AIDs research is more than adequate. Last year the government spent over 1.2 billion on AIDS research.

 How does this compare  Over time?

 Other diseases?

 Total budget?

 College degrees pay off. A recent survey found that workers with a bachelor’s degree were earning an average of $35,000 per year in the spring of 2000.  Where is the comparison of equally intelligent non college students? What type of average?

Evaluate:

 A crime wave has hit our city. Homicides have increased by 67 percent in the last year…  Problem- What are the absolute numbers?

 3 to 5, versus 300 to 500…

Conclusions

 Ask – How does the author or speaker know? What are their motivations?

 How did they choose their sample?

 Be curious about types of averages  Blind yourself to the stats and see what stats you would want to support conclusions. Do they match?

 Look only at Stats and form your own conclusion. Does it match?

 What information is missing? Take care with misleading numbers and percentages and for missing comparisons.