Beth Chance Allan Rossman

Overview

 What do want students to know and do at the end of the course  Our dream content   Top ten essentials No client disciplines   What would we cut to have time to get there   Assumptions about current content in many courses Reality vs. fantasy  Are we there yet?

 Example assessment items JSM 2010 2



#1 Understand the statistical process of investigation

Repeatedly experience the process as a whole 1. Formulate research question 2. Collect data 3. Examine the data 4. Draw inferences from the data 5. Communicate the results JSM 2010 3

#1 So what to cut?

 Compartmentalizing the topics in the course  Data analysis, data collection, statistical inference  Instead: one categorical variable, compare two groups on quantitative response…  Some specific techniques  Example? Chi-square, ANOVA, regression  Possible out of class explorations JSM 2010 4



#2 Describe how to collect relevant data to answer research question

Research question vs. variable  Do the data answer the question   Example: Songs about the heart “Worry questions” >> Make sure students have an opportunity to write their own research questions and to critique measurement/data collection methods JSM 2010 5

#2 So what to cut?

 Ordinal, nominal, interval, ratio scales  Specifics of different sampling methods (cluster, stratified) and experimental designs  Though do make sure they realize not everything is an SRS or CRD  Acronyms!

 Short-hand terminology (e.g., sampling distributions) and symbols (e.g., Ho/Ha) JSM 2010 6

#2 Assessment question

 Pose a research question of interest to you that involves comparing two groups (but not one we discussed this quarter),  Identify observational units, explanatory and response variable(s),   Describe a detailed plan to collect data to investigate this question  Be sure to provide a detailed enough plan that someone else could carry out the actual data collection.

Explain whether (and why) your plan will involve random sampling and/or random assignment, or neither.

JSM 2010 7



#3 Determine scope of conclusions based on data collection methods

Random sampling: generalize to population  Random assignment: cause/effect between explanatory and response variables  Some studies use only one, some (few) use both, some (many) use neither >> Get students in habit of always commenting on both of these issues whenever they summarize the conclusions of a study.

JSM 2010 8

#3 So what to cut?

 Nothing; this point is too important  Move: Data collection issues to beginning of course, descriptive analysis of bivariate quantitative data to end of course  Students can discuss confounding variables in context of observational studies JSM 2010 9

#3 Assessment question

  Students using cursive writing on the essay portion of the SAT in 2005-06 scored significantly higher, on average, than those who used printed block letters.

 Can you conclude that cursive writing causes higher scores? Explain.

Different study: Identical essays were given to graders, some with cursive writing and some with printed block letters. Those with cursive writing scored significantly higher.

 Can you conclude that cursive writing causes higher scores? Explain.

JSM 2010 10



#4 Appreciate value/necessity of graphing data

Always start with a graph  Explain what see  Example: number of letters memorized  Make sure statements/conclusions about the data follow from the graph  Sometimes the graph is enough!

JSM 2010 11

#4 So what to cut?

 Pie charts  Choice of histogram bin width  But use technology explore different choices   Normal probability plots Stemplots…  Boxplots!!

JSM 2010 12

#4 Assessment question

 Did distribution of inter-eruption times of Old Faithful change between 1978 and 2003?   If so, how?

How are changes favorable for tourists?

  How are changes less favorable for tourists?

What other interesting features are apparent, have changed?

JSM 2010 13

#5 Use proportional thinking

 Especially important with categorical data, two-way tables  Conditional proportions   Proportion vs. percentage vs. percentage change vs. baseline risk vs. relative risk Don’t need equal sample sizes to compare proportions or averages  Summary already takes sample size into account to produce a “fair” comparison JSM 2010 14

#5 So what to cut?

 Formal probability rules, counting rules   Instead use two-way tables of counts, proportions Bayes’ rule  Simpson’s paradox JSM 2010 15

#5 Assessment question

 Data from murder trial of nurse Kristen Gilbert: Death occurred on shift Death did not occur on shift Gilbert working on shift 40 217 Gilbert not working on shift 34 1350  Of the 74 shifts with a death, 40 (54.1%) were Gilbert shifts, not significantly more than half.

 Is this a reasonable calculation to perform here, to assess the evidence against Gilbert? Explain. If not, perform a more relevant calculation and explain why it’s more relevant.

JSM 2010 16

#6 Develop distributional thinking

 Conjecture how a variable will behave  Not everything follows a normal distribution  Example: Matching variables to graphs (ala

ABS

)  Appreciate the nature of variability  Think in terms of the distribution as an “aggregate”  Don’t let one value (data value or summary statistic) drive a conclusion  Focus on tendency, effects of outliers JSM 2010 17

#6 So what to cut?

 Mode  Relative frequency distributions   Cumulative distributions 1.5

× IQR criterion for outliers  Details on calculating mean and median  Have to start making students responsible for having seen this before JSM 2010 18

#6 Assessment question

 Which would have more variability: ages of customers at McDonald’s near freeway or ages of customers at snack bar on campus? Explain.

JSM 2010 19

#6 Assessment question

 Are pamphlets containing information for cancer patients written at an appropriate level that cancer patients can understand?

 Analyze these data to address the research question. Summarize and explain your conclusions.

JSM 2010 20



#7 Consider variability in data when making comparisons

Comparing a particular outcome to a constant  Comparing outcomes in two different groups  Standardization can be a special case  Using a measure of variability to produce “ruler” for which we judge distances  Standard deviation (

z

-score)  Box lengths… JSM 2010 21

#7 So what to cut?

 Calculation of standard deviation by hand  Short-cut calculation formulas (SD, correlation)  ANOVA table calculations  Linear transformations on summary statistics JSM 2010 22

#7 Assessment question

Traffic Deaths year   Sketch a graph of data from 1950-1960 where the change observed between 1955 and 1956 would be considered noteworthy.

Now sketch a graph where the change observed between 1955 and 1956 would not be considered noteworthy.

JSM 2010 23



#8 Consider variation of statistics when making comparisons

Averages vary less than individual values  Less and less with larger and larger samples  Larger samples give more precise estimates  Precision must be considered when making conclusions  Example: Three coin flips is not enough to decide whether a coin is fair JSM 2010 24

#8 So what to cut

 Rules for means and variances   / 

n

 Central Limit Theorem  Instead use simulations, graphs JSM 2010 25

#8 Assessment question

 In a rodeo roping contest, a contestant’s score is the average of two times. Explain why it is more fair to use this combination of two scores instead of relying only on one score.

JSM 2010 26

#9 Understand the logic of inference

 When can “chance” be eliminated as a plausible explanation?  Consider chance variability due to random sampling or random assignment  Strength of evidence vs. proof  Cobb (2007) argued that the reasoning process of statistical significance can best be introduced via simulation of randomization tests rather than normal-based models  “What if” distribution JSM 2010 27

#9 So what to cut?

 Rejection region approaches  Tables of probability distributions  Randomization approach does not require probability distributions   Even with traditional tests, technology can calculate p-values, critical values But still focus on well labeled sketches of “what if” distributions  Technical conditions  20-100% of specific (parametric) procedures JSM 2010 28

#9 Assessment question



MythBusters

: Is yawning contagious?

Subject yawned Subject did not yawn Total Yawn seed planted 10 24 34 10/34  29% Yawn seed not planted 4 12 4/16 16  25% Total 14 36 50  Was

MythBusters

justified in concluding that the data provide strong evidence that yawning is contagious?  Conduct your own analysis  Explain reasoning process behind your conclusion JSM 2010 29

#10 Consider margin of error

   Importance of interval estimate not only a point estimate  More than simply assessing statistical significance  Estimate + 2 SE Focus on idea of interval of plausible values  Understand what parameter is being estimated Issues that do/do not affect margin of error  Random sampling  Sample size  Population size JSM 2010 30

#10 So what to cut?

 Solving algebraically for sample size  Any level other than 95% confidence  Any multiplier other than 2!

 Interpretation of “confidence level” JSM 2010 31

#10 Assessment question

 Suppose you want to estimate the proportion of the over 305,000,000 Americans who prefer cats to dogs within a 3% margin-of error. Approximately what sample size would you need with a random sample?

10 1,000 100,000 1,000,000 10,000,000 JSM 2010 32

#1 Assessment question

     What type of study was this? Advantages and disadvantages?

What graph could you examine to summarize these data?

What is meant by “a 16 percent decreased risk of death”?

What does it mean for the average life expectancy to be “significantly” longer?

Is this an appropriate headline? Explain.

JSM 2010 33

Conclusions

 Fun to start from ground zero   What is your bare minimum of essential content?

Make sure “stat methods” courses don’t prevent “stat literacy”  Take advantage of computer/calculator power  Emphasize interpretation over calculation  Assess what you value JSM 2010 34

Questions?

 Allan Rossman [email protected]

 Beth Chance [email protected]

 http://www.rossmanchance.com/jsm2010.ppt

JSM 2010 35