Looking at data: distributions

Displaying distributions with graphs

IPS section 1.1

© 2006 W.H. Freeman and Company (authored by Brigitte Baldi, University of California-Irvine; adapted by Jim Brumbaugh-Smith, Manchester College)

Objectives

Displaying distributions with graphs

 Recognize numerical vs. categorical data  Construct graphs representing a distribution of numerical data  Histograms  Stemplots  Describe overall patterns in numerical data  Identify exceptions to overall patterns  Discuss pros and cons of histograms vs. stemplots

Terminology

      Individual (or “observation”) Variable   numerical (or “quantitative”) categorical (or “qualitative) Value Frequency  absolute  relative Frequency table Distribution

Terminology (cont’d)

       Graphs for quantitative data   Histogram Stemplot (or “stem-and-leaf diagram”)  Boxplot (section 1.2) Symmetric Skewed right (or “positively skewed”) Skewed left (or “negatively skewed”) Peak (or “mode”) Unimodal vs. Bimodal Outlier

Variables

In a study, we collect data from

individuals ,

more formally known as

observations

. Observations can be people, animals, plants, or any object or process of interest.

A

variable

is a characteristic that varies among individuals in a population or in a sample (a subset of a population).

Example: age, height, blood pressure, ethnicity, leaf length, first language The

distribution

of a variable tells us what

values

the variable takes and how often it takes on these values. A distribution can also be thought of as the pattern of variation seen in the data.

Two types of variables

 Variables can be either

numerical

(a/k/a

quantitative

) …  Something that can be counted or measured for each individual and then added, subtracted, averaged, etc. across individuals in the population.

 Example: How tall you are, your age, your blood cholesterol level, the number of credit cards you own  … or

categorical

(a/k/a

qualitative

).

 Something that falls into one of several categories. What can be computed is the count or proportion of individuals in each category.

 Example: Your blood type (A, B, AB, O), your hair color, your ethnicity, whether you paid income tax last tax year or not

How do you know if a variable is categorical or quantitative?

Ask:  What are the individuals in the sample?

  What is being recorded about those individuals?

Is that a number ( “quantitative”) or a statement (“categorical”)?

Individuals in sample

Patient A Patient B Patient C Patient D Patient E Patient F Patient G

Categorical

Each individual is assigned to one of several categories.

DIAGNOSIS

Heart disease Stroke Stroke Lung cancer Heart disease Accident Diabetes

Quantitative

Each individual is attributed a numerical value.

AGE AT DEATH

56 70 75 60 80 73 69

Ways to chart categorical data

Because the variable is categorical, the data in the graph can be ordered any way we want (alphabetical, by increasing value, by year, by personal preference, etc.) 

Bar graphs

Each category is represented by a bar.



Pie charts

Peculiarity: The slices must represent the parts of one whole.

Example: Top 10 causes of death in the United States 2001

Rank Causes of death

1 Heart disease 2 Cancer 3 Cerebrovascular 4 Chronic respiratory 5 Accidents 6 Diabetes mellitus 7 Flu and pneumonia 8 Alzheimer’s disease 9 Kidney disorders 10 Septicemia

Counts

700,142 553,768 163,538 123,013 101,537 71,372 62,034 53,852 39,480 32,238

% of top 10s

37% 29% 9% 6% 5% 4% 3% 3% 2% 2%

% of total deaths

29% 23% 7% 5% 4% 3% 3% 2% 2% 1% All other causes 629,967 26%

For each individual who died in the United States in 2001, we record what was the cause of death. The table above is a summary of that information.

Bar graphs

Each category is represented by one bar. The bar’s height shows the count (or sometimes the percentage) for that particular category.

800 700 600 500 400 300 200 100 0 H ea rt di se as es Top 10 causes of deaths in the United States 2001 The number of individuals who died of an accident in 2001 is approximately 100,000.

C an ce C rs er eb ro va C sc hr ul on ar ic re sp ira to ry A cc id D en ia ts be te s m el Fl lit us u & pn eu A lz m on he ia im er 's d is ea K id se ne y di so rd er s S ep tic em ia

800 700 600 500 400 300 200 100 0 H ea rt di se as es Top 10 causes of deaths in the United States 2001 Bar graph sorted by rank  Easy to analyze C an ce C rs er eb ro va C sc hr ul on ar ic re sp ira to ry A cc id D en ia ts be te s m el Fl lit us u & pn eu A lz m on he ia im er 's d is ea K id se ne y di so rd er s S ep tic em ia 800 700 600 500 400 300 200 100 0 A cc A id lz en he ts im er 's d is ea se Sorted alphabetically  Much less useful C an ce C rs er eb ro va C sc hr ul on ar ic re sp ira D to ia ry be te s m el lit Fl us u & pn eu m on H ia ea rt di se as K es id ne y di so rd er s S ep tic em ia

Pie charts

Each slice represents a piece of one whole. The size of a slice depends on what percent of the whole this category represents.

Percent of people dying from top 10 causes of death in the United States in 2000

Percent of deaths from top 10 causes Percent of deaths from all causes

Make sure your labels match the data.

Make sure all percents add up to 100.

Child poverty before and after government intervention—UNICEF, 1996

What does this chart tell you?

•The United States has the highest rate of child poverty among developed nations (22% of under 18).

•Its government does the least—through taxes and subsidies —to remedy the problem (size of orange bars and percent difference between orange/blue bars).

Could you transform this bar graph to fit in 1 pie chart? In two pie charts? Why?

The poverty line is defined as 50% of national median income.

Histograms

 Vertical bar chart where horizontal axis is a numerical scale corresponding to the data.

 Vertical axis represents frequency (how many) or relative frequency (what proportion)  “Peaks” correspond to commonly occurring data values.

 “Valleys” and “tails” correspond to values which do not occur as frequently.

  Most states have between 0 and 10 percent Hispanic residents A very small number have between 25 and 45 percent.

Histograms

The range of values that a variable can take is divided into equal size intervals. The histogram shows the number (i.e.,

frequency

) of individual data points that fall in each interval.

The first column represents all states with a percent Hispanic in their population between 0% and 4.99%. The height of the column shows how many states (27) have the percent Hispanic residents in this range.

The last column represents all states with a percent Hispanic between 40% and 44.99%. There is only one such state: New Mexico, at 42.1% Hispanics.

Creating a histogram

What “class size” should you use?

 Use an appropriate number of classes − usually between 5 and 15 work well, depending on number of data values being represented.  Either too few or too many classes will obscure the pattern in the data.

 Not so detailed that it is no longer summary   Avoid using many classes having frequency of only 0 or 1 Not overly summarized so that you lose all the information  rule of thumb: start with 5 to 10 classes Look at the distribution and refine your classes.

(There isn’t a unique or “perfect” histogram.)

Guidelines for histograms

 Label the horizontal axis with a consistent numerical scale right side of the graph) . (Don’t leave any gaps in the scale or compress the scale toward the left or  Use vertical bars of equal width.

 Label the horizontal scale on the class boundaries.

 An exception is when each bar corresponds to a single whole number. Then it is reasonable to label each bar at its midpoint.

Same data set

Not summarized enough Too summarized

Interpreting histograms

When describing the distribution of a quantitative variable, we look for the overall pattern and for striking deviations from that pattern. We can describe the

overall

pattern of a histogram by its

shape, center,

and

spread.

Histogram with a line connecting each column  too detailed Histogram with a smoothed curve highlighting the overall pattern of the distribution

Most common distribution shapes

 A distribution is

symmetric

if the right and left sides of the histogram are approximately mirror images of each other.

 A distribution is

skewed to the right

if the right side of the histogram (side with larger values) extends much farther out than the left side. It is

skewed to the left

if the left side of the extends much farther out than the right.

Complex, multimodal distribution Symmetric distribution Skewed right distribution  Not all distributions have a simple overall shape, especially when there are few observations.

IMPORTANT NOTE: Your data are the way they are. Do not try to force them into a particular shape.

Histogram of Drydays in 1995

It is a common misconception that if you have a large enough data set, the data will eventually turn out nice and symmetrical.

Outliers

An important kind of deviation is an

outlier .

Outliers are observations that lie outside the overall pattern of a distribution. Always look for outliers and try to explain them.

The overall pattern is fairly symmetrical except for two states clearly not belonging to the main trend. Alaska and Florida have unusual representation of the elderly in their population.

A large gap in the distribution is typically a sign of an outlier.

Alaska Florida

More on Outliers

 If outlier is incorrect data  Correct it if possible.

(e.g., data entry error, false response)   Discard if absolutely sure it is wrong.

Discarding data might introduce a bias if uncorrectable errors tend to be mainly high (or mainly low) values.

 If outlier is correct data consider effect on analysis  Which statistical technique is most appropriate?

 How are conclusions affected by the outlier?

 Some Other Strategies:  Refine population definition if unusual responses are not part of intended study group.

 If sample size is quite small collect more data to see if any gaps fill in.

Stemplots (or “stem-and-leaf ” diagrams)

How to make a

stemplot:

1) Truncate (or “trim”) the data to appropriate level of accuracy.

2) Separate each observation into a

stem,

consisting of all but the final (rightmost) digit, and a

leaf,

which is the remaining final digit. Stems may have as many digits as needed, but each leaf contains a single digit.

3) Write the stems in a column with the smallest value at the top, and draw a vertical line at the right of this column.

4) Write each leaf in the row to the right of its stem, in increasing order out from the stem. Leaves should be aligned in vertical columns. (Why?) STEM LEAVES

State

Alabama Alaska Arizona Arkansas California Colorado Connecticut Delaware Florida Georgia Hawaii Idaho Illinois Indiana

Percent

Iowa Kansas Kentucky Louisiana Maine Maryland Massachusetts Michigan Minnesota Mississippi Missouri Montana Nebraska Nevada NewHampshire NewJersey NewMexico NewYork NorthCarolina NorthDakota Ohio Oklahoma Oregon Pennsylvania RhodeIsland SouthCarolina SouthDakota Tennessee Texas Utah Vermont Virginia W ashington W estVirginia W isconsin W yoming 4.7

1.2

1.9

5.2

8 3.2

8.7

2.4

1.4

2 32 9 0.9

4.7

7.2

0.7

3.6

6.4

1.5

4.1

25.3

2.8

32.4

17.1

9.4

4.8

16.8

5.3

7.2

7.9

10.7

3.5

2.8

7 2.1

2 5.5

19.7

1.7

13.3

42.1

15.1

1.5

2.4

0.7

4.3

6.8

3.3

2.9

1.3

Step 1: Sort the data

State

Maine W estVirginia Vermont NorthDakota Mississippi SouthDakota

Percent

Alabama Kentucky NewHampshire Ohio Montana Tennessee Missouri Louisiana SouthCarolina Arkansas Iowa Minnesota Pennsylvania Michigan Indiana W isconsin Alaska Maryland NorthCarolina Virginia Delaware Oklahoma Georgia Nebraska W yoming Massachusetts Kansas Hawaii W ashington Idaho Oregon RhodeIsland Utah Connecticut Illinois NewJersey NewYork Florida Colorado Nevada Arizona Texas California NewMexico 0.7

0.7

0.9

1.2

1.3

1.4

1.5

1.5

1.7

1.9

2 2 2.1

2.4

2.4

2.8

2.8

2.9

3.2

3.3

3.5

3.6

4.1

4.3

4.7

4.7

4.8

5.2

5.3

5.5

6.4

6.8

7 7.2

7.2

7.9

8 8.7

9 9.4

10.7

13.3

15.1

16.8

17.1

19.7

25.3

32 32.4

42.1

Step 2:

Percent of Hispanic residents

Assign the values to stems and leaves

in each of the 50 states

Stemplots versus histograms

Stemplots are quick and dirty histograms that can easily be done by hand, therefore very convenient for back of the envelope calculations. However, they are rarely found in scientific publications.

Stemplots versus histograms

1) Advantages of Stemplots  Quick to do by hand (no frequency table needed)  Maintains all the numerical data  Good for comparing two distributions (using back-to-back plots) 2) Disadvantages  Not as crisp visually (compared to a graphics-quality histogram)  Most people like numerical scale on horizontal axis.

Variations on stemplots

1) Stem-splitting  Use to increase the number of stems  Create one stem for leaves 0-4  Second stem for leaves 5-9 2) Back-to-back Stemplots  Use to compare two data sets measured on same scale (e.g., male vs. female heights).

 Use single stem with leaves on opposite sides for different data sets.

Ways to chart quantitative data

 Line graphs: time plots Use when there is a meaningful sequence, like time. The line connecting the points helps emphasize any change over time.  Histograms and stemplots These are summary graphs for a single variable. They are very useful to understand the pattern of variability in the data.

 Other graphs to reflect numerical summaries (see Chapter 1.2)

Line graphs: time plots

In a time plot, time always goes on the horizontal, x axis. We describe time series by looking for an overall pattern and for striking deviations from that pattern. In a time series: A

trend

is a rise or fall that persist over time, despite small irregularities.

A pattern that repeats itself at regular intervals of time is called

seasonal variation .

Retail price of fresh oranges over time

Time is on the horizontal, x axis.

The variable of interest —here “retail price of fresh oranges”— goes on the vertical, y axis. This time plot shows a regular pattern of yearly variations. These are seasonal variations in fresh orange pricing most likely due to similar seasonal variations in the production of fresh oranges.

There is also an overall upward trend in pricing over time. It could simply be reflecting inflation trends or a more fundamental change in this industry.

A time plot can be used to compare two or more data sets covering the same time period.

Date 1918 influenza epidemic # Cases # Deaths

week 1 week 2 week 3 week 4 week 5 week 6 week 7 week 8 week 9 week 10 week 11 week 12 week 13 week 14 week 15 week 16 week 17 36 531 4233 8682 7164 2229 600 164 57 722 1517 1828 1539 2416 3148 3465 1440 0 0 130 552 738 414 198 90 56 50 71 137 178 194 290 310 149

1918 influenza epidemic 1918 influenza epidemic

10000 9000 10000 800 800 700 700 600 600 500 500 400 400 300 2000 1000 0 0 we w ee k 1 we w ee k 3 we w ee k 5 we w ee k we 7 ek w ee k we 9 w ee 1 k we 11 w 3 ee k we 13 w ee 5 k we 15 ek w 7 ee k 17 300 200 200 0 100 100 0 # Cases # Cases # Deaths # Deaths The pattern over time for the number of flu diagnoses closely resembles that for the number of deaths from the flu, indicating that about 8% to 10% of the people diagnosed that year died shortly afterward from complications of the flu.

Scales matter

How you stretch the axes and choose your scales can give a different impression.

Death rates from cancer (US, 1945-95) 250 200 150 100 50 0 1940 1950 1960 1970 Years 1980 1990 2000 Death rates from cancer (US, 1945-95) 250 200 150 100 50 0 1940 1960 Years 1980 2000 Death rates from cancer (US, 1945-95) 220 200 180 160 140 120 1940 1960 Years 1980 2000 Death rates from cancer (US, 1945-95) 250 200 150 100 50 0 1940 1960 Years 1980 2000 A picture is worth a thousand words, BUT  There is nothing like hard numbers.

Look at the scales.

Why does it matter?

Cornell’s tuition over time Cornell’s ranking over time What's wrong with these graphs?

Careful reading reveals that: 1. The ranking graph covers an 11-year period, the tuition graph 35 years, yet they are shown comparatively on the cover and without a horizontal time scale.

2. Ranking and tuition have very different units, yet both graphs are placed on the same page without a vertical axis to show the units. 3. The impression of a recent sharp “drop” in the ranking graph actually shows that Cornell’s rank has IMPROVED from 15th to 6th ...