A Logic of Diversity II

Scott E Page Complex Systems, Political Science, Economics and Institute for Social Research University of Michigan Santa Fe Institute Michigania 02005

Enlarging The Mantra

Identity, Training, Experiential Diversity

Diverse Perspectives Better Outcomes Michigania 02005

Monday’s Talk: Unpacking The First Box Diverse Perspectives Michigania 02005

Monday’s Talk: Unpacking The First Box Perspectives Heuristics Interpretations Michigania 02005

Today’s Talk: Demonstrating Causality Diverse Perspectives Better Outcomes Michigania 02005

Specific Tasks Problem Solving Prediction Preference Aggregation Michigania 02005

Why Construct Models?

Models allow us to provide conditions for when a statement is true.

The Pythagorean Theorem: ``A-squared equals B squared plus C squared’’ only holds for right triangles.

Michigania 02005

Finding the Conditions ``Two heads are better than one!’’ ``Too many cooks spoil the broth’’ Which one wins? Which do we apply in a given setting.

Michigania 02005

Finding the Conditions ``Two heads are better than one!’’ ``Too many cooks spoil the broth’’

Condition:

For an irreversible process, too many cooks spoil the broth.

Michigania 02005

Swarm of Bees Almost all of social science looks at averages and changes in those averages.

Analogy:

if you look at a swarm of bees, the path of any one bee is hard to predict and understand, but in the swarm all of those idiosyncratic behaviors cancel out and we can identify general trends.

Michigania 02005

The Buzz Bee hives must stay around 96 degrees in order for bees to reach maturation. Bees achieve this by genetic mechanisms that drive two behaviors: When hot: fan out or leave the hive When cool: huddle together Michigania 02005

Diversity and Homeostasis

Genetically homogeneous bees:

All get cool (or hot) at the same time. Temperature in hive fluctuates wildly. (1930’s heating system)

Genetically diverse bees:

Get cool (or hot) at different temperatures. Temperature stabilizes. Michigania 02005

Rethinking the Swarm The logic of cancellation does not hold because there are

feedbacks

averages. between the bees. Those feedbacks imply we cannot look at Groups of people solving problems, making predictions, and making choices create feedbacks in abundance.

Michigania 02005

Problem Solving Michigania 02005

Problem Solving

Perspectives Heuristics

Michigania 02005

135 91 The Idea A,b,x x,M Thermometer: SAT,IQ Michigania 02005 Toolbox: skills, heuristics

size Perspective number of chunks Ben & Jerry’s Ice Cream Array Michigania 02005

size Heuristic number of chunks Ben & Jerry’s Ice Cream Array Michigania 02005

Consultant perspective: caloric rank Michigania 02005

Consultant perspective: caloric rank heuristic: look left and right Michigania 02005

Performance • Average Performance Given – solution in perspective – application of heuristics • Ben and Jerry – average quality of solution = 82 • Consultant average quality of solution = 74 Michigania 02005

Perspective Diversity Ben and Jerry stuck at 83 80 75

83

81 73 Michigania 02005

Perspective Diversity Ben and Jerry stuck at 83 80 75

83

81 73 consultant Gets to 86 80 86 83 74 Michigania 02005

Diversity or Ability: A Test Create a bunch of artificial problem solving agents and rank these agents by their average performances on a difficult problem.

All of the agents must be “smart” Michigania 02005

Two Groups • Group 1: Best 20 agents • Group 2: Random 20 agents Have each group work collectively - when one agent gets stuck at a point, another agent tries to find a further improvement. Group stops when no one can find a better solution.

Michigania 02005

The IQ View 139 138 137 135 132 135 121 75 84 135 111 31 Alpha Group Diverse Group Michigania 02005

And the winner is..

“Most of the time” the diverse group outperforms the group of the best by a substantial margin.

See Lu Hong and Scott Page Proceedings of the National Academy of Sciences (2002) Michigania 02005

The Toolbox View ABC ABD ACD BCD ADE BCD AHK EZ FD BCD AEG IL Alpha Group Diverse Group Michigania 02005

Formal Version

Theorem:

Given a set of diverse problem solvers, a random collection outperforms a collection of the “best” individual problem solvers provided -the set is large -the problem is hard -the problem solvers are smart Michigania 02005

. Prediction Michigania 02005

. Prediction

Interpretations Mental Models

Michigania 02005

The Madness of Crowds We tend to think of crowds of people as irrational mobs. And that can be true. When people hear the ideas and opinions of others, they often succumb to peer pressure rather than speaking their own minds. Michigania 02005

A: B :

Which Line is Longer?

_____________ ___________

Michigania 02005

The dim boy claps because the others clap.

- Richard Hugo Michigania 02005

The Wisdom of Crowds If people do not hear the opinions of others, or if they render their true predictions anyway crowds can be incredibly wise. Michigania 02005

Suroweicki’s Examples Morton Thiokol’s stock plunge Prediction Markets Hollywood Stock Exchange Iowa Electronic Market Sports Betting Markets Who Wants to be a Millionaire 1906 West of England Fat Stock and Poultry Exhibition Michigania 02005

Two Separate Phenomena 1. Information known by part of the crowd 2. Aggregative diverse predictive models Michigania 02005

Revealing Known Information Which of the following books would you NOT find in the Point o’ Pines Library A.

The Periwinkle Steamboat

- Lancaster B.

Curtains -

Agatha Christie C.

Unabridged Crossword Puzzle Dictionary

D.

I am Charlotte Simmons -

Tom Wolfe Michigania 02005

Information Rising Suppose that no one know the answer but that 18 people know one of the books on the list is in the library and that 18 people know two of the books on the list are in the library. This means that 64 people guess randomly.

Michigania 02005

Information Rising Of 64 Clueless: Correct answer gets 16 Of 18 know one: Correct answer gets 6 Of 18 know two: Correct answer gets 9 Total 31 Other answers get 23 (on average) Michigania 02005

The Answer Is… Which of the following books would you NOT find in the Point o’ Pines Library B.

Curtains -

Agatha Christie Michigania 02005

Aggregating Diverse Predictions In most of the situations described, people do not know the answer yet. We can assume that people have diverse predictive models. We’d like to understand how that aggregation occurs and what roles diversity and ability play.

Michigania 02005

H Experience MH ML L G G G B Reality Charisma H MH ML L G G B G G B B B G B B Michigania 02005

Experience Interpretation 75 % Correct Experience H MH ML L G G G B G G B G G B B G B B B B Michigania 02005

Charisma Interpretation 75% Correct H MH ML L G G G B G G B B B G B B G Michigania 02005 B B

Balanced Interpretation 75% Correct H Good to be extreme on one MH measure, bad on other ML L G G B H MH ML L G G B B B G B G B Michigania 02005

Voting Outcome Charisma H MH ML L H MH ML L GGB GGG GBG BGB GGG GGB GBG BGG BBG BBB BBG BGB BGG BBG BBB Michigania 02005

The Mathematics of Prediction Prediction: # runs scored by winning softball team Brad Mon 8 Tue Wed 10 9 Matt 10 12 8 Michigania 02005

“Crowd’’ Prediction Mon Brad 8 Matt Crowd 9 Tue Wed 10 10 10 12 8 11 9 Michigania 02005

Actual Numbers Mon Brad 8 Matt Crowd

Actual

10 12 8 9 Tue 10 10 11

8 12

Wed 9

9

Michigania 02005

Squared Errors Brad: Matt : (8-8) 2 +(10-12) 2 +(10-9) 2 = 5 (10-8) 2 +(12-12) 2 +(8-9) 2 = 5 Crowd: (9-8) 2 +(11-12) 2 +(9-9) 2 = 2 Michigania 02005

Diversity of Predictions (Brad-Crowd) 2 = 1 + 1 + 1 = 3 (Matt-Crowd) 2 = 1 + 1 + 1 = 3 Michigania 02005

Notice: 2 = 5 - 3 Crowd Error = Average Error - Diversity Michigania 02005

Diversity Prediction Theorem Crowd Error = Average Error - Diversity (note: proven by statisticians, computer scientists, and economists) Michigania 02005

Crowd = Average - Diversity • Diversity as important as ability • Limit to how much diversity (otherwise crowd error would be negative) Michigania 02005

Experts on NFL Draft Player #1 #2 #3 #4 #5 #6 #7 #8 Alex Smith 1 1 1 1 1 1 1 2 Ronnie Brown Braylon Edwards 2 2 4 2 2 5 2 6 3 3 2 7 3 2 3 3 Cedric Benson Carnell Williams Adam Jones 4 4 13 4 8 4 8 4 8 5 5 5 4 13 4 8 16 9 6 8 6 6 9 17 Error^2 158 89 210 235 112 82 39 300

Average Error: Diversity: 153.13

101.52

Michigania 02005

Crowd of Experts on NFL Draft Player Alex Smith Ronnie Brown Braylon Edwards Cedric Benson Carnell Williams Adam Jones

Error^2

Crowd 1.13 3.13

3.25

6.13

6.50

9.63

51.61

Michigania 02005

Crowds Beat Averages Law Crowd Error < Average Error Michigania 02005

Does Crowd Beat Best?

In the NFL draft example, the best predictor Pete Brisco had an error of only 39. He outperformed the crowd, which had an error of 51.6. Michigania 02005

Novices and Experts

Novices:

Base their models on only a few variables or a few boxes.

Experts:

Base their models on many variables or many boxes.

Michigania 02005

In Praise of Experts

Theorem:

If an expert contains every variable considered by any one of the novices, the expert predicts better than the crowd of novices. Michigania 02005

Crowds vs Experts

Test Set:

Linear functions defined over 20 variables.

Crowd:

Each of 100 novices looks at N randomly chosen variables

Expert:

Looks at E>N variables

Training:

300 independent variables

Contest:

300 independent variables Michigania 02005

N 3 E 20 3 15 5 10 5 7 Crowds vs Experts % of Time Expert Wins 94.66% 34.66% 29.33% 9% Michigania 02005

What’s Happening

Expert:

Getting best fit over all his variables.

Crowd:

Getting an average of many fits over many distinct subsets of variables. Michigania 02005

Put Another Way

Expert:

Great partial view

Crowd:

So-so complete view Michigania 02005

Diversity and Prediction Diverse predictors generate better predictions unless someone’s head is large enough and data is sufficient enough for a complete model.

Michigania 02005

Preference Aggregation Michigania 02005

Instrumental vs Fundamental

Fundamental Preferences:

Preferences over outcomes

Instrumental Preferences:

Preferences over policies to attain outcomes Michigania 02005

Instrumental Politics “I am the _____ candidate” A. Pro crime B. Anti child C. Anti environmental D. Pro drug addiction E. Higher health care costs Michigania 02005

Preference Diversity Problems • Preference Cycles • Manipulation • Underprovision Michigania 02005

Preference Cycle A = Arts & Crafts, B = Boating, T = Tennis Lindsey: A > B > T Samuel: B > T > A Becca : T > A > B Michigania 02005

Preference Cycle Lindsey: A > B > T Samuel: B > T > A Becca : T > A > B • Majority Vote Outcome: A > B > T > A Michigania 02005

Manipulation Given any voting rule, people with diverse preferences will always have an incentive to misrepresent themselves.

Implication:

People in diverse groups will not trust one another as much. Michigania 02005

Under Provision If we want different outcomes and have a fixed budget, we are likely to spread our money too thin.

Idea:

Rather than have a good car or a nice boat, we have a lousy car and a lousy boat.

Michigania 02005

Theoretical Summary Tasks involve - Solution Generation (problem solving) - Evaluation (prediction) - Choice (preference aggregation) Michigania 02005

“Diversity is Ability” To be different is to be able to make a contribution.

Diversity Trumps Ability

: Diverse group does better than “able” group at problem solving Michigania 02005

“Diversity is Ability”

Diversity Prediction Theorem

Crowd Error = individual error - diversity (ability and diversity enter equally) Michigania 02005

Complication Preference diversity creates cycles. It creates incentives to act strategically and to manipulate agendas. At the same time, preference diversity may be a primary cause of the other types of diversity. Michigania 02005

Summary The empirical evidence suggests that diverse perspectives, mental models, and tools lead to “better outcomes” but that value diversity creates problems. Michigania 02005

Pudding Michigania 02005

Quick Look at the ``Facts’’ • Growth of modern civilization • National level GDP • City level productivity • Diverse team performance Michigania 02005

Rise of Modern Civilization • Jared Diamond: diversity/easy problems • Joel Mokyr: exploiting diversity • Michael Kremer: 1 million years of data shows growth and population size correlated Michigania 02005

National Level GDP • Paul Romer: Diversity crucial to economic growth • Ethnic Linguistic Fractionalization (ELF): strongly negatively correlated with economic growth Michigania 02005

Performance of Cities (42) • Doubling of city size increases productivity by 6% to 20% • Arrow, Lucas: spillovers within an industry (silicon valley) • Jacobs, Auerbach: spillovers between industries (just in time)* Michigania 02005

Identity Diverse Teams • Generate more solutions (many worse) • Thomas and Ely: do better if they have diverse heuristics and perspectives • People in diverse groups are less happy - world views are challenged - feel like outcomes were manipulated Michigania 02005

The End of Great Scientists Physics Nobels: Chemistry Nobels: First 10 14 10 Michigania 02005

The End of Great Scientists Physics Nobels: Chemistry Nobels: First 10 Last 10 14 28 10 27 (There’s a maximum of three) Michigania 02005

Final Thought Individual ability not likely to grow much. Collective diversity can grow. Diversity is our best hope to solve problems and to create innovations. Michigania 02005

www.cscs.umich.edu/~spage Michigania 02005