Transcript How the Internet Changed Social Science Research

How the Internet Changed
Social Science Research
Richard J. Butler
Economics
Gratitude for help: Sebastian Nilsson, Neil Larson, Daniel
Butler, Matt Butler, Sam Butler, Department of Economics’
Faculty especially Lars Lefgren, Andi Sneed, Katie Johnson,
the FHSS College and the family of Martin Hickman.
Lecture Browser
• Academic Papers Before the Internet
• Internet changed the making of academic
papers through 3 channels of influence:
– Data accessibility
– Finding co-authors
– Working with co-authors
*** It’s Not Your Full Professor’s World ***
Getting an idea
Search for related research
Go the stacks (full government depository)
The Research Librarian
The Departmental Seminar
March 3, 2004
Journal of Irreproducible Results
May B. Hope, Editor
Aurora Borealis College
Camelot, Iowa 66666
Dear Professor Hope,
Attached to this email, please find a copy of “How the
Internet Changed Social Science Research” which Marg N.
Error and I would like to submit for publication in JIR.
Combining cutting-edge primate theory with a deep sense
of entitlement, this is most significant social commentary to
enter the Western tradition since Augustine’s City of God
(OK, maybe Elvis’ 1957 release of Jailhouse Rock). After
your referees have had a chance to examine this paper—
nay, even before—we are sure that you will want to publish
this in your most excellent journal. We look forward to
receiving your acceptance letter.
Humbly,
Richard J. Butler
March 2, 2005
Professor Richard J. Butler
183 FOB, BYU
Provo, Utah 84602
Dear Professor Butler,
We thought you could never produce another paper as bad
as “The Grand Unified Theory of Canine Gangs: Lessons
for Deconstructionism,” but we were wrong. It has taken a
full year to reject your paper because we went through
eight referees before finally finding one with a sufficiently
severe sleep disorder. Please find enclosed a $250 check
which you may keep if you promise never to submit
anything again.
Most Sincerely,
May B. Hope, Editor
The internet changed this
process…
Getting an idea
Has it been done before? Marg at State did
something similar.
Interested,
Marg?
I’ll write up model and
literature review if you do
the data collection and
regression.
Getting
the Data
The Departmental Seminar
More Data
Analysis.
More Revising.
March 3, 2004
Journal of Irreproducible Results
May B. Hope, Editor
Aurora Borealis College
Camelot, Iowa 66666
Dear Professor Hope,
Attached to this email, please find a copy of “How the
Internet Changed Social Science Research” which Marg N.
Error and I would like to submit for publication in JIR.
Combining cutting-edge primate theory with a deep sense
of entitlement, this is most significant social commentary to
enter the Western tradition since Augustine’s City of God
(OK, maybe Elvis’ 1957 release of Jailhouse Rock). After
your referees have had a chance to examine this paper—
nay, even before—we are sure that you will want to publish
this in your most excellent journal. We look forward to
receiving your acceptance letter.
Humbly,
Richard J. Butler
March 2, 2005
Professor Richard J. Butler
183 FOB, BYU
Provo, Utah 84602
Dear Professor Butler,
We thought you could never produce another paper as bad
as “The Grand Unified Theory of Canine Gangs: Lessons
for Deconstructionism,” but we were wrong. It has taken a
full year to reject your paper because we went through
eight referees before finally finding one with a sufficiently
severe sleep disorder. Please find enclosed a $250 check
which you may keep if you promise never to submit
anything again.
Most Sincerely,
May B. Hope, Editor
1. Internet changed data accessibility
2. The internet lowered the cost of finding
a co-author.
3. The internet lowered the cost of
working with a co-author.
Three Channels to examine, but first...
• Measuring the “academic internet” effect
= (# working papers with email address/# total
working papers)
• Journals included in the analysis
NBER Working Paper Email Addresses
percent with email addresses
1
0.9
D8: Information
& Uncertainty
0.8
0.7
0.6
G2: Financial
Institutions
0.5
0.4
J3: Wages,
Compensation,
Labor Costs
0.3
0.2
0.1
0
1995
1996
1997
year
1998
1999
% with email addresse
NBER Working Paper Email Addresses
D8: Information
and Uncertainty
1.2
1
0.8
G2: Financial
Institutions
0.6
0.4
J3:Wages,
Compensation,
Labor Costs
0.2
0
1994
1996
1998
year
2000
Internet Variable
CORRELATION OF INTERNET VARIABLE WITH WORKING PAPER SERIES:
ECONOMICS
POLITICAL SCIENCE
SOCIOLOGY
NBER-D2
NBER-G2
NBER-J3
Harris School, Chicago
0.8
Am Pol Sci Assoc, CGOTS 0.78
Inst for Advanced Studies 0.94
Russell Sage Foundation 0.56
Heinz School, Carnegie
0.85
UpJohn Institute
0.81
Johns Hopkins, Pop Center 0.62
NORC, Chicago
0.84
Pop Res Center, Penn State0.96
Univ
Harvard Cntr, Pop&Dev
0.89
Pop Agng Center, UofPA 0.88
0.98
0.99
0.97
Journals included
•
Anthropology
• CA-Current Anthropology (1990-1999)
• EH-Enthohistory (1990-2004)
•
Economics
• AER-American Economic Review (1990-2000)
• JPE-Journal of Political Economy (1990-2000)
•
Political Science
• AJPS-American Journal of Political Science (1990-2002)
• APSR-American Political Science Review (1990-2000)
• JP-Journal of Politics (1990-2000)
•
Psychology
• PB-Psychological Bulletin (1990-2004)
• PR-Psychological Review (1990-2004)
• SPQ-Social Psychology Quarterly (1990-2002)
•
Sociology
• AJS-American Journal of Sociology (1990-2000)
• ASR-American Sociological Review (1990-2002)
• SF-Social Forces (1990-2004)
1. Internet changed data accessibility
Percent Increase in Empirical Articles
2.5
1.5
1
0.5
-1
SF
ASR
AJS
SPQ
PR
PB
JP
APSR
AJPS
JPE
-0.5
AER
0
CA
percent increase
2
2. The internet lowered the cost of finding
a co-author.
Presentation Quality
Regression Quality
Presentation Quality
Minimum
Presntion
Quality
Regression Quality
Minimum Regression Quality
Presentation Quality
Regression Quality
So, before internet we had…
Six—800-pound gorillas who publish, go to
conferences, review each others work
Two—chimps only good enough in the
presentation dimension
Three—chimps only good enough in the
regression dimension
Eleven—monkeys not good enough in either
dimension
But as the internet lowers “finding a co-author
costs”, then…
Presentation Quality
Regression Quality
Lower “finding co-authors” costs lets
chimps combine and specialize:
They become gorillas
More new authors
Higher quality papers
More multiply-authored articles
and if costs of finding a foreign co-author is
reduced relative to a domestic co-author…
Presentation Quality
Regression Quality
Lower “finding co-authors” costs lets
chimps combine and specialize:
They become gorillas:
More new authors
Higher quality papers
More multiply-authored articles
and if costs of finding a foreign co-author is
reduced relative to a domestic co-author…
More papers with foreign authors
Lowered “finding co-authors”
costs will tend to spread
publications across a greater
number of academics.
full sample
Percent Increase in New Authors
non-empirical sample
3
2.5
1.5
1
-1
-1.5
-2
SF
ASR
AJS
SPQ
PR
PB
JP
APSR
AJPS
JPE
-0.5
AER
0
EH
0.5
CA
percent increase
2
full sample
Percent Increase in Foreign Authors
non-empirical sample
3
2
-2
-3
-4
-5
SF
SR
A
JS
A
SP
Q
PR
PB
JP
PS
R
A
JP
S
E
JP
ER
A
A
-1
EH
A
0
C
percent increase
1
3. The internet lowered the cost of
working with a co-author.
Costs lowered in that:
• Cheaper and faster to share files
• data
• word documents
• Cheaper and faster to monitor work of coauthors
• Costs of specialization lower:
• Literature Review/research context searches
• Computer code (download off the internet)
Presentation Quality
Regression Quality
Will Gorillas want to coauthor?
Co-authoring increases Gorrilla’s output if:
• Two co-authored articles in the AER are better
than one solely authored article
– McDowell, John M., and Janet Kiholm Smith. 1992. "The
Effect of Gender-Sorting on Propensity to Coauthor:
Implications for Academic Promotion." Economic Inquiry
30(1): 68-82.
• There are constant or increasing returns due to
specialization
If the Internet “lowers working with Coauthors” costs, then Gorillas (established
authors) will co-author more often and
there will be:
Relative increase in the number of coauthors for established authors
Relatively more articles published for
established authors
More multiply-authored articles with
established authors
Higher quality papers (on average,
with no necessary improvement at top)
Lowered “working with co-authors”
costs may concentrate
publications toward established
authors.
% Increase in Same Dept Co-Authors
year effects
linear trend
0.8
0.6
0.2
-0.4
-0.6
-0.8
-1
-1.2
-1.4
SF
SR
A
JS
A
PR
SP
Q
PB
JP
R
PS
A
JP
S
A
JP
E
ER
A
-0.2
EH
A
0
C
percent increase
0.4
The last slide on probability of the “same dept coauthors” (P) was a (percent) “difference” estimator:
(Pafter  Pbefore )
But to compare established authors with other authors
we will use “difference in difference” estimators:
estab. auth  P estab. auth )  (P other auth  P other auth )
(Pafter
before
after
before
Relative % Increase in Co-Authorship among
Estab. Authors
year effects
linear trend
0.5
0.3
0.2
0.1
-0.2
-0.3
SF
SR
A
JS
A
PR
SP
Q
PB
JP
R
PS
A
JP
S
JP
E
ER
A
A
-0.1
EH
A
0
C
percent increase
0.4
Ordered Logit: Number of Authors with
Established Authors after the Internet
year effects
linear trend
0.4
0.2
0.1
-0.1
-0.2
-0.3
SF
SR
A
JS
A
PR
SP
Q
PB
JP
R
PS
A
JP
S
A
JP
E
ER
A
EH
A
0
C
percent increase
0.3
Percent Increase in 3+-Author Articles,
Established Author Effect
year effects
linear trend
0.4
0.2
0.1
-0.1
-0.2
-0.3
SF
SR
A
JS
A
PR
SP
Q
PB
JP
R
PS
A
JP
S
A
JP
E
ER
A
EH
A
0
C
percent increase
0.3
Percentage Increase in Established
Authors' Articles
total articles
co-authored articles
1.6
1.2
1
0.8
0.6
0.4
0.2
SF
AJ
S
AS
R
PR
SP
Q
PB
JP
AJ
PS
AP
SR
AE
R
JP
E
EH
0
CA
percentage increase
1.4
Percentage Increase in Established Authors'
Articles--Top 10 Programs
total articles
co-authored articles
1.5
0.5
-0.5
-1
-1.5
SF
SR
A
JS
A
PR
SP
Q
PB
JP
R
PS
A
JP
S
A
JP
E
ER
A
EH
A
0
C
percentage increase
1
Internet’s three channels?
• More empirical papers (exception: econ)
• No evidence of a significant increase in
the fraction of new authors
• However, established authors publishing
more:
• More co-authored articles
• With more co-authors
• Significant increase in the number of articles
Economics of superstars:
• Special skills
• Technology allows for a mass market
(economies in distribution of product)
Superstar academics:
• Special skills
• Internet allows for simultaneous projects
(economies in production of papers)
Not the same as the old days:
• Higher quality articles (for a given journal)
• More co-authored articles:
• Harder to publish solely authored works for junior
faculty
• More articles with theses advisers and other
departmental gorillas as co-authors
• Harder to evaluate jointly authored work in
promotion and tenure decisions
The End
***no actual pixals were harmed in
the preparation of this lecture***
Academic Help Wanted Ads:
1200-pound Gorilla seeking indentured servants. Grad School, University of Chicago. Low
self esteem preferred but on the job conditioning available: www.uchicago.edu/indentured
Clever presentation-Chimp desperately seeking regression-Gorilla. Send sample output to:
[email protected].
Barrel of Monkeys seeking Gorillas for long term relationship. Yacht available. Contact:
recruiting_chair@san_diego_polytechnical_university.edu.
4000-pound Gorillas seeking primates with the board view. NMNA (“No monkeys need apply,”
unless you are heterodox). Contact: recruit_staff@university_on_the_hill.edu.