Transcript image - Stanford Computer Science

We are very happy with the students that we get from this
university. . . . We just wish we could hire two to three
times as many of them.
— Bill Gates at Stanford, February 19, 2008
Computing and Competitiveness:
Implications of the Programmer Shortage
Eric Roberts
Professor of Computer Science
Stanford University
CISAC
February 4, 2009
Overview
• Computing and information technology underlie much of the
world economy and offer some of the best employment
opportunities for college graduates in the United States today.
• At the same time, student interest in these disciplines has
plummeted throughout the developed world. The Computing
Research Association estimates that computing enrollments in
the United States have fallen by 50 percent since their peak in
2000. In many European countries, the decline has been even
more severe. The decrease in student interest has, moreover,
been particularly pronounced among women and minority
students, reducing diversity as the pool shrinks.
• Declining student interest in technical fields represents a
serious threat to economic competitiveness at a critical time.
The actions that developed countries take in response to this
challenge will have a profound effect on the health of the
world economy.
The Changing World Order
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
The Challenge of Global Change
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
In a world where advanced knowledge
1)
America’s
talent pool
is Increase
widespread
and low-cost
labor by
is
vastly
improving
K-12
mathematics
readily available, U.S. advantages in
science education;
theand
marketplace
and in science and
technology
have
begun totheerode.
A
2)
Sustain and
strengthen
nation’s
comprehensive
commitment and
to coordinated
long-term federal
basic
effort
is
urgently
needed
to
bolster
U.S.
research;
competitiveness and pre-eminence in
3)
Develop,
retain top
these
areas.recruit,
This and
congressionally
students, report
scientists,
engineers
requested
by aandpre-eminent
from both the U.S.
and abroad; and
committee
makes
four
recommendations along with 20
4)
Ensure that theactions
United that
Statesfederal
is the
implementation
premier place
in the
for
policy-makers
should
take world
to create
innovation. jobs and focus new
high-quality
science and technology efforts on
meeting the nation’s needs, especially
in the area of clean, affordable energy:
The Pipeline Paradox in Computing
• The computing industry offers some of the best employment
opportunities for college graduates in the United States today:
– The number of jobs in the domestic software industry are at an all-time
high and are projected to grow dramatically over the next decade.
Employment
(thousands)
Top 10 job growth categories (2006-2016)
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Network systems and data communications analysts
Personal and home care aides
Home health aides
Computer software engineers, applications
Veterinary technologists and technicians
Personal financial advisors
Makeup artists, theatrical and performance
Medical assistants
Veterinarians
Substance abuse and behavioral disorder counselors
2006
2016
Growth
262
767
787
507
71
176
2
465
62
83
402
1,156
1,171
733
100
248
3
148
84
112
53.4
50.6
48.7
44.6
41.0
41.0
39.8
35.4
35.0
34.3
Source: U.S. Department of Labor, Bureau of Labor Statistics, Employment Projections: 2006-16, December 2007.
Degree Production vs. Job Openings
160,000
140,000
120,000
100,000
Ph.D.
Master’s
Bachelor’s
Projected job openings
80,000
60,000
40,000
20,000
Engineering
Physical Sciences
Biological Sciences
Computer Science
Sources: Adapted from a presentation by John Sargent, Senior Policy Analyst, Department of Commerce, at the
CRA Computing Research Summit, February 23, 2004. Original sources listed as National Science
Foundation/Division of Science Resources Statistics; degree data from Department of Education/National
Center for Education Statistics: Integrated Postsecondary Education Data System Completions Survey;
and NSF/SRS; Survey of Earned Doctorates; and Projected Annual Average Job Openings derived from
Department of Commerce (Office of Technology Policy) analysis of Bureau of Labor Statistics 2002-2012
projections. See http://www.cra.org/govaffairs/content.php?cid=22.
The Pipeline Paradox in Computing
• The computing industry offers some of the best employment
opportunities for college graduates in the United States today:
– The number of jobs in the domestic software industry are at an all-time
high and are projected to grow dramatically over the next decade.
– Salaries for newly minted B.S. graduates in Computer Science are
high, sometimes exceeding the $100,000 mark.
– In 2005, Money magazine rated software engineer as the number one
job in America.
– Employment in this area is vital for national competitiveness.
• At the same time, student interest in these disciplines has
plummeted. The Computing Research Association (CRA)
Google and Facebook are fighting hard to hire this year’s
estimates
that computing
enrollments
crop of computer
science graduates,
we’ve heard, andhave fallen by almost 50
ground zero is Stanford. Most of the class of 2008 already
percent
since
their
peak
in is2000.
have job
offers even
though
graduation
months away.
Last year, salaries of up to $70,000 were common for the
best students. This year, Facebook is said to be offering
$92,000, and Google has increased some offers to $95,000
to get their share of graduates. Students with a Master’s
degree in Computer Science are being offered as much as
$130,000 for associate product manager jobs at Google.
• Why this disconnect?
The Need for Greater Understanding
I have not seen a compelling narrative for the decline in student interest.
— Bill Gates, Stanford University,
February 19, 2008
• The failure of universities to produce a sufficient number of
graduates with the necessary computing skills is now widely
recognized as a crisis in both academia and industry.
• Unfortunately, the underlying causes for the decline in student
interest are not well understood. Although several theories
seek to explain the decline in student interest, they do not
provide a comprehensive explanation of student behavior.
• These slides represent an early attempt toward developing a
“compelling narrative” of the sort Bill Gates described during
his visit to Stanford last year. That narrative is as yet a work
in progress, and I welcome any comments and criticism.
Why this Paradox?
1. The kind of exposure students get to computing at the
elementary and secondary level tends to push people
away from the discipline long before they reach the
university.
2. Fears about the long-term economic stability of
employment in the computing industry continue to have a
profound effect on student interest in our discipline.
3. The image of work in the field—and, more importantly,
all too much of the reality of work in the field—is
unattractive to most students and no longer seems fun,
particularly in comparison to other opportunities that
bright students might pursue.
Why this Paradox?
1. The kind of exposure students get to computing at the
elementary and secondary level tends to push people
away from the discipline long before they reach the
university.
2. Fears about the long-term economic stability of
employment in the computing industry continue to have a
profound effect on student interest in our discipline.
3. The image of work in the field—and, more importantly,
all too much of the reality of work in the field—is
unattractive to most students and no longer seems fun,
particularly in comparison to other opportunities that
bright students might pursue.
The Problem Starts Early
The UCLA HERI study shows that students have already made
their decisions before they reach university.
Source: Higher Education Research Institute at UCLA, 2005
Computing Faces Huge Challenges in Schools
• People who have software development skills command high salaries
and tend not to teach in schools.
• In many schools, computing courses are seen as vocational rather
than academic. The NCAA has eliminated academic credit for all
computing courses on this basis.
• Students who are heading toward top universities are advised to take
non-CS courses to bolster their admissions chances.
• Because schools are evaluated on how well their students perform in
math and science, many schools are shifting teachers away from
computer science toward these disciplines. Those disciplines,
moreover, actively oppose expanding high-school computer science.
• Administrators find tools like PowerPoint more sexy and exciting. J
• Computing skills in general—and programming in particular—have
become much harder to teach.
• Teachers have few resources to keep abreast of changes in the field.
CS is Losing Ground
• The Computer Science exam is the only Advanced Placement
exam that has shown declining student numbers in recent years.
Eliminated in 2009
CS Is Tiny Compared with Other Sciences
Why this Paradox?
1. The kind of exposure students get to computing at the
elementary and secondary level tends to push people
away from the discipline long before they reach the
university.
2. Fears about the long-term economic stability of
employment in the computing industry continue to have a
profound effect on student interest in our discipline.
3. The image of work in the field—and, more importantly,
all too much of the reality of work in the field—is
unattractive to most students and no longer seems fun,
particularly in comparison to other opportunities that
bright students might pursue.
Myths of a Jobs Crisis Persist
There is no shortage of evidence that people believe the myths
about the lack of jobs and the danger of outsourcing.
December 1, 2005
Blue Skies Ahead for IT Jobs
BY MARIA KLAWE
Contrary to popular belief, career
opportunities in computer science
are at an all-time high. We’ve got to
spread that message among students
from a rainbow of backgrounds, or
risk becoming a technological
backwater.
All this talk about “Blue Skies” ahead just
can’t hide the stark fact that Americans
who don’t wish to migrate to India and/or
some other off-shore haven are going to
have a difficult career.
Why would any smart American undergrad
go into IT when companies like IBM and
HP are talking of stepping up their offshoring efforts in the coming years? They
want cheap labor, no matter the real cost.
I have been very successful in IT, but I
certainly wouldn’t recommend it today to
anyone except people who are geeks. . . .
Maria Klawe
Harvey
Mudd
College
I thinkPresident,
the latest
figures
from
the U.S.
(at the
at Princeton)
Department
oftime,
LaborDean
are not
correct.
Myths about Offshoring
1. All IT jobs will soon be outsourced to India and China.
2. Good IT workers will be easy to find in the new “flatter” world.
3. Companies will always seek the lowest-priced labor.
The 2006 ACM report on Globalization
and Offshoring of Software finds that
even though offshoring of software has
increased the number of computing jobs
in India and China, it has also increased
the number of jobs in the United States.
Thus, at least in computing, globalization
has functioned exactly as the theory of
comparative advantage suggests.
A Thought Experiment about Offshoring
• Suppose that you are Microsoft and that you can hire a
software developer from Stanford whose loaded costs will be
$200,000 per year. Over in Bangalore, however, you can hire a
software developer for $75,000 per year. Both are equally
talented and will create $1,000,000 annually in value. What do
you do?
• Although the developer in Bangalore has a higher return, the
optimal strategy is to hire them both. After all, why throw away
$800,000 a year?
• Any elementary economics textbook will explain that one hires
as long as the marginal value of the new employee is greater
than the marginal cost. The essential point is that companies
seek to maximize return, and not simply to minimize cost.
Industry Reports a Labor Shortage
— April 28, 2005
Gates Cites Hiring Woes, Criticizes Visa Restrictions
By David A. Vise
Microsoft Corp. Chairman Bill Gates said yesterday the software giant is having
enormous difficulty filling computer jobs in the United States as a result of tight visa
restrictions on foreign workers and a declining interest among U._S. students in computer
science.
<tab>Speaking on a technology panel at the Library of Congress, Gates said a decline in the
number of U._S. students pursuing careers in science and technology is hurting Microsoft
in the short run, and could have serious long-term consequences for the U._S. economy if
the problem is not addressed.
<tab>“We are very concerned that the U._S. will lose its competitive position. For
Microsoft, it means we are having a tougher time hiring,” Gates said. “The jobs are there,
and they are good-paying jobs, but we don’t have the same pipeline.”
http://www.washingtonpost.com/wp-dyn/content/article/2005/04/27/AR2005042702241.html
And Various Data Seem to Agree
Working
in thethe
lifepattern
sciences
requires a vs.
degree
in biology
In160,000
computing,
of typically
degree production
employment
is
Ph.D.
140,000
or
some closely
related field, but relatively few biology majors
reversed.
Master’s
Employment
(thousands)
120,000 end up working in the field.
actually
Bachelor’s
Top 10 job growth categories (2006-2016)
100,000
Projected job openings
2006
2016
Growth
1. Network systems and data communications analysts
262
402
53.4
80,000
2. Personal
and home in
carecomputing
aideslife sciences
767 1,156
50.6
• 39%
80%
of workers
the
have have
3. Home health aides
787 1,171
48.7
60,000
degrees
in computing.
the life sciences.
4. Computer software engineers, applications
507
733
44.6
40,000
5. Veterinary technologists and technicians
71
100
41.0
20,000
6. Personal financial advisors
176
248
41.0
7. Makeup artists, theatrical and performance
2
3
39.8
Engineering
Physical Sciences Biological Sciences
Science
8. Medical
assistants
465 Computer
148
35.4
62
84
35.0
9. Veterinarians
•Sources:
14%
71%
of
graduates
students
with
with
degrees
degrees
in
in
the
life
Adapted from
a presentation
by John
Sargent,counselors
Senior Policy Analyst,83Department
at the
10. Substance
abuse
and behavioral
disorder
112 of Commerce,
34.3
CRA Computing
Research
February 23, 2004.
sciences
computing
work
remain
in
those
inSummit,
the
fields.
field.
Original sources listed as National Science
Source: U.S.
Department of Labor,
Bureau Resources
of Labor Statistics,
2006-16,
December 2007.
Foundation/Division
of Science
Statistics;Employment
degree data Projections:
from Department
of Education/National
Center for Education Statistics: Integrated Postsecondary Education Data System Completions Survey;
and NSF/SRS; Survey of Earned Doctorates; and Projected Annual Average Job Openings derived from
Department of Commerce (Office of Technology Policy) analysis of Bureau of Labor Statistics 2002-2012
projections. See http://www.cra.org/govaffairs/content.php?cid=22.
Sources: National Science Foundation/Division of Science Resources Statistics, SESTAT (Scientists and Engineers
Statistical Data System), 1999, as presented by Caroline Wardle at Snowbird 2002.
But There Are Contrary Arguments
— January 26, 1998
Now Hiring! If You’re Young
By Norman Matloff
DAVIS, Calif—Readers of recent reports about a shortage of computer programmers
would be baffled if they also knew that Microsoft hires only 2 percent of its
applicants for software positions. Even among those applicants whom Microsoft
invites to its headquarters for interviews, according to David Pritchard, the director of
recruiting, the company makes offers to only one in four.
<tab>You don’t have to be a “techie” to see that such a low ratio, typical for the
industry, contradicts the claims of a software labor shortage. If companies were that
desperate, they simply could not be so picky.
http://query.nytimes.com/gst/fullpage.html?res=9E0CE6DF123BF935A15752C0A96E958260
Matloff argues that there is no programmer shortage. The only
clear conclusion one can draw from the data is that companies
perceive a shortage in applicants who pass their quality threshold.
Variations in Programmer Productivity
• In 1968, a study by Sackman, Erikson, and Grant1 revealed that
programmers with the same level of experience exhibit variations
of more than 20 to 1 in the time required to solve particular
programming problems.
• More recent studies2, 3, 4 confirm this high variability.
• Most industry insiders believe that the productivity variance is
even higher today. In 2005, Google’s VP for Engineering, Alan
Eustace, told The Wall Street Journal that one top-notch engineer
is worth 300 times or more than the average.5
1
2
3
4
5
H. Sackman, W. J. Erikson, and E. E. Grant. Exploratory experimental studies comparing on-line and off-line programming
performance. Communications of the ACM, January 1968.
W. Curtis. Substantiating programmer variability. Proceedings of the IEEE, July 1981.
T. DeMarco and T. Lister. Programmer performance and the effects of the workplace. Proceedings of the 8th International
Conference on Software Engineering. IEEE Computing Society Press, August 1985.
G. E. Bryan. Not all programmers are created equal. In Richard Thayer, Software Engineering Project Management (second
edition), IEEE Computer Society, 1997.
T. Pui-Wing and K. Delaney. Google’s growth helps ignite Silicon Valley hiring frenzy. Wall Street Journal, November 23, 2005.
The Microsoft Programming Personae
Microsoft’s cultural lore defines three types of programmers:
Mort is your most common developer, who doesn’t have a CS
background, may even be a recent newcomer, and doesn’t quite
understand what the computer is doing under the covers, but who
writes the dinky IT programs that make businesses run. Elvis, more
knowledgeable, cares about code quality, but has a life too. Einstein
writes some serious-ass piece of code like device drivers, wants to
get things done, needs to be able to go low level and high level,
needs a language without restrictions to get his job done.
— Wesner Moise, “Who are you? Mort, Elvis or Einstein,” September 25, 2003
http://wesnerm.blogs.com/net_undocumented/2003/09/who_are_you_mor.html
For the most part, Microsoft (along with Google
and other first-rank companies) are seeking to hire
the Einsteins, which explains the low hiring ratio.
Productivity Variations Are Common
• The idea that individuals might differ in productivity by two or
more orders of magnitude initially seems hard to believe.
• In fact, such differences in efficacy and productivity occur
across a range of occupational categories:
–
–
–
–
–
–
Mathematicians
Creative artists (writers, composers, poets, painters)
Performers (musicians, actors)
Motion picture directors
Financial wizards, CEOs
Professional athletes
• No one achieves mastery in any of these fields on the basis of
raw talent alone. Training and practice are essential.
Alternative Models of Software Education
In many creative disciplines, students learn
from mentors who are masters of their craft.
It may make sense to create “conservatories”
for the teaching of software arts, similar to
music conservatories. One possibility might
be some sort of New England Conservatory
of Coding. (Or perhaps a Hogwarts School
for Software Wizardry.)
Wizardry.
SEMESTER AT C++
SEA
Another model might be to create intensive
programs that encourage students to focus on
the art of software development, in much the
same way that programs like the University
of Virginia’s Semester at Sea program offers
a concentrated immersion in oceanography,
geography, and cultural anthropology.
Paul Graham’s Hackers and Painters
When I finished grad school in computer science I went to
art school to study painting. A lot of people seemed surprised
that someone interested in computers would also be
interested in painting. They seemed to think that hacking and
painting were very different kinds of work—that hacking
was cold, precise, and methodical, and that painting was the
frenzied expression of some primal urge.
<tab>Both of these images are wrong. Hacking and painting
have a lot in common. In fact, of all the different types of
people I’ve known, hackers and painters are among the most
alike.
<tab>What hackers and painters have in common is that
they’re both makers. Along with composers, architects, and
writers, what hackers and painters are trying to do is make
good things.
http://www.paulgraham.com/hp.html
Paul Graham
(photo by Niall Kennedy)
Why this Paradox?
1. The kind of exposure students get to computing at the
elementary and secondary level tends to push people
away from the discipline long before they reach the
university.
2. Fears about the long-term economic stability of
employment in the computing industry continue to have a
profound effect on student interest in our discipline.
3. The image of work in the field—and, more importantly,
all too much of the reality of work in the field—is
unattractive to most students and no longer seems fun,
particularly in comparison to other opportunities that
bright students might pursue.
The Real Image Problem
QuickTime™ and a
H.263 decompressor
are needed to see this picture.
http://www.youtube.com/watch?v=CmYDgncMhXw
The Reality Is Also a Problem
Has anyone considered the possibility that it’s just not fun anymore?
—Don Knuth, October 11, 2006
• Students at Stanford have expressed the following concerns:
– Long hours with little chance for a balanced life
– A less pleasant social milieu than other occupations
– A sense that success in programming is possible only for those
who are much brighter than they see themselves to be
– Work that is often repetitive and unchallenging, particularly
when it involves maintaining legacy technology
– Programming has become more difficult than it used to be
– No chance for a lasting impact because of rapid obsolescence
– Fears that employment with an individual company is dicey even
though opportunities are good in the industry as a whole
– Frustration at being managed by nontechnical people who make
more money but are not as bright
– A perception that programmers are definitely on the labor side of
the labor/capital divide
Dilbert Offers an Instructive Lesson
Most students today would rather be Dilbert’s boss than Dilbert.
Capital
Labor
Dangerous Trends
We have met the enemy and he is us.
— Walt Kelly
• As an illustration of this trend, consider the following post that
appeared on SIGCSE-MEMBERS on August 14, 2006:
I have an idea for a panel that I’d like to organize for SIGCSE’07. I’m asking for
volunteers (or nominations of others) to serve on the panel. The panel I’d like to
organize would have a title something like:
“Alternative Models for a Programming-lite Computer Science Curriculum”
The theme of the panel would be to share ideas and thoughts on how we might
reduce (or eliminate) the emphasis on programming within a computer science
curriculum. The basic idea is to cause discussion centered on the knowledge and
skills students of tomorrow will need in the global economic workspace and the
implications for the CS curriculum. As more and more aspects of software
development of “offshored”, what kind of curriculum would allow a student to be
successful in the IT field?
Industry Is Not Amused
• Every technical person in the industry with whom I’ve spoken
is horrified by the prospect of reducing the emphasis on
programming in the undergraduate curriculum.
• At an ACM Education Council meeting in September 2007, a
panel of technical people from companies like Microsoft,
Google, Amazon, and Boeing were united in their concern
about the scarcity of competent software developers. I have
summarized their position as “the computing curriculum is not
nearly as broken as it seems likely to become.”
• Employers in developed countries with high-tech sectors are
desperate for more people with programming talent. In his
keynote at ITiCSE 2007 in Dundee, Scottish entrepreneur Chris
van der Kuyl said that the lack of programming talent was the
greatest limiting factor in the industry. He called it coding.
Programming Remains Central
• Calls to “reduce or eliminate” programming from computing
curricula arise from some undeniable assumptions:
– There are more jobs in IT that don’t require programming.
– Programming is not particularly popular with students today.
– Offshoring of programming jobs has increased.
• Unfortunately, this analysis ignores the following equally valid
propositions:
–
–
–
–
There are more jobs in IT that do require programming.
Programming has historically been what attracts students the most.
Globalization has created more IT jobs in India/China and the U.S.
Offshoring exists largely because of a shortfall of skilled employees.
What We Need To Do
• Encourage the federal government to launch a sputnik-scale
initiative to advance education in science and technology.
• Press government and industry to improve computing education
at the K-12 level, possibly through public-private partnerships.
• Take creative steps to bolster both the image and the reality of
work in the profession.
• Make it clear to students (as well as faculty) that programming
remains essential to much of the work in the field.
• Emphasize the “beauty” of programming by focusing more
attention on software as an art.
The End