Transcript KSE631: Content Networking

KSE801: Human Computation
and Crowdsourcing
Uichin Lee
Sept. 5, 2011
Knowledge Service Engineering Department
The Rise of Crowdsourcing
By Jeff Howe (Wired Magazine, 2006)
Remember outsourcing? Sending jobs
to India and China is so 2003.
The new pool of cheap labor:
everyday people using their spare
cycles to create content, solve
problems, even do corporate R&D.
The Rise of Crowdsourcing
•
•
•
•
•
The Professional
The Packager
The Tinkerer
The Masses
And the Age of the Crowd
The Professional
• A story of “Claudia Menashe”
– A project director at the National
Health Museum in Washington,
DC
– Putting a series of interactive
kiosks devoted to potential
pandemics like the avian flu
– An exhibition designer created a
plan for the kiosk; now she wants
to have images to accompany the
text..
• Hire a photographer?
• Pre-existing images—stock
photograph
The Professional
• She ran across a stock photo collection by
Mark
We don’t have much money…
I’ll give you some
discount: how about
$100-$150 per
photograph?
That’s about half of what
a cooperate client would
pay!
Claudia
Mark
Great! I’ll buy 4 images!!
The Professional
One dollar!! That’s a steal!!
iStockphoto: a marketplace for the work of amateur photographers (e.g., homemakers,
students, engineers, dancers); over 20,000 contributors which charge about $1 to $5 per
basic image
The Packager
• Viral videos; how to repurpose content to make compelling
TV on a budget?
• Web Junk 20 at VH1: American television program in which
VH1 and iFilm collaborate to highlight the twenty funniest
and most interesting clips collected from the Internet that
week
• Michael Hirschorn (creator of Web Junk 20)
– “I knew we offered something YouTube couldn’t; television.
Everyone wants to be on TV”
• TV 2.0 – user generated content
– As user generated TV matures, the users will become more
proficient and the networks better at ferreting out the best of
the best..
The Tinkerer:
The Future of Cooperate R&D
• InnoCentives
– Launched in 2001 to connect with brainpower outside the company
– Companies pay solvers anywhere from $10,000 to $100,000 per solution
– Jill Panetta (CSO) says
• More than 30% of the problems are solved!
• The odds of a solver’s success increased in the fields in which they had no formal
expertise
• “The strength of weak ties”– Mark Granovetter
• Similar Services:
• Ed Melcarek
– On most Saturdays, Melcarek attacks problems that have stumped some of
the best cooperate scientists at Fortune 100 companies
– “not bad for a few weeks’ work” (e.g., Colgate problem: $25,000)
• P&G’s R&D:
– “We have 9,000 people on our R&D staff and up to 1.5 million researchers
working through our external networks”
The Masses
• The Turk:
– The first machine capable of beating a human
at chess, built around the late 1760s by a
Hungarian nobleman named Wolfgan von
Kempelen
• Amazon’s Mechanical Turk
– Crowdsourcing for the masse (no specific
talents required)
– Web based marketplace that helps companies
to find people to perform “human intelligence
tasks” (HITs) computers are lousy at
– Examples: identifying items in a photo,
skimming real estate documents to find
identifying information, writing short product
description
– HITs cost from a few cents to a few dollars or
more
“Human Intelligence inside”
Our focus in this class: The Masses – Labor Marketplaces, Games,
Ubiquitous Sensing, Social Networking/Q&A
The Age of the Crowd
• Distributed computing projects: UC Berkeley’s SETI@home?
– Tapping into the unused processing power of millions of individual
computers
• “Distributed labor networks”
– Using the Internet (and Web 2.0) to exploit the spare processing
power of millions of human brains
• Successful examples?
– Open source software: a network of passionate, geeky volunteers
could write code just as well as highly paid developers at Microsoft or
Sun Microsystems
– Wikipedia: creating a sprawling and surprisingly comprehensive online
encyclopedia
– eBay, Facebook: can’t exist without the contributions of users
The Age of the Crowd
• The productive potential of millions of
plugged-in enthusiasts is attracting the
attention of old-line business too
• For the last decade or so, companies have
been looking overseas for cheap labor
• But now it doesn’t matter where the laborers
are, as long as they are connected to the
Internet
The Age of the Crowd
• Technological advances in everything (from
product design software to digital video cameras)
are breaking down the cost barriers that once
separated amateurs from professionals
• Crowds (e.g., hobbyists, part-timers, dabblers)
now suddenly have a market for their efforts
• Smart companies in industries tap the latent
talent of the crowd
“The labor isn’t always free, but it costs a lot less than paying
traditional employees. It’s not outsourcing: it’s crowdsourcing”
Course Information
Lecture Schedule
Labor
Marketplace
Games
Ubiquitous
crowdsourcing
Social
Q&A
Systems
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
W1: Introduction (taxonomy and survey)
W2: Mechanized labor market places (Amazon’s M-Turk, Taskcn)
W3: Using M-Turk for research experiments
W4-5: M-Turk apps: HCI, DB, search
W6: Collaboration, creativity, ethics of crowdsourcing
W7: Task/workflow design
W7: Work quality in the labor marketplaces
W9: Pricing and incentives in the labor marketplaces
W10: Task search and recommendation
W10: Human computation based games (e.g., reCAPTCHA)
W11: Ubiquitous crowdsourcing (smartphones, social networks)
W12: Task planning and incentives in ubiquitous crowdsourcing
W13: Social Q&A systems (Naver KIN, Yahoo! Answer)
W14: Quality and incentives social Q&A systems
W14: Q&A over social networks (Aardvark, Quora)
W15: Real-time Q&A systems
Logistics
• Grading policy
–
–
–
–
–
Mid-term exam: 30%
Quiz: 10%
Critiques: 10%
Course project: 40%
Class participation: 10% (volunteer presentation)
• Office hours:
– M/W: 5:20-6PM
– By appointments
Critique Guidelines
• Provide a summary of the article in your own words (few
sentences)
• Provide a brief description of the results based on your
perspectives; please do not simply repeat the words in the
paper, but try to explain things on your words in a less
formal way
• State the main concept and key contributions (how the
work differs from existing work)
• State the strength of the paper (how well the paper is
formulated/written? what parts do you like? how the paper
improves the state-of-the-arts?)
• State the weakness of the paper (how this paper can be
improved? any follow-up questions to pursue?)
Course Project Guidelines
• A team of two students (or alone)
• Any topic related to human computation and
crowdsourcing is OK
– Project ideas/areas will be also posted at the end of this week
•
•
•
•
Detailed proposal by Sept 23, 2011
Mid-term presentation (progress report)
Final presentation (final’s week)
Final report submission (IEEE double column, <= 8 pages)
by Dec 23, 2011
– Extra 10 points if the paper is submitted to 1st Workshop on
“Social Network Mining and Human Computation” (held at
KAIST in January 2012)
Learning Objectives
• Knowledge
–
–
–
–
Factual knowledge
Theories and principles
Professional skills and viewpoints
Discipline’s methods
• Capabilities
– Thinking and problem solving
– Creative capacities
– Effective communications
• Personal development
– Self-reliance, self-discipline
– Interests, talents, values, etc.
– General liberal education
Next Class (Wed, Sept. 7, 2011)
• Introduction to human computation and
crowdsourcing (taxonomy and survey)
• Reading assignment:
– “Human Computation: A Survey and Taxonomy of
a Growing Field,” Alexander J. Quinn, Benjamin B.
Bederson, CHI 2011