Transcript 'The Hiring Problem' Academic HCI vs. the Real World of

"The Hiring Problem" :
Academic HCI vs. the Real World of
Practice
Lynn Cherny
(a talk given at UW in 2004, while I
was at Adobe)
7/16/2015
Lynn Cherny
1
Contents
• My background
• Industry Today
• Usability and Design
• We need HCI Executives!
• The Professionals: Contractors vs. In-House folks
• Interviewing for HCI folks
• Example job ad and interview contents
• But is it the real job or the advertised job?
• HCI in Education and Industry Preparation
• Please, Teach the Design/HCI Students … [A Big List!!]
• And Teach…
• Your Research Students
• Possible Lessons for Faculty
7/16/2015
Lynn Cherny
2
My background
•
•
•
•
•
HCI Research at AT&T Labs
Excite, Senior UI designer
TiVo, UI designer & then manager
Axance (consulting), Dir of Methodology
Adobe, Senior UI Designer for crossproduct issues
• The Mathworks (current), Principal
Usability Specialist
7/16/2015
Lynn Cherny
3
Industry Today: Usability vs. Design
• Usability Engineering: Evaluation, field work, lab
work
 often highly-educated voices stuck in the lab
ghetto.
• Interface Designers: A variety of backgrounds,
doing a lot of different jobs; not all from HCI.
• Strong’s NSF report (1995): researchers, research
professionals, converts from “soft disciplines,”
ergonomists, converts from computer disciplines,
graphic designers…
7/16/2015
Lynn Cherny
4
Industry Today: We Need Designers and HCI
Executives
“The rise in the dependence of HCI on usability labs is basically a regression.
Design is where the action is.”
-- Stu Card, CHI 2002
“We do not create anything of substance: we are critics. The Design profession
flourishes because they do things, they create. We must become designers.” –
Don Norman, CHI 2002
“If you want to be the low status, low on the totem pole person in your company,
then yeah, rejoice in the fact that you are hiring user testers. User testing is not
where the action is. The action is with those people who decide what product
to build in the first place. That isn't the user tester community, but it should be
the CHI community.” –Don Norman, interactions (2000)
 Design skills will help break the glass ceiling on promotion for HCI
professionals. We want more designers (and executives) coming from solid
HCI backgrounds.
7/16/2015
Lynn Cherny
5
Industry Today: Who are the
“Professionals”?
•
Mantei and Hewett provided a high-level analysis into the professional
categories at work in HCI:
1. “researcher,”
2. “profesionally oriented researcher,”
3. “research oriented professional,”
4. “professional”
•
What I see:
•
Consultants: The senior folks you see and hear from most (from CHI, UPA, etc.)
•
In-House permanent staff (the “professional” category):
•
•
Go to fewer conferences
Are much younger and very often somewhat alienated from the “profession” – nothing
new, heard it all before, nothing will benefit me there…
• Don’t belong to ACM/Sigchi, don’t read the articles (or write)
• Don’t know the theory, quite often
• Do far less original design work than you’d expect!
 Let’s assume these folks are your students
7/16/2015
Lynn Cherny
6
Interviewing: Example Job Ad
Responsibilities: Works with multidisciplinary software development teams to
help understand user needs, specify usability requirements, and verify
requirements and needs are met. Conducts user research, including contextual
inquiry. Facilitates user-based requirements discovery and user-centered
design. Plans and conducts lab and field usability tests. Analyzes, reports,
and presents results and recommendations to development team and
product management.
Qualifications & Experience:
· 4-Year degree or Graduate degree in Computer Science, Software
Engineering, Cognitive Psychology, Human Factors, or related field.
· Practical knowledge and experience in usability engineering principles and
methods and familiarity with various prototyping tools and techniques.
· Strong communication, interpersonal, and organizational skills.
· Familiarity with chemistry and the pharmaceutical industry helpful.
7/16/2015
Lynn Cherny
7
Interviewing: Hiring a UI
Designer
• Traditional questions: “describe a difficult team
interaction around a design and what you did to resolve
it”
• Portfolio work review (how much is theirs?)
• Design critique
• Design problem – whiteboard
• General vocabulary assessment: “UCD,” “prototype,”
“flowchart,” “usability,” “iterative”…
 It is famously difficult to hire “good designers.” The
question is, do we really need to worry about this?
7/16/2015
Lynn Cherny
8
Interviewing: But is it for the
Advertised Job or the “Real” Job?
• “Evangelism” often comes up in ads/interviews – a warning sign of
dysfunction
• Note that many designers’ portfolios don’t show “their work” – but are
they really trying to mislead you?
• Not everyone knows what the real job consists of (at hiring and
performance eval time)
• We interview in a self-deluded state – how we want it to be (or to
evolve) vs. how it really is; hence we hire on the wrong criteria
sometimes.
 Exactly how important is design talent for success? Possibly less so
than a complete skill set for getting design DONE.
 Case Study Example: Hiring Sally, a “good designer”
7/16/2015
Lynn Cherny
9
HCI and Education for Practice
• SigCHI was 68 people in 1982, over 6000 in
2002; well-known conference rejection rate 
and many satellite conferences
• Much interest internally in status of the
profession and HCI education:
• Strong’s NSF Report (1995),
• Hewett, T. T. et al Curricula in HCI (1992),
• Perlman & Gasen’s The ACM SIGCHI HCI
Education Survey (1993), etc.
• multiple panels at CHI…
7/16/2015
Lynn Cherny
10
HCI and Education: Industry
Preparation? A known problem.
It’s difficult to address needs of industry in HCI curricula because:
• HCI itself is expanding, parts are ill-defined, and we don’t have
a good understanding of what some topics have to offer
industry,
• “Industry” is a collective term for a wide range of different
activities done by people whose needs vary considerably
• When you talk to people in industry they usually don’t know
their own needs and they certainly don’t know what their
colleagues needs are, many don’t even know what most of their
colleagues do.
--Jenny Preece, CHI 94 on “Is HCI Education Getting a Passing Grade from
Industry?”
7/16/2015
Lynn Cherny
11
HCI and Education: Supposed Activities of
Practice (Strong’s NSF Report, 1995)
7/16/2015
Lynn Cherny
12
HCI and Education:
Reality of Practice
From Strong’s NSF Report in interactions (1995):
“Many HCI practitioners experience identity
conflicts within the first few years of entering an
applied position. In general, they must reorient
their views of how they work, what questions they
ask, and what about their work is valued.”
 Can we improve their experience by better
preparation?
(Internships aren’t sufficient.)
7/16/2015
Lynn Cherny
13
Teach them… The Language of Business &
Marketing instead of HCI & UCD
Don: Why do you think [marketing has] so much of a role
determining the products? It's not because they're brighter,
and it's not because they have any more truths. It's because
they know how to play the game better. What I suggest is
that it's time we learn how to play the game.
Janice: The advantage that marketing has is that it's a wellknown concept. Executives have typically gone to business
school where marketing is well-known. One of the
problems that we have is that we don't have usability
engineering and HCI in most curricula for business and
engineering.
Don: One reason we don't is that we don't talk the language of
business.
Don Norman and Janice Rohn, Interactions, Volume 7, Number 3 (2000), Pages 36-60
7/16/2015
Lynn Cherny
14
Teach Them… Product Design instead of
Interface Design, If You Can!
Think about the big picture: building and selling a
product, not doing just the UI design
• From market research, write a business plan
• Identify the crucial factors to distinguish your product
in the marketplace
 Consider that usability alone doesn’t motivate
intent to purchase. Study some business cases and
learn what makes something sell – this will give
perspective and credibility and reduce their
anxiety tremendously!
7/16/2015
Lynn Cherny
15
Teach Them… Usability
Methods
Produce designers who know usability evaluation methods:
• We’ve won the battle of the usability evaluation at most
companies – this is a needed job skill now.
• But teach them to make strong recommendations from user
problems, that are credible, grounded, realistic.
 All field methods, surveys, focus groups, usability test
design, basic statistics, interviewing, content analysis…
 Researchers who want to make an impact on product
definition should learn how to “disguise” themselves as
designers till they can achieve the impact they want.
7/16/2015
Lynn Cherny
16
Teach them… How to Determine
Appropriate Study Methods
Propose the appropriate diagnosis method to
get at useful data for a design problem
•
•
•
•
What usability technique will get useful data?
What type of data will be wanted/understood?
What arguments are needed for it?
Guard against the naive proposals: “Let’s just
ask users on the beta list.”
 Consulting is therapy for dysfunctional
companies.
7/16/2015
Lynn Cherny
17
Teach Them… To Characterize their
Customers
Marketing thinks in “segments” –
it’s our job to define and
identify who will really be
using the products.
 Be able to get the data, or make
a reasonable guess and review
with team. The design should
be tied to the user/customer
profiles, this will save much
pain!
 Don’t go through the motions;
if it’s not in a form that’s
obviously useful, you’ve done
something wrong.
7/16/2015
Lynn Cherny
18
Teach them… To Identify and
Characterize Problems
To identify and summarize the key problems to solve – in any
domain!
• By what criteria
• According to what evidence
• Identify the causes of the problem that require action– break
it down usefully.
 Stated concisely:
“Inconsistent color” is a user complaint – but the operational problem is
not in the same terms.
 Skill that requires strong analysis and abstraction skills,
backed up with excellent communication ability.
 Teach them how to THINK through an issue!
7/16/2015
Lynn Cherny
19
Teach them… To Review and Critique
Designs
To critique designs and find improvements, both
minor and major.
 Most practical UI design is incremental
improvement; and new work is always relative to
existing products in some way. Don’t skip this
step, it seeds new ideas.
 Design test at interviews often checks on this:
7/16/2015
Lynn Cherny
20
Teach Them… Different Kinds of
“Design”
Expose them to many kinds of design: visual, industrial, IA
(web), interaction, object-oriented software design,
patterns, functional specification, chart/table, document
layout, experimental design…
• They all have different types of deliverables – consider a
project with several of them, and make them distinguish
their contributions and boundaries/limits
• Not everyone is equally good at each, but learning limits
now is better.
 Try brainstorming and use methods to unlock creativity
7/16/2015
Lynn Cherny
21
Teach them… Design for Multiple
Devices
Consider design for
•
•
•
•
•
•
•
7/16/2015
Mobile devices
Desktop apps
Social/community
Web
TV
Physical objects
Games (several types)
Lynn Cherny
22
Teach Them… To do Fast Low-Fi
Design
To produce fast preliminary designs –
multiple concepts per problem
• With callouts/text explaining concisely the
differences
• Produce multiple formats of low fidelity:
workflow diagrams, task breakdowns,
wireframes, sketches…
 Accept that others will “help” with this and
that’s useful, not a threat.
7/16/2015
Lynn Cherny
23
Teach Them… To Prepare Multiple
Deliverables
To prepare and explain value of all typical
design stage deliverables:
• Workflows, flowcharts, wireframes,
storyboards, high fidelity mockups
• Write specifications in increasing detail,
focusing on the important details at each stage
7/16/2015
Lynn Cherny
24
Teach them… To Recognize the Value of
Multiple Levels of Abstraction
• Describe the user’s
mental model and how
it relates to the UI in
taskflow/vocabulary
• Flowcharts of the UI
experience vs. the
storyboard; taskflow
vs. the data flow…
7/16/2015
Lynn Cherny
Polar Orbit
eccentric
orbit
low Earth
orbit
Geosynchronous
orbit
25
Teach them… To Present Their
Design Work
Make them present their designs to diverse groups:
business, engineering…
•
•
•
•
7/16/2015
With appropriate justification of thought process
With strategic defenses prepared
Orally, but with supporting materials
Make them respond to naive questions and criticisms of
any type
 Consider asking them to present someone else’s
work – it will strengthen the arguments, and require the
author to distill their points; and this won’t provoke so
defensive a response to critique from the naïve.
Lynn Cherny
26
Teach them… To Find and Interpret
Previous Research
Research literature can be useful, but requires interpretation,
summarization, digestion. If they can’t do this, the gulf
between practice and the academy will grow.
 Consider book reports – oral recaps to audiences who are
naïve; make it interesting and relevant to them!
 Sometimes “previous work” is not reviewed research –
don’t forget about online white papers, etc.
 Identify ways to work research strategically and tactically
into design work
 Make them use the ACM digital library!!
7/16/2015
Lynn Cherny
27
Teach them… To Relate In-House
Research to Design
To summarize the important conclusions from
research/evaluations and how they relate to their
design decisions
• Throughout the design process, at all stages!
• This will be a cover-your-ass strategy (for HCI as a
profession) but will also pay off in a better design, if
we’re really doing the right things.
 Gap between field research and design is still
huge! Be an advocate for usability methods!
7/16/2015
Lynn Cherny
28
Teach them… Justify without
Defensiveness
• They must justify their work gracefully.
• Everyone thinks she’s a designer; every decision
will be questioned no matter how minor.
 Note that bad designers can’t necessarily
recognize good designs. (The question is not will
this happen, it’s how insane will it make them.)
7/16/2015
Lynn Cherny
29
Teach them… Software Development
Lifecycle Models
To pay attention to engineering lifecycle concerns, and learn
how to argue for and adapt UCD to work with them (Rapid
Development, Agile and XP, RUP, boxcar…)
Strong (1995): HCI practice can help to inform and improve life cycle
activities. Within such a life cycle, HCI emphasizes iteration and
concrete communication. HCI research has demonstrated that
straightforward activity categories-such as “analysis,” “design,” and
“evaluation’‘ are not separable in practice. … HCI practitioners can
help show the weaknesses of the waterfall model, but not without risk.
The risk is that the somewhat marginalized field of HCI practice may
lose some of its credibility and influence if it is perceived within
organizations as opposing their received wisdom.
7/16/2015
Lynn Cherny
30
Teach Them… How to Interview
Have them interview each other for jobs, and
ask each other portfolio questions –
challenge each other with design tests
 Review resumes with them
Their peers will be hiring them eventually!
7/16/2015
Lynn Cherny
31
Teach them… To Drive Projects
• They should assume that nothing will happen between high
level marketing requirements and engineering
implementation; they’re the connector:
MRD/Biz plan
Design
Implementation
Organize meetings, identify stakeholders, investigate technical
issues, do research if needed, design and get it reviewed.
 But drive decisions!
7/16/2015
Lynn Cherny
32
Teach them… Most Design
Happens in Committees
The challenge is how to have an influence
over the committee – how to prepare the
right materials and arguments.
 Come in having thought about it harder
than anyone else and expecting to present
ideas in some concise form.
7/16/2015
Lynn Cherny
33
Teach Them… Public Speaking
Require presentations –
confidence, poise,
clarity,
extemporaneous
speech
 Start with article/book
reports, move on to
design presentations
7/16/2015
Lynn Cherny
34
Teach them… To Study Their
Own Organization
They must be ethnographers of the
workplace, to understand the best
methods of success and tailor
their message for their audience.
 How do people get promoted?
What gets rewarded? What’s
teamwork? Who’s in charge?
 How can you document the
impact of your own contribution?
7/16/2015
Lynn Cherny
35
Teach them… To be Solid Experts
but not Shrill Evangelists
Teach them to justify in plain language, but not to be pushy.
Evangelism is dangerous and marginalizes over the long
term.
Siegel and Dray (interactions, Aug 2003): Actively demonstrate your identification
with concerns that exceed your discipline and with the company’s larger goals.
Whenever possible, speak a common language rather than the language of
your discipline. All your recommendations should be clearly supported not by
reference to the principles or belief system of your discipline, but by clear
evidence of what is best for the company in a practical way. Consider
reframing what UCD is about, in ways that show its links to larger, shared
concerns—due diligence, product planning, quality, reduced risk, efficiency,
and innovation—and to marketing and business goals like customer
satisfaction and retention, and, ultimately, profitability.
7/16/2015
Lynn Cherny
36
Teach them… Organizational
Planning
Make them plan for team projects:
• How will the team resolve debates?
• How will the team prioritize features vs.
schedule and resources?
• Track action items and evolution of work?
• How do you plan to estimate work across team
and disciplines? Scope work?
7/16/2015
Lynn Cherny
37
Teach them… To Know and Evaluate
Books on Design
Engineers regularly ask for references on how
to do design. (See Jeff Johnson’s panel at
CHI 2002 on books on design)
• Consider Cooper’s books, Johnson’s, Mayhew,
Mullet and Sano, Norman, Tufte, etc.
• Books on requirements analysis that include
usability research are useful recommendations
too
7/16/2015
Lynn Cherny
38
Teach them… The Difference
Between Peers and Resources
There are two kinds of people in organizations—there are peers and there are
resources. Resources are like usability consultants—we go out and we hire them.
We'll hire a consultant or we'll have a little section that does usability and think of it as
a service organization. We call upon them when we need them to do their thing, and
then we go off and do the important stuff. That's very different than peers, where a
peer is somebody I talk to and discuss my problems with, and who helps to decide
upon the course of action. As you get higher and higher in the organization, this
becomes more of an issue. The executive staff talks to the executive staff, and they
have beneath them all this organization, which are their resources that they deploy.
But the big decisions are being made among peers. And it's really important to
advance in the world to be thought of as peers. …Our usability labs are resources
that we call in when we think we have some trouble with usability. We go and spend
a few hours and we worry about the budget. But we don't take it seriously, we only
take peers seriously. And you only get to be peers if you speak the right language
and if you're making a contribution at the level the company cares about...which is
profitability.
--Don Norman, interactions (2000)
7/16/2015
Lynn Cherny
39
And Teach Your Research
Students:
Ben Shneiderman, CHI 2002: “We must recognize
that nothing is so practical as a good theory and
that theory thrives when challenged by practice.
Our goals should include development of
predictive, explanatory, and generative theories
that systematically support the next generation of
innovations.”
 Express the implications of your research clearly
for your colleagues in practice or it will have
limited impact.
7/16/2015
Lynn Cherny
40
Possible Lessons for Faculty…
• Stay in touch with practice, not just via consulting and
conferences
• Build departmental bridges, to business schools in
particular
• Consider offering co-taught “Product Innovation” and
“Product Management” courses to cross-pollinate
• Encourage good software engineering education and
practice as well: concern for architecture, teamwork,
documentation, test plans…
• Evaluate and improve your own teaching with feedback
from former students in professional practice
7/16/2015
Lynn Cherny
41
Some Takeaways
• We have to produce Supermen, Know-it-alls – obviously
difficult.
• Not everyone will be a “good designer,” but this may be
less important than many of the other skills used on the job
• You can’t teach everything – in a pinch, sacrifice some of
the theory (move to advanced classes) and instead teach
how to do research/where to find it, and how to read it.
Plus the practical items I described!
• Usability engineering (of the evaluation sort) is clearly a
valuable and important skillset, BUT:
 By getting more design into this practice, the door to true
and deep impact will open wide.
7/16/2015
Lynn Cherny
42
Thanks for listening!
Lynn Cherny
[email protected]
www.ghostweather.com
7/16/2015
Lynn Cherny
43