Good Computing

Ethical Integrity in the Fast-Paced World of Computing Chuck Huff St. Olaf College Yes, I know Marcus Aurelius or Vauvenargues or Chesterton has already said this, and far better; but let’s face it -- you weren’t listening then either.

-Don Patterson, Oct. 2005

Poetry

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Collaborators and Supporters

Student Collaborators •Britta Anderson •Nathan DeWall •Erin Engelbart •Mike Knoll •Cassie Seningen •Joe Stewart •Jenny Ingebritsen •Kristyn Aasen •Craig Enlund •Nicole Gilbertson •Laura Barnard •Lauren Cassat Faculty Collaborators • Simon Rogerson, DeMontfort Univ. Leicester, UK • William Frey , University of Puerto Rico, Mayaguez • Julie Legler , Statistics, St. Olaf College Supporters: • National Science Foundation • Curriculum Based Resources DUE-9980786 and DUE-9972280 • Life Stories of Moral Exemplars in Computing SES-0217298 •

Centre for Computing and Social Responsibility

, Demontfort University, UK • St. Olaf College For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Overview

• Organizational climate and moral decision making by individuals

(the bad news)

• Our standard advice

(the inadequate news)

• Moral exemplars in computing who rise above this context

(the good news)

• Can we create a model to teach virtues?

(spreading the good news)

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Organizational Influence on Moral Decisions

• Organizations control scope and complexity by using “goal based rationality” (Max Weber) • This concentrates attention and energy on the efficient achievement of organizational goals • As the organization concentrates on efficiency, it ignores other values, to its own detriment. • This irrationality increases as narrowness of focus increases For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Individual Response to Organizational Influence

• Milgram’s

obedience study

is a microcosm of the response to organizational pressure • The experimenter requires efficient performance of duty.

• The individual is caught between duty to the experimenter and duty to the victim.

• The punch line: most people obeyed orders For the Linda Hall Library of Science, Engineering & Technology, March, 2007

The Obedience Paradigm

• The

experimenter

instructs a

teacher

to shock a

student

every time the student makes a mistake.

• • • The experimenter would not take no for an answer; As the shocks increased in intensity, the victim went from screaming to eerie silence.

Milgram ran many variations to determine what influenced the teacher’s willingness to continue.

He also observed the teacher’s attempts to cope with the conflict.

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Coping with Immoral Orders

• Two thirds of the teachers chose to continue shocking the victim while the victim: • Protested the pain • Withdrew permission to continue • • • Mentioned a heart condition Screamed to be let out Fell completely silent • • • • • • All the teachers: Protested the victim’s treatment Tried to call off the study Pressed the shock buttons so as to minimize shock Showed empathy for the victim Were worried about the victim’s health Showed clear signs of distress • • • • Many of the teachers: Minimized harm to the victim Distanced themselves from the pain Narrowed their focus to efficient performance of duty Blamed the victim For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Organizational Processes of Moral Compromise

• Organizations

irrationally

ignore other values in favor of efficiency.

• Ordinarily moral employees

immorally

ignore other values in response.

• The technology isolates designers and decision makers from the effects of their decisions.

• Selective recruitment, socialization, and groupthink perpetuate this cycle. • And it is more complicated: multiple moral agendas.

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

The Bad News

(in the worst possible light)

Highly committed, well-meaning, narrowly focused, competent computer professionals, guided by organizational imperatives and isolated from the effects of their design decisions can be induced to commit negligent, immoral, unethical, possibly even evil acts.

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Simplistic Moral Advice

• • •

Adopt moral values Have integrity Exercise willpower

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Simplistic Virtue Advice

•

Practice the Virtues (wisdom, justice, courage, temperance,faith, hope, love)

•

Have integrity

•

Exercise willpower

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

But given what we know about:

•

Organizational Irrationality

•

The power of the situation to influence us And what we don’t know about:

• •

How to “practice” a virtue How to “strengthen” our willpower Both these are inadequate

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

The Good News There are professionals who operate with moral integrity in this environment. We call them moral exemplars.

Who are these people?

How did they become the sorts of people who can resist organizational pressure?

First

, how do we identify and study them?

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Identifying Moral Exemplars

I recruited a panel of computing ethics experts to ground the selection of exemplars.

The panel then agreed on criteria for selection.

Prof. Simon Rogerson, DeMontfort University, UK Prof. Don Gotterbarn , East Tennessee State University, US Dr. Alison Adams , University of Salford, UK Prof. Göran Collste , Linköping University, Sweden Dr. Barbara Begier , Gdansk Polytechnic University, Poland Prof. Barrie Thompson , University of Sunderland ,UK Prof. Jeroen van den Hoven , Erasmus University, The Netherlands.

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Defining Moral Exemplars in Computing

The classic moral exemplars study was Colby & Damon’s (1992) study of exemplars in social service. These were activists in human rights, hunger, poverty, child abuse etc. We began with their criteria, but modified them to suit the domain of computing. The final criteria were: • Either a) a sustained commitment to moral ideals or ethical principles

in computing

, or b) sustained evidence of moral virtue

in the practice of computing

.

• A disposition to make

computing

decisions in accord with one's moral ideals or ethical principles. • A willingness to risk one's self-interest for the sake of one's moral values.

• A tendency to be inspiring to other

computing professionals

and thereby to move them to moral action.

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Recruiting Moral Exemplars in Computing

Each panel member nominated 3-5 potential exemplars in the UK and in Scandinavia (Denmark, The Netherlands, Norway, Sweden). The unanimous panel approved those selected for contact. The sample was expanded by snowball from the interviewees. 36 in UK and 27 in Scandinavia were nominated (total, 63). 35 were contacted. 24 interviews conducted (a 74% response).

Profile of the Sample • 13 had significant experience in academia • 15 had significant experience in industry • 3 had significant experience in government • 11 were in the final decade of career and 4 were retired • 4 were in first decade of career • 9 were female For the Linda Hall Library of Science, Engineering & Technology, March, 2007

The Interviews

Life story interviews

(McAdams, 2001) lasting 3 hours in which they told a series of stories (from 19 to 30) about their professional life.

• Life chapters (and overview) • Peak experience, Nadir experience, Turning point, Earliest memory, Childhood scene, Early career scene, Recent career scene, One other scene • Positive and negative influences • Influential stories about others (four of these) • Positive and negative futures • Value questions (e.g. 3 most important values in system design) For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Who Are They?

• • • • • • • • • •

Simon Rogerson

: Founder of EthiCOMP, first Professor of Computer Ethics

Elizabeth France

: First Data Protection Registrar in the UK

James Towell

: Cambridge Grad, Private Consulting

Steve Shirley

: Early pioneer in business computing

Enid Mumford

: Early pioneer in socio-technical systems

Francis Grundy

: Pioneer in encouraging women in computing

Alan Newell

: Pioneer in developing systems for the handicapped

Alan Cox

: LINUX Pioneer,

Jan Holvast

: Pioneer privacy advocate

Ove Ivarsen

: Developer of the USER Award from LO For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Common Themes

• Most maintained a positive focus • Most cultivated a network of social support • Most cited multiple people as positive influences • None saw themselves as morally extraordinary • All were active problem solvers; problems were often framed as moral/social/technical • All practiced a set of social/technical skills to create solutions to problems • Craftspersons focused on users, reformers focused on systems For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Craftspersons

Designing computing technology towards ethical ends • Drew on pre-existing values in computing • Focused on users or customers who had needs • Took the role of providers of service/product • Viewed barriers as inert obstacles, puzzles to be solved • Believed they were effective in their role Example:

Prof. Alan Newell; James Towell

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Reformers

Changing social systems • Attempted to change organizations and their values • Took the role of moral crusader. • Viewed individuals as victims of injustice.

• Viewed barriers as active opposition. • Believed in the necessity of systemic reform. Examples:

Francis Grundy; Ove Ivarsen

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Rating the stories

• Use of technical expertise • Designing for users or clients Form Craftsperson Index • Perceiving harm to victims • Noticing a need for reform • Taking action for reform • Social Support • Social Antagonism • Effectiveness & Ineffectiveness • Negative and Positive Emotion • Use of social expertise Form Reformer Index For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Finding Craft & Reform Exemplars 2 1 Elizabeth France Francis Grundy Ove Ivarsen Jan Holvast Steve Shirley Simon Rogerson 0 Viiveke Fåk -1 Fred Eisner Alan Cox Andrew McGettrick Enid Mumford Niklas Halberg -1 0 1 James Towell Alan Newell Craft Theme Index (X=0, SD =1) 2 For the Linda Hall Library of Science, Engineering & Technology, March, 2007

From Exemplars to Virtues

• There are different kinds or expressions of moral exemplars. • There are likely more than two kinds.

• The virtues they express are a result of a complex mix of the technical, social, and personal.

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Pritchard’s Engineering Virtues

Based on interviews with exemplars in engineering. Though they would say little about their own virtues, they were vocal about what they hoped for in colleagues: • Responsibility • Honesty • Justice (fairness) • Articulateness • Perseverance • Loyalty • Cooperativeness • Creative Imagination • Habit of Documenting Work • Civic-Mindedness • Courage • Openness to Correction • Commitment to Quality • Integrity But the list is likely to be different for different moral ecologies, professions, & cultures. There may be no canonical list: (What do all saints have in common?) For the Linda Hall Library of Science, Engineering & Technology, March, 2007

External Personality A Four Component Model of Virtue Moral Ecologies Moral Skill Sets Integration of Morality into the Self System Internal Low Mutability High For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Influences on the expression of virtues (1 & 2)

Personality • Extraversion (

r

reform,

p

= .026) = .45 with • Conscientiousness • Neuroticism • Agreeableness • Openness to Experience (

r

= .39 with craft,

p

= .059) • Others?

Moral Ecology • Complex overlapping ecologies • Diversity both within and between ecologies • National culture • Organizational culture • Professional culture • Support networks • Other micro-climates For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Influences on the expression of virtues (3)

Integration of Morality in the Self System Affiliations & Relationships Stories & Defining Memories Possible Selves Elements of the temporally extended and contextually distributed self Personal Projects Roles, Life Tasks, Motives & Strivings Beliefs, Attitudes, & Values Past Behavior & Experience Adapted from McGregor & Little (1998). For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Influences on the expression of virtues (4)

These skill sets undergird the work of our moral exemplars. Both social and technical skills predict effectiveness in our exemplar’s stories. • • • •

Moral Imagination

: projecting oneself into the perspective of others.

Moral Creativity

: generating solutions to moral challenges while responding to multiple constraints.

Reasonableness

: Gathering relevant evidence, listening to others, giving reasons, changing plans/positions based on reason.

Perseverance

: planning moral action and responding to unforeseen circumstances while keeping moral goals intact.

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Influences on the expression of virtues (4)

These skill sets include a variety of skills, supported by intermediate and basic knowledge. •

Skills:

constructing the relevant stakeholders in a socio-technical system; data collection about stakeholders; understanding stakeholder perspectives; identifying value conflicts; constructing solutions under constraint; engaging in reasoned dialogue; revising plans under constraint •

Knowledge:

domain specific (e.g. privacy, safety, access, intellectual property); socio-technical systems

;

methods of data collection and argument; technical knowledge of constraints and opportunities; ethical argument;ethical dissent and whistleblowing For the Linda Hall Library of Science, Engineering & Technology, March, 2007

How to acquire these?

• Narvaez & Lapsley (2005) draw on the work in expertise to suggest that practical wisdom should be learned in structured environments that • Reward correct solutions • Match explicit theory and strategy with practice and coaching • Provide extensive, focused practice

We learn a craft by producing [its] product; … we become builders by building

. Aristotle, Nicomachaen Ethics, Book II For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Teaching the Virtues in the Computing Professions Prepare for these External Moral Ecologies Teach these Personality Be aware of this Moral Skill Sets Integration of Morality into the Self System Influence this Internal Low Mutability High For the Linda Hall Library of Science, Engineering & Technology, March, 2007

Good Computing

• There are practitioners of good computing.

• Good computing is not a narrow straightjacket: Exemplars come in a variety of kinds and this diversity may be a strength.

• There are multiple influences on the ability to express the virtues in the computing professions.

• There are skill sets that these practitioners employ.

• These skill sets can be learned (and taught).

• This does not guarantee good computing, but it makes it more likely.

For the Linda Hall Library of Science, Engineering & Technology, March, 2007

References

Aristotle (1941) Ethica Nicomachea. In R. McKeon (ed.)

The basic works of Aristotle

, (pp. 927 – 1112). New York: Random House. Caprara, G.V., Barbaranelli, C., Pastorelli, C. & Perugini, M. (1994). Individual differences in the study of aggression.

Aggressive Behavior, 20

, 291-303. Carver, C. S. (2005). Impulse and Constraints: Perspectives from personality psychology, convergence with theory in other areas, and potential for integration.

Personality and Social Psychology Review, 9

, 312-333.

Cawley, M.J., Martin, J.E., & Johnson, J. A. (2000). A virtues approach to personality.

Personality and Individual Differences

, 28, 997 1013.

Colby, A., & Damon, W. (1992).

Some do care: Contemporary lives of moral commitment

. New York: Free Press.

Crisp, R., & Slote, M. (eds.) (1997).

Virtue Ethics: Oxford readings in philosophy

. Oxford, UK: Oxford Press. Fischman, W., Solomon, B., Greenspan, D., & Gardner, H. (2004).

Making good: How young people cope with moral dilemmas at work

. Cambridge, MA: Harvard Press.

Huff, C. W., & Brown, R. (2004). Integrating ethics into a computing curriculum: A case study of the Therac-25. In A. Akera & W.

Aspray (Eds.),

Using history to teach computer science and related disciplines

. (pp. 255-277) Washington DC: Computing Research Association. Huff, C. W., & Frey, W. (In press). Moral Pedagogy and Practical Ethics.

Science and Engineering Ethics

.

Little, B.R. (1983) Personal Projects: a rationale and method for investigation.

Environment and Behavior, 15,

273-309.

Markus, H, (1977). Self-schemata and process information about the self. students.

Personality and Social Psychology Bulletin, 27 Journal of Personality and Social Psychology, 35,

63-78.

McAdams, D. P., Reynolds, J., Lewis, M., Patten, A. H., Bowman, P. J. (2001). When bad things turn good and good things turn bad: Sequences of redemption and contamination in life narrative and their relation to psychosocial adaptation in midlife adults and in (4), 474-485.

McCrae, R.R., & Costa, P.T. (1999). A five-factor theory of personality. In L.A. Pervin & O. P.Jphn (Eds.).

Handbook of personality: Theory and research

. New York: Guilford.

McGregor, I., & Little, B.R. (1998). Personal projects, happiness, and meaning: On doing well and being yourself.

Journal of Personality and Social Psychology, 74

, 494-512.

Milgram, S. (1983).

Obedience to Authority: An Experimental View

. New York: Harper/Collins.

Narvaez, D., & Lapsley, D. (2005). The psychological foundations of moral expertise. In D. Lapsley & Power, C. (Eds.),

Character Psychology and Character Education

. Notre Dame, IN: University of Notre Dame Press.

Oliner, S. P., & Oliner, P.M. (1988).

The altruistic personality

. New York: Free Press.

Peterson, C. & Seligman, M.E.P. (2004).

Character strengths and virtues: a handbook and classification

. New York: Oxford Press.

Pritchard, M. (1998). Professional responsibility: Focusing on the exemplary.

Science and Engineering Ethics, 4(2

), 215-234. Rest, J.R., & Narvaez D. (Eds) (1994).

Moral development in the professions: psychology and applied ethics.

Hillsdale (NJ): Elrbaum.

Weber, M. (1904/1930).

The Protestant Ethic and the Spirit of Capitalism

. Translated by Talcott Parsons. New York: Charles Scribner's Sons.

For the Linda Hall Library of Science, Engineering & Technology, March, 2007