Transcript Computer
HCI
The Computer
Interacting with computers
To understand Human-Computer Interaction
… need to understand computers!
what goes in and out of computer
devices, paper, sensors, etc
what can the computer do?
memory, processing, etc
Von Neumann Architecture
Timeline
Vacuum Tube
Triode (1908)
Transistor
1947
(silicon,
germanium)
Intel Itanium (Tukwila)
2008: 2 billion transistors
Very Large Scale
Integrated (VLSI)
Circuits; 1970s-(> 1,000 transistors
per chip)
The Computer
A computer system is made up of various elements
Each of these elements affects the interaction
input devices – text entry and pointing
output devices – screen (small&large), digital paper
virtual reality – special interaction and display devices
physical interaction – e.g. sound, haptic, bio-sensing
paper – as output (print) and input (scan)
memory and processing– memory, access, speed of processing,
networks
How many computers …
in your house?
PC
DVD, HiFi, TV
microwave, cooker,
washing machine, fridge
Air Condition
security system
can you think of more?
in your pockets?
PDA
smartphone
digital camera
smart card
electronic car key
USB memory
try your pockets and
bags
Richer interaction
sensors
and devices
everywhere
TEXT ENTRY DEVICES
keyboards (QWERTY et al.)
chord keyboards, phone pads
handwriting, speech
Input (1)
Batch
Large
amounts
Pre-prepared
information
Standardised
data loaded
quickly using automated routines
Used
a lot in the past (punch
cards)
Keyboards
Most common text input device
Allows rapid entry of text by experienced
users
Keypress closes connection, causing a
character code to be sent
Usually connected by cable, but can be
wireless
Keyboards
QWERTY Keyboard (1878)
the official standard of computer
keyboards [ISO9995]
was designed for two finger typing
prevents stuck keys (on a typewriter)
the position of the keys the hands are
held close together helping to cause
Carpal Tunnel Syndrome
can be used without training
is cheap and easy to find
Any other keyboard?
Keyboards (ctd) - layout
Alphabetic layout
keys arranged in alphabetic order
not faster for trained typists
not faster for beginners either!
Dvorak layout
common letters under dominant fingers
biased towards right hand
common combinations of letters alternate between hands
10-15% improvement in speed and reduction in fatigue
But no significant adoption, when compared to QWERTY
Keyboards
Over the years, attempts were made to physically
improve keyboards …
Tilting
Curving
Splitting
Keyboards (ctd) - special keyboards
Maltron left-handed keyboard
Virtual keyboard
Projection keyboard
Chord keyboards
only a few keys - four or 5
letters typed as combination of keypresses
compact size – ideal for portable applications
short learning time – keypresses reflect letter shape
fast – once you have trained
BUT - social resistance, plus fatigue after extended use
NEW – niche market for some wearables
Keyboards
Chord Keyboards
Don’t use an alphabet
Use 4/5 keys
Rely on combination of keys
Ideal for one hand or mobile devices
Can suffer fatigue after use
New application in mobile devices?
Phone keypad and T9 entry
use numeric keys with
multiple presses
2–abc
3-def
4-ghi
5-jkl
6-mno
7-pqrs
8-tuv
9-wxyz
hello = 4433555[pause]555666
surprisingly fast!
T9 predictive entry
type as if single key for each letter
use dictionary to guess the right word
hello = 43556 … ‘hello’ not ‘gdjjm’
note 26 could mean ‘am’ or ‘an’; phone gives those options
Handwriting recognition
Text can be input into the computer, using a pen and a
digesting tablet
natural interaction
Technical problems:
capturing all useful information (path, pressure, etc) in a natural
manner
segmenting joined up writing into individual letters
coping with different styles of handwriting
etc
Used in PDAs, tablet computers, smart white boards …
… leave the keyboard on the desk!
Speech recognition
Most successful when:
single
user – initial training and learns peculiarities
limited vocabulary systems
Improving, but challenges such as:
external
noise interfering
imprecision of pronunciation/articulation e.g.
vagueness, pauses, clearing throat, repetition, etc
large vocabularies
different speakers
POSITIONING, POINTING AND
DRAWING
mouse, touchpad
trackballs, joysticks etc.
touch screens, tablets
eyegaze, cursors
Mouse
Movement of mouse moves screen cursor
screen cursor oriented in (x, y) plane
mouse movement in (x, z) plane …
Since the mouse sits on a desk
moving it about is easy
users suffer little arm fatigue
cursor is small and can thus be manipulated without obscuring
the display
Disadvantage
hand-eye coordination problems for novice users
Mouse (ctd)
Two methods for detecting motion
Mechanical
ball on underside of mouse turns/rotates as mouse is moved
can be used on almost any flat surface
Optical
light emitting diode on underside of mouse
may use special grid-like pad or just on an appropriate surface
detects fluctuating alterations in reflected light intensity to
calculate relative motion
less susceptible to dust and dirt
Mouse (ctd)
Footmouse
Motivation
Relevant
foot pedals are used heavily in musical instruments (pianos, organs,
drums) and in mechanical equipment (cars, cranes, sewing machines)
hands-occupied situations
users with disabilities or with high-back or neck problems
Challenges
feet tend to be less precise than hands
practicality of having foot controls in the work environment (e.g. pedals
under a desk may be operated accidentally, etc)
Touchpad
a pointing device featuring a specialized surface that
can translate the motion and position of a user's fingers
to a relative position on screen
operated by stroking a finger over the surface
used mainly in laptop computers
good ‘acceleration’ settings important
fast stroke
lots of pixels per inch moved
initial movement to the target
slow stroke
less pixels per inch
for accurate positioning
Trackball and thumbwheels
Trackball
ball is rotated inside static housing
like an upside down mouse!
separate buttons for picking/selection
very fast for gaming
used in some portable and notebook computers
Thumbwheels
two dials for X-Y cursor position
for fast scrolling – single dial on mouse
Joystick and keyboard nipple
Joystick
pressure of stick translates to velocity of movement
buttons for selection e.g. on top or on front like a trigger
often used for computer games, aircraft controls and 3D navigation
Keyboard nipple or pointing stick
a pointing device typically mounted in a keyboard on laptops
pointing stick can be considered to be a miniature joystick
pointing stick operates by sensing applied force
the velocity of the pointer depends on the applied force
force applied on the pointing stick is echoed on the screen by movements of
the pointer
Touch-sensitive screen
Detect the presence of finger or stylus on the screen.
Increasingly being accepted in the mass market especially in
mobile devices e.g. smartphones and tablets
Advantages:
direct pointing device
more natural gestures through multitouch (i.e. ability to recognize the presence of two
or more points of contact with the screen surface, and therefore supporting more
natural gestures such as pinching and reverse pinching)
fast, and requires no specialised pointer
good for menu selection
suitable for use in hostile environment: clean and safe from damage.
Disadvantages:
finger can mark screen
imprecise (finger is a fairly blunt instrument!)
if screen is vertical, lifting arm can be tiring unless supported
Stylus
small pen-like pointer to draw directly on screen
may use touch sensitive surface or magnetic detection
used in PDA, tablets PCs and drawing tables
pros
very direct
obvious to use
cons
can obscure screen
Skinput: Body as an Input Surface
30
CHI’2010
Use a tiny projector on body to show menus
Microphones to listen to taps on hand/arm
Signal processing and machine learning to
differentiate positions
Pointing Devices
Integrated gaze and face tracking system. A camera and IR
light source tracks gaze by computing the angle between the
corneal reflection and the centroid of the pupil.
Eye gaze or eye tracking
a laser light reflects off the retina
provides a control interface by tracking eye/visual
movement on the screen
e.g. look at a menu item to select it
high accuracy requires headset
cheaper and lower accuracy devices available
mainly used for evaluation of interactive systems e.g. in
order to provide information on clicking and scrolling
patterns of web users
it is also relevant for disabled users
Discrete positioning controls
in phones, TV controls etc
mini-joysticks
(mobile phones)
central cursor pads & discrete left-right, updown buttons (TV remote controls)
e.g.
+/- for controlling volume & navigating channels
mainly for menu selection
Pointing Devices (6)
Smart Systems
Minority Report … reality or fiction?
DISPLAY DEVICES
bitmap screens (CRT & LCD)
large & situated displays
digital paper
Output Devices
Permanent output
Paper
Microfiche
Film
Impermanent output
Screen
Sound
Tactile
Bitmap displays
bitmap display or raster scan
the whole screen is scanned sequentially and horizontally
screen is vast number of coloured dots
Visual Output
Cathode Ray Tube (CRT)
Text based to high resolution
Produce minor health hazards
Since they are radiation emitting
devices!
Liquid Crystal Display (LCD)
Alternative systems might not
use a screen …
Lights
Gauges
Dials
Can you give examples?
Large and situated displays
Large displays
Used for meetings, lectures, etc
Some use gas plasma technology to create large flat bitmap
displays
Where very large screen areas are required, several smaller
screens, either LCD or CRT, can be placed together in a video
wall where each screen displays a portion of the whole (or
separate sub-image) and the result is an enormous image
Situated displays
As well as for lectures and meetings, display screens can be
used in various public places to display information; they could
even be interactive
These are often called situated displays as they take their
meaning from the location in which they are situated i.e. the
meaning of information or interaction is related to the location
PAPER: PRINTING AND SCANNING
print technology
fonts, page description, WYSIWYG
scanning, OCR
Permanent Output
On paper using printers
Impact
Printers (Noisy)
Dot
Matrix (Slow + Low Quality)
Daisy Wheel (Very Slow + 1 Font)
Chain and Band printers (Fast + Expensive)
Non-Impact
Printers (Silent, High Quality, Fast)
Inkjet
Thermal
(Expensive)
Laser (Cost effective + Colour expensive)
Permanent Output
Printers
Speed measured in Pages Per Minutes
Resolution measured in Dots Per Inch (dpi)
Store information in firmware such as
80 dpi (dot matrix) = Near Letter Quality
300 dpi – 1200 dpi (non-impact) = Near Book Quality
Typeface (Fonts)
Character set
Size
More complex printers use Page Description Language (PDL)
Communicates complex commands directly to printer
Used in pdf and postscript
Fonts
Font – the particular style of text
Courier font
Helvetica font
Palatino font
Times Roman font
§´ (special symbol)
Size of a font
This is ten point Helvetica
This is twelve point
This is fourteen point
This is eighteen point
and this is twenty-four point
Fonts (ctd)
Pitch
fixed-pitch
– every character has the same width e.g.
Courier
variable-pitched – some characters wider e.g.
compare the “i” and the “m” in Times New Roman
Serif or Sans-serif
sans-serif
– square-ended strokes
e.g. Helvetica
serif
– with splayed ends (such as)
e.g. Times Roman
Readability of text
lowercase
All
lowercase text considered to be
easier to read than all uppercase
UPPERCASE
better
for individual letters and non-
words
e.g. flight numbers: BA793 vs. ba793
Scanners
Take paper and convert it into a
digital document
Useful in
desktop
publishing for incorporating
photographs and other images
document storage and retrieval systems,
instead of physical paper storage
Optical character recognition
OCR converts bitmap back into text
different fonts
create
problems for simple “template matching”
algorithms
more complex systems segment text, decompose it into
lines and arcs, and decipher characters that way
page format
columns,
pictures, headers and footers
VIRTUAL REALITY AND 3D
INTERACTION
positioning in 3D space
moving and grasping
seeing 3D (helmets and caves)
Positioning and viewing in 3D
Positioning in 3D space
3D mouse: six-degrees of movement: x, y, z + roll, pitch, yaw
data glove: to detect finger position
VR helmets: to detect head motion and possibly eye gaze
whole body tracking: accelerometers, image/video processing, etc
Viewing in 3D
often use stereoscopic vision ( two 2D images are presented separately to
the left and right eye of the viewer, which are then combined in the brain
to give the perception of depth), but also possible to use real 3D display
(displaying an image in three full dimensions)
Simulators and VR caves
scenes projected on walls
realistic environment
real controls
other people
PHYSICAL CONTROLS, SENSORS
ETC.
special displays and gauges
sound, touch, feel, smell
physical controls
environmental and bio-sensing
Sounds
can take many forms
beeps
bongs (deep ringing sounds)
clonks (loud thudding sounds)
whistles
whirrs (like the sound of rapidly vibrating wings)
used for
error indication (e.g. incorrect command/input) or risky
situations (e.g. when deleting files)
confirmation of actions e.g. keyclick
notification of events/status e.g. a new email has arrived,
download has been completed
Haptic devices - example
BMW iDrive
for selecting and controlling menus
one feels little bumps along the way as one maneuvers through
options and menus
makes it easier to select options by feel
uses haptic technology from Immersion Corp
Physical controls
specialist controls needed …
industrial
controls, consumer products, etc.
easy-clean
smooth buttons
large buttons
multi-function
control
clear dials
tiny buttons
Environment and bio-sensing
sensors all around us
car
courtesy light
small
switch on door
ultrasound
security,
RFID
detectors
washbasins
security tags in shops
temperature, weight, location
MediaCup
cup has sensors
heat, movement, pressure
broadcasts state (IR)
used for awareness
user is moving, drinking, …
Han's purpose to drink coffee
incidentally colleagues are aware
OTHER ASPECTS
Compression
reduce amount of storage required
lossless
recover
exact text or image – e.g. GIF, ZIP
look for commonalities:
text:
AAAAAAAAAABBBBBCCCCCCCC
10A5B8C
lossy
recover
something like original – e.g. JPEG, MP3
exploit perception
JPEG:
lose rapid changes and some colour
MP3: reduce accuracy of drowned out notes
Storage formats - media
Images:
many storage formats :
(PostScript, GIFF, JPEG, TIFF, PICT, etc.)
plus different compression techniques
(to reduce their storage requirements)
Audio/Video
again lots of formats :
(QuickTime, MPEG, WAV, etc.)
compression even more important
also ‘streaming’ formats for network delivery
Finite processing speed
Designers tend to assume fast processors, and
make interfaces more and more complicated
But problems occur, because processing cannot
keep up with all the tasks it needs to do
cursor
overshooting because system has buffered
keypresses
icon wars - user clicks on icon, nothing happens, clicks
on another, then system responds and windows fly
everywhere
Also problems if system is too fast - e.g. help
screens may scroll through text much too rapidly
Computer Memory
Short Term
RAM
Long Term
Secondary
Hard
Backing Storage
disk
Disk drive
CD’s
Memory Sticks
Permanent Storage
Cheaper
Slower
Permanent (Non-volatile)
Several types
Different capacities
Magnetic
Optical
Floppies 1.4 Mb
Memory Stick 64 Mb +
CD 640 Mb
DVD 4 Gb – 18 Gb
Blue Ray 15 Gb – 30 Gb
Harddisk 120 Gb + … Tera Bytes
Two writing types
Rewritable (Hard disk)
Write Once Read Many WORM (CD, punch cards)
Recap: Boolean Logic
Boolean Expression
Boolean Circuit
Truth table:
Value of E for every
possible D, S.
TRUE=1; FALSE= 0.
E = S AND D
S
E
D
D
0
S
0
E
0
0
1
1
1
0
0
1
1
0
Truth table has
2 k rows if
the number of
variables is k
Combinational circuit for binary addition?
25
11001
+ 29
11101
54
110110
Want to design a circuit to add any two N-bit
integers.
Is the truth table method useful for N=64?
A Full Adder (from handout)
R-S Flip-Flop
S
R
M
A more convenient form of memory
No
“undefined”
outputs ever!
Fact: “Data Flip-Flop” or “D flip flop”;
can be implemented using R-S flip flops.
“Register” with 4 bits of memory
RAM: Implementing “Write”
Data
Decoder
(Demux)
Clock
RAM
The decoder selects
which cell in the RAM
gets its “Write” input
toggled
(simple combinational
circuit; see logic handout)
K-bit address
(in binary)
Greatly simplified view
of modern CPUs.
Program (in binary)
stored in memory
Memory Registers
Arithmetic and
Logic Unit
(ALU)
Control FSM
Instruction PointerLots of Custom Hardware
RAM
The “symphony” inside a computer
Clock
Combinational
circuit
Memory
Clocked
Sequential
Circuit
(aka
Synchronous
Circuits)
Limitations on interactive performance
Computation bound
Computation takes too long, causing frustration for the user
Storage channel bound
Bottleneck in transfer of data from disk to memory
Graphics bound
Common bottleneck: updating displays requires a lot of effort sometimes helped by adding a graphics co-processor optimised
to take on the burden
Network capacity
Many computers networked - shared resources and files,
multiple access to printers, etc - but interactive performance can
be reduced by slow network speed
Viruses and Worms
Automated ways of breaking in;
Use self-replicating programs
(Recall self-replicating programs:
Print the following line twice, the second time
in quotes. “Print the following line twice, the
second time in quotes.” )
Email Viruses
Infected program, screen saver, or Word
document launches virus when opened
Use social engineering to entice you to open the
virus attachment
Self-spreading: after you open it, automatically
emails copies to everyone
in your address book
The Melissa Virus (1999)
Social engineering: Email says
attachment contains porn site
passwords
Self-spreading: Random 50
people from address book
Traffic forced shutdown of
many email servers
$80 million damage
20 months and $5000 fine
David L. Smith
Aberdeen, NJ
Combating Viruses
Constant battle between attackers and defenders
Example:
Anti-virus software finds “signature” of known virus
Attacker response: Polymorphic virus –
to thwart detection, change code when reproduced
Anti-virus software adapts to find some kinds of
polymorphism
But an infinite number of ways to permute viruses available
to attackers
The Morris Worm (1988)
First Internet worm
Created by student at Cornell
Exploited holes in email servers,
other programs
Infected ~10% of the net
Spawned multiple copies,
crippling infected servers
Sentenced to 3 years probation,
$10,000 fine, 400 hours
community service
Robert Tappan Morris
The Slammer Worm (2003)
Fastest spreading worm to date
Only 376 bytes—Exploited buffer overflow in Microsoft
database server products
Spread by sending infection packets to random servers as fast
as possible, hundreds per second
Infected 90% of vulnerable systems within 10 minutes!
200,000 servers
No destructive payload, but packet volume shut down large
portions of the Internet for hours
911 systems, airlines, ATMs — $1 billion damage!
Patch already available months previously,
but not widely installed
Zombies
Attacker’s
Program
Bot
Bot program runs silently in the background, awaiting
instructions from the attacker
DDOS Attacks
“Distributed Denial of Service”
“Attack www.store.com”
Objective: Overwhelm target site with traffic
Sending Spam
“Forward this message:
Subject: Viagra!
…”
Messages are hard to filter because there are
thousands of senders
Spyware/Adware
Hidden but not
self-replicating
Tracks web
activity for
marketing, shows
popup ads, etc.
Usually written by
businesses: Legal
gray area
Conclusion
HCI depends on computer operating as an
intermediary, achieving appropriate interaction
through a variety of I/O devices
WHAT COMPUTERS
JUST CANNOT DO.
M.C. Escher
Print Gallery
High-level description of program that selfreproduces
Print 0
A
Print 1
.
.
.
Print 0
......
B
......
......
......
}
}
Prints binary code of B
Takes binary string on tape,
and in its place prints (in
English) the sequence of
statements that produce it,
followed by the translation
of the binary string into
English.
How do you represent music?
Score:
Audio samples
Spectrogram
Creating music: Synthesis
Review Questions
What are the most common interaction devices?
What are their limitations?
How can some of them be overcome by novel devices?
Suggest ideas for an interface which uses speech and
sound effectively.
How can sound be used to convey information?
Review Questions
What are some of the factors you might consider when
designing systems like a public access interactive kiosk?
Identify the strengths and weaknesses of multimedia
and hypermedia systems.
Discuss how the speed of access to memory can affect
the speed of an interactive computer system. Which
other factors can affect such speed?
Revision