Transcript Soarian™ User Interface

1
Interaction Devices

Interaction Devices
 The Standard: QWERTY (or Sholes) Keyboard
 Mobile devices are driving the need for future input devices
 Pointing devices (mouse, touchscreen)
 Gestural input
 Two-handed input
 3-d pointing
 Voice input/output
 Wearable devices
 Whole body involvement
 Niche applications
• Eye-trackers
• DataGloves
• Haptic/force-feedback
• Brain-controlled mouse movement
2
Interaction Devices

Interaction Devices
 Multimodal interfaces
• Combine several modes of input/output
• E.g., voice commands with pointing devices
 Likely direction
• Giving users the ability to switch between modes depending on their needs
• E.g., Driving a car
– Operate navigation systems with touch or voice input
– Invoke visual or voice output based on location (e.g., moving in traffic vs. at
a stop sign)
– DB adjustment for ambient noise
 Context-aware computing
• Sensors:
– Global Positioning System (GPS)
3
Interaction Devices
4
Interaction Devices

Interaction Devices
 Context-aware computing
• Sensors:
– Global Positioning System (GPS)
– Cell-phone sources
– Wireless connections
• Make detailed information available about the users surroundings
– Restaurants, gas stations
– Museum visitors or tourists
– Auto-connect to a printer based on room location
– Yelp
• Privacy issues
5
Interaction Devices

Keyboards and Keypads
 QWERTY
• Beginners approximate 1 keystroke per second
• Average office worker is 5 keystrokes per second (50 words per minute)
6
Interaction Devices

Keyboards and Keypads

Potential for higher rates of information input
• Courtroom recorders (300 words per minute)
• Piano Keyboard
– Allows several finger presses at once
– Responsive to different pressures and durations
• Chord keyboards
• One-handed keyboards
– Useful for tasks requiring one hand manipulation of an object
7
Interaction Devices

Keyboards and Keypads
 Reduction of ulnar abduction and pronation
 Ulnar tunnel syndrome
• Ulnar tunnel syndrome is caused by pressure on the ulnar nerve at the wrist
• This nerve is found on the pinkie-finger side of the wrist
8
Interaction Devices

Keyboards Layouts
 QWERTY
• Keep frequently used letters apart,
• Slow down users to avoid key jamming
 Dvorak
• Reduces finger travel time
• Increases speed, reduces errors
9
Interaction Devices

Keyboards Layouts
 Number Pads – phone pads versus calculator pads
• Most computer keyboards use the calculator layout
• Performance is slightly better with the phone layout
10
Interaction Devices

Keyboards Layouts
 Keyboard for those with disabilities
• Combination of small hand movements and small finger
presses selects the letters and controls the cursor
• No finger or wrist movement is needed
• Helpful to users with carpal tunnel syndrome or arthritis
• Each dome slides into one of eight zones to type a
character
• Either dome can slide first or move both at the same
time.
• Domes slide toward the center of their color or character
zones. (not directly at the characters)
• Slide the right dome to the zone of the character you
wish to type; slide the left dome to the color of that
character.
11
Interaction Devices

Keyboards Layouts
 Dasher
• Predicts probable characters and words as users make their selections
• Continuous two dimensional pointing with a mouse, touchpad or eye-tracker
• http://www.youtube.com/watch?v=K6jnY2bNIzw
12
Interaction Devices

Keys
 Modern keyboards
• ½ inch square
• Slightly concave surfaces
• Matte finish
• 40 to 125 gram force
• 3 to 5 millimeters of displacement (when the key is pressed far enough to send a
signal, both tactile and audible
• Larger keys (ENTER, SHIFT, CTRL)
• Keys with indicators (CAPS LOCK, NUM LOCK)
• Combination Keys (Ctrl-V)
• Cursor movement keys
• Other special keys (HOME, PAGE UP, END, PAGE DOWN)
• Number Pad
13
Interaction Devices

Keyboards and Keypads for Small Devices
 Foldable Keyboards
 Virtual Keyboards
14
Interaction Devices

Keyboards and Keypads for Small Devices
 Mobile Phones and Devices
• Multi-tap to select a letter
• Word prediction
• Palm Graffiti
15
Interaction Devices

Pointing Devices
 Avoids learning commands
 Reduces the chances of typographical errors
 Keeps the user’s attention on the display
 Operations
• Select
• Position (e.g., move)
• Orient (e.g., rotate)
• Path (e.g., curving line in a drawing program)
• Quantity (e.g., numeric selection)
• Text (e.g., enter, move, edit)
 Direct Control Devices
• Examples: Light Pen, Touch Screen
 Indirect Control Devices
• Examples: Mouse, Trackball, Joystick
16
Interaction Devices

Pointing Devices
 Criteria for success
• Speed and accuracy
• Learning time
• Cost and reliability
• Size and weight
 Example: Touch screen
• Gestures
• Touch screen is the only input that has survived the
Disney theme parks
 Indirect Control Pointing Devices
• Eliminate hand fatigue and hand obscuring the
screen of direct input
• Required more cognitive processing and eye/hand
coordination
• Examples: Mouse, trackball, joystick, graphics tablet
17
Interaction Devices

Comparison of Pointing Devices
 Light pen and touch screen (fast but inaccurate)
 Mouse (fast and accurate)
 Keyboard entry in sometimes faster than the mouse (e.g., data entry)
 Joysticks and trackballs are often preferred over mice by users with motor
disabilities
 Alternative keyboard entry should be provided
 Touch screen and trackball are durable for public access
 Mouse, trackball, graphic tablet and touch pad are good for pixel level pointing
18
Interaction Devices

Fitts’s Law
 Predicts the amount of time to point at an object
 Helps to decide the location and size of buttons
 Helps to decide on the types of pointing devices as a function of task
 The time for hand movements was dependent on the distance users had to
move (D) and the target size (W)
 Doubling the distance took longer, but not twice as long
 Increasing the target size enabled the user to point at the object more rapidly
MT = a + b log2(D/W + 1)
a = approximates the start/stop time in seconds for a given device
b = inherent speed of the device
if a = 300 msec, b = 200 msec/bit, D = 14 cm, W = 2 cm then
MT (movement time) = 900 msec
19
Interaction Devices

Fitts’s Law

The farther away a target is, the longer it takes to acquire it with the mouse

The smaller a target is, the longer it takes to acquire it with the mouse

The inverse of both statements is true as well (closer and bigger targets can be
more quickly acquired)

There are two main ways to improve mouse efficiency: put the controls closer,
or make them bigger

As monitors have gotten bigger and screen resolutions have increased, Fitts'
law dictates that actual mouse efficiency has gone down

In other words, the same button takes much longer to click than it did fifteen
years ago

Office 2007 and Fitts' Law
•
Most controls in the Ribbon are labeled. This helps discoverability and usability
considerably, but it also makes the buttons bigger and easier to target.
20
Interaction Devices

Fitts’s Law

The Mini Toolbar was designed with Fitts' Law in mind as well. Whenever you
select text or right-click selected text, a small toolbar appears directly next to
the mouse cursor. As you move closer to it, it fades in; as you move away, it
fades out.

The controls on the Mini Toolbar are small, but because they're located directly
next to your cursor, they're easy to target
21
Interaction Devices

Fitts’s Law

Mile-High Menus and Magic Corners
•
One of the most useful aspects of applying Fitts' Law to computers is that screen
size is bounded. No matter how far you move your mouse to the left, the cursor will
never go farther than the left side of the screen
•
Because the Quick Access Toolbar is in the title bar, the buttons are effectively
infinitely tall. You can target and click each of the buttons very quickly; they're a
"mile high."
Office Button
22
Interaction Devices

Novel Devices
 Foot Controls – Foot Mouse (takes twice as long as a hand operated mouse)
 Eye-tracking – still mostly a research tool
 Data Glove
 Haptic Feedback – moving a control (e.g., a mouse) and feeling resistance
 Bimanual Input – non-dominant hand makes a coarse selection, the dominant
hand performs a precise movement
 Ubiquitous Computing
• Embeds sensing technologies in a room
• The user wears badges so the sensors detect the presence of a user
• Application: tracking patients in a hospital
23
Interaction Devices

Novel Devices
 Digital Pen (by Logitech)
• Records the strokes written on digital paper
• When the pen is placed back in its holder, the data is transferred to the computer
 Digital Microscopes
• Stores magnified pictures and annotations
24
Interaction Devices

Speech and Auditory Interfaces
 The vision of computers chatting with users may be more of an uninformed
fantasy than a desirable reality
 Voice commanding is more demanding of user’s working memory than is
hand/eye coordination
 Speech requires use of limited resources, while hand/eye coordination is
processed elsewhere in the brain, enabling a higher level of parallel processing
 Planning and problem solving can proceed in parallel with hand/eye
coordination, but they are more difficult to accomplish while speaking
 Variations related to speech and audio technologies
• Discrete-word recognition
• Continuous-speech recognition
• Voice information systems
• Speech generation
• Non-speech auditory interfaces
25
Interaction Devices

Speech Systems
 Potential uses
• Users with vision impairments
• When the user’s hands are busy
Siri
http://www.youtube.com/watch?v=
EySAW1C60is&feature=related
http://www.youtube.com/watch?v=
0ZeJwWVbx5U&feature=related
• When mobility required
• When the speaker’s eyes are occupied
• When harsh or cramped conditions preclude use of a keyboard
 Technologies
• Speech store and forward
• Discrete-word recognition
• Voice information systems
• Speech generation
 Issues with speech recognition
• Increases cognitive load when compared to pointing
• Speaking commands or listening disrupts planning and problem solving
• Interference from noisy environments
• Unstable recognition across changing users, environments and time
26
Interaction Devices

Speech Systems
 Issues with speech output
• Slow pace of speech output when compared to visual displays
• Ephemeral nature of speech
• Difficulty in scanning/searching
27
Interaction Devices

Discrete-word recognition
 Recognition of individual words
• Accurate 90% to 98% for 100 to 10,000 word vocabularies
• Speaker-dependent training (the user trains the system)
• Speak-independent (not as accurate)
• Keyboards and pointing devices are more rapid, and actions and command are more
visible for easy editing
• Error handling is difficult and slow
 Continuous-speech recognition
• Difficult to recognize the boundaries between spoken words
• Problems with accents, variable speaking rates, disruptive background noise, and
changing emotional conditions
• Fields with distinct terminology may be able to use this technology due to the
limited and precise vocabulary
28
Interaction Devices

Voice information systems
 Interactive Voice Response (IVR)
• Examples: Help support, banking, ordering, voice mail
 Challenges
• Complex, deep structures
• Slow pace of voice output
• Difficulty scanning
 Apple’s iPod
• Management of large audio databases
• Retrieval of specific segments
29
Interaction Devices

Speech Generation
 Automobiles systems (e.g., “Turn left onto Route 896)
 Control rooms (e.g., “Danger, temperature rising”)
 Privacy related to audio messages
 Is the human voice a better means of communication (e.g., Atlanta Airport)?
 Are tones sometimes better than voice (e.g., “Headlights on”)
 Microsoft’s Windows Narrator (reads text display on screens)
 Voice tagging of web pages (VoiceXML)
 Googles TalkBack

Non-Speech Auditory Interfaces
 MIDI (Musical-instrument digital interfaces) for music composition
 Auditory icons – distinctive familiar sounds
 Earcons – abstracts sounds whose meanings must be learned
 Example Earcons
http://upload.wikimedia.org/wikipedia/commons/
0/07/Emergency_broadcast_system.ogg
http://www.youtube.com/watch?
v=IvEwOfL21Uo
30
Interaction Devices

Displays – Small and Large
 The primary source of feedback to the user
 Physical dimensions (diagonal)
 Resolution (number of pixels)
 Number of available colors
 Luminance, contrast and glare
 Power consumption
 Refresh rates
 Cost
 Reliability
31
Interaction Devices

Displays Technologies
 Raster-scan cathode-ray tubes (CRTs)
• Similar to televisions
• Electron beams sweeping out lines of dots to form letters and graphics
 Liquid-crystal displays (LCDs)
• Voltage changes influence the polarization of tiny capsules of liquid crystals
32
Interaction Devices

Displays Technologies
 Plasma display panels (PDPs)
• Rows of horizontal wires are slightly separated from vertical wires by small glassenclosed capsules of neon-based gases
• When the horizontal and vertical wires receive a high voltage, the gas glows
 Light-emitting diodes (LEDs)
• Certain diodes emit light when a voltage is applied
• The curved display in New York’s Time Square
33
Interaction Devices

Displays Technologies
 Electronic ink
• Paper-like resolution (80 to 200 dpi)
• Uses tiny capsules containing negatively charged black particles and positively
charged white particles
34
Interaction Devices

Displays Technologies
 Smart Phone penetration
35
Interaction Devices

Displays Technologies
 Large Displays
• Informational wall displays (control room usage)
36
Interaction Devices

Display Technologies
 Large Displays
• Interactive Wall Displays
• SMART board
37
Interaction Devices

Display Technologies
 3D Smart Phones
 Visual Strain?
38
Interaction Devices

Display Technologies
 Perceptive pixel on CNN
 Check this out: CNN Perceptive Pixel
 Notice
• Gestures
• Resizing capabilities
• Selection capabilities
 A world of Glass
 http://www.youtube.com/watch?v=bBjvqnKQsTI&feature=related
39
Interaction Devices

Display Technologies
 Multiple desktop displays
 Six 19” Monitors
 Facilitate side-by-side comparisons of documents, software debugging,
information visualization and analysis
40
Interaction Devices

Display Technologies
 Heads-up and helmet-mounted displays
• Project information on a partially silvered windscreen of an airplane cockpit or car
• Operators can remain focused on the surroundings while receiving computer
generated information
 Mobile device displays
• Making phone calls
• Texting
• Checking appointments
• Reading email
• Finding a locations
• Obtaining directions
• Listening to music
• Internet browsing
http://www.youtube.com/watch?v=T5YOffkw0_I
8 minutes into video
41
Interaction Devices

Display Technologies
 Printers
• Speed
• Quality
• Cost
• Compactness
• Quietness
• Type and size of paper
• Support for special formats
• Reliabilty
http://www.youtube.com/watch?v=KGKJQ2WMT
dY