Transcript No Slide Title

Getting Physical
Saul Greenberg
Concepts
Ubiquitous Computing
Tangible Media
Foreground interaction
Ambient displays
Collaborative interaction
Physical controls
Ubiquitous Computing
Mark Weiser (Xerox Parc):
• A less-traveled path I call the invisible; its highest
ideal is to make a computer so imbedded, so fitting,
so natural, that we use it without even thinking
about it.
• Provide hundreds of wireless computing devices per
person per office, of all scales (from 1" displays to
wall sized)… It is invisible, everywhere computing
that does not live on a personal device of any sort,
but is in the woodwork everywhere.
Extracted from Mark Weiser’s UbiqCom web site
Ubiquitous Computing
Invisible Everywhere Computing
• invisible:
o tiny, embedded, attachable…
• everywhere:
o wireless, dynamically configurable, remote access, adapting…
Extracted from Mark Weiser’s UbiqCom web site
Ubicomp is Situated Computing
Makes use of simple shared
context
•
•
•
•
space
time
proximity
affordances
Participation in the context
• is physical
• is out here with us
• is in many small and large
places, including trivial ones
Extracted from Mark Weiser’s UbiqCom web site
The Major Trends in Computing
Extracted from Mark Weiser’s UbiqCom web site
Technology trends for Ubicomp
Displays
• very small (inches) to very large (walls)
Processors:
• cheap, small, dedicated, microprocessors
Low Power
• small batteries, solar (?)
Wireless
• Wireless ethernet, infrared, mobile standards, Bluetooth (inroom), in-building, metropolitan
Operating systems
• Linux on a chip, Windows CE, …
Packaging
• non-conventional devices
Modified from Mark Weiser’s UbiqCom web site
ParcTab
Mobile hardware
• infrared
• room-sized cells
• location information
Device
•
•
•
•
•
small case with belt clip, ½ size of PDAs
touch sensitive 128x64 pixels display
3 finger-operated mechanical buttons (chorded)
piezo-electric speaker
low power needs (~ 1 week between charges )
Can be used in either hand
• rotates display
Extracted from Mark Weiser’s UbiqCom web site
Tangible Media
Hiroshi Ishii (MIT)
Tangible Bits
• gives physical form to digital information, seamlessly coupling
the dual worlds of bits and atoms.
Tangible User Interfaces
• employ physical objects, surfaces, and spaces as tangible
embodiments of digital information.
Adapted from Tangible Media Group web site
Foreground interactions
Graspable objects
Augmented surfaces
Link between physical and virtual actions
• exploit human senses of touch and kinesthesia.
MusicalBottles - controlling sound, light
Interaction
One opens and closes bottles
to control digital contents e.g.,
opening it tells a story
Music bottles
movement and uncorking of
the bottles controls the
different sound tracks and the
patterns of colored light rearprojected onto the table’s
translucent surface.
Extracted from Tangible Media Group web site
MusicalBottles - controlling sound, light
Ishii, MIT Extracted from Tangible Media Group web site
IAMASCOPE
Wooden Mirror - wooden pixels, image
Daniel Rozin, NYU Ishii, MIT
Wooden Mirror - wooden pixels, image
Ishii, MIT
Daniel Rozin, NYU
Triangles
– building a digital story
When triangles connect together,
they trigger digital events.
These events influence the progress
of a non-linear story, or allow users
to organize media elements in order
to create their own story space.
Extracted from Tangible Media Group web site
Triangles
– building a digital story
Extracted from Tangible Media Group web site
Ambient displays
Background information
• communicate digitally-mediated senses of activity and presence
at the periphery of human awareness.
• ambient light sound, airflow, water movement, object motion…
• peripheral displays
Extracted from Mark Weiser’s UbiqCom web site
Extracted from Mark Weiser’s UbiqCom web site
Pinwheels -
ambient digital wind
Extracted from Tangible Media Group web site
Pinwheels -
ambient digital wind
Extracted from Tangible Media Group web site
Information Perculator
bubbles of digital patterns
Heiner, Hudson & Tanaka
Information Perculator
bubbles of digital patterns
Heiner, Hudson & Tanaka
Ambient Light Display
light reflection from water onto
ceiling
Ambient Room
ACM CHI’98, Tangible Media Group
Personal Ambient Display
Small, physical devices worn to
display information to a person
in a subtle, persistent, and
private manner.
Ambient information is displayed
solely through tactile modalities
such as heating and cooling,
movement and vibration, and
change of shape.
Extracted from Tangible Media Group web site
Physical controls
Physically-mediated computer controlled
interactivity
Marble Answering Machine
Incoming voice messages are
physically instantiated as
marbles.
The user can grasp the message
(marble) and drop it into an
indentation in the machine to
play the message.
The user can also place the
marble onto an augmented
telephone, thus dialing the caller
automatically.
Durrell Bishop
TouchCounters
Computational tags track the
usage of physical objects.
TouchCounters sense activity
through magnetic, acceleration,
and infrared sensors, and
indicate their status on bright
LED displays.
TouchCounters can be networked
to a web server that generates
use histograms for each object.
Extracted from Tangible Media Group web site
TouchCounters
Extracted from Tangible Media Group web site
Tagged Objects
ACM CHI’99, Xerox Parc
mediaBlocks
Extracted from Tangible Media Group web site
SenseTable
A system for tracking the positions and
states of multiple objects wirelessly on a
flat surface.
Objects can be equipped with various
controls -- dials or buttons -- which can be
monitored in real-time.
When coupled with a projector, the system
can display information about the objects
on or near the objects themselves.
Extracted from Tangible Media Group web site
HandScape
A vectorizing digital tape
measure for digitizing field
measurements, and visualizing
the volume of the resulting
vectors with computer
graphics.
Using embedded orientationsensing hardware, it captures
relevant vectors on each linear
measurements and transmits
this data wirelessly to a
remote computer in real-time.
Extracted from Tangible Media Group web site
Manipulative User Interfaces
Customizable Physical Interfaces
ACM UIST 2002, Greenberg & Boyle
Collaborative interactions
Physical objects connecting collaborators
• collaborative interaction
• awareness
Adapted from Tangible Media Group web site
PingPong Plus
features a "reactive table" that
incorporates sensing, sound, and
projection technologies.
Projectors display patterns of
light and shadow on the table;
bouncing balls leave images of
rippling water; and the rhythm of
play drives accompanying music
and visuals.
Extracted from Tangible Media Group web site
PingPong Plus
Extracted from Tangible Media Group web site
InTouch – collaborative haptics
Force-feedback technology is
employed to create the illusion
that people, separated by
distance, are interacting with a
shared physical object.
When one of the rollers is
rotated, the corresponding
roller on the other distant
object rotates in the same
way. Two people separated by
distance can then play…
Extracted from Tangible Media Group web site
InTouch – collaborative haptics
Extracted from Tangible Media Group web site
Bench
…two cold steel benches located in
different cities.
When a person sits on one of these
benches, a corresponding position on
the other bench warms, and a bidirectional sound channel is opened.
At the other location, after feeling the
bench for "body heat," another
person can decide to make contact
by sitting near the warmth.
Initially the sound channel is
distorted, but as the second party
lingers, the audio channel clears.
--summarized by Ishii and Ullmer
Anthony Dunne and Fiona Raby at the RCA
Physical but Digital Surrogates
Proximity detector
ultrasonic sensor
Hydra unit
video, camera,
speakers, microphone
Rotating figurine
servo motor
Tippable figurine
light sensors
Physical but Digital Surrogates
Privacy preserving media space
Roomware
i-land
Computer-augmented room elements
• like doors, walls, furniture (e.g. tables and chairs) with integrated information
and communication technology.
From the GMD Darmstadt
web site on I-Land
Roomware
i-land
Dynawall
From the GMD Darmstadt web site on I-Land
Roomware
i-land
CommChair
From the GMD Darmstadt web site on I-Land
Roomware
i-land
ConnecTable
By moving multiple
ConnecTables together, they can be
arranged to form a large display area.
Integrated sensors measure the distance
between the ConnecTables and initiate
the automatic coupling of the displays
From the GMD Darmstadt web site on I-Land
Roomware
i-land
InteracTable
•touch-sensitive plasma-display (PDP) is integrated into the table top
•Border for leaning…
From the GMD Darmstadt web site on I-Land
Roomware
i-land
Phillips –
Intelligent Information Surfaces
Tokens
From the Philips Lime Video CD
Phillips –
Intelligent Information Surfaces
From the Philips Lime Video CD
Designing out of the box
The problem:
• programming / designing with physical devices is hard
o
o
o
o
o
o
o
circuit design (electrical engineering)
microprocessor interface to digital/analog devices
‘wire’ interface (serial, USB, wireless, IR…)
wire protocol
connection/disconnection/intermittent connectivity
software to use devices
maintenance and extensibility
• simple things take a long time to do
• most people don’t bother
Solution 1: Interdisciplinary team
Works, but…
Proximity detector
ultrasonic sensor
• Still takes time
• When one of the team
leaves, knowledge is
lost
• systems hard to
maintain &
extend
Hydra unit
video, camera,
speakers, microphone
Rotating figurine
servo motor
Tippable figurine
light sensors
Solution 2: Hack existing devices
Programmable Embodied
Agents (Kaminsky et al)
• hacked Microsoft Actimates
Solution 2: Hack existing devices
Programmable Embodied
Agents (Kaminsky et al)
•
•
•
Arm position -> quantity
Squeezing hand/leg -> counting
Movement-> task progress
Proxy for other person

squeeze hand, other’s hand goes up)
Event monitoring

Signal document is printing, then complete

Barney Email biff
Solution 3: Phidgets
Physical Widgets
• simple, easy to program devices and component-based software
with well-defined API
• building blocks for physical interfaces
• analogous to GUI widgets
Phidget Examples
PhidgetServo:
• Control 1 or 2 servo motors
PhidgetPowerbar
• Control power state of outlets on a power bar
PhidgetInterfaceKit
• 8 simple input and outputs plus 2 sensors
• A constructor kit
PhidgetProximitySensor
• Returns how close something is to it
PhidgetMotionDetector
• Periodically returns the amount of motion in a space
Related areas
Mobile Computing
Augmented Reality
Context-aware computing
Reactive Environments
Ubiquitous Media
Cooperative Buildings