Transcript DSP Interaction and Media Mark Smith Sensing and Mobility

DSP
Interaction and Media
Video and Multimedia
Imaging and Communication
Sensing and Devices
Mark Smith
Sensing and Mobility
HP Labs, Palo Alto
[email protected]
•
participate in hp strategy
creation
•
deliver technology that
enables hp to win in hp’s
selected strategies
through:
– breakthrough
technologies
HP labs’ roles
– technology
advancements
•
create opportunities that
go beyond hp’s current
strategies
•
invest in fundamental
science and technology
in areas of interest to hp
Tradition of Innovation
OmniBook 300
Lightest
Portable PC
Hand-Held
Calculator
Audio
Oscillator
in
Disney’s
“Fantasia”
Atomic
Clock
1939 ‘51
‘66 ‘67 ‘68
HighFrequency
Counter
HP’s
LaserJet
Printer
‘72
Desktop
Scientific
Calculator
HP 9100
Computer
‘84
Real time
Ultra-Sound
Cardiac
Analysis
‘91
‘86
Access7
Telecom
Monitoring
System
’93 ’94 ’95
World’s
Brightest
LEDs
InkJet
Printer
PA-RISC
HP 95LX
Palmtop PC
Atomic
Resolution
Storage
JetSend
Protocol
IA-64 / EPIC
Computer
Architecture
‘97
‘98
‘2000
Web QoS
Digital
Photography
Future
Molecular
Computing
Web
Wristwatch
(w/ Swatch)
Change:
Applications deploy
an end to end space.
Content
• Applications are network
sourced.
• Services lie outside the
network.
• Devices are aggregated
into what the application
and user wants.
Transaction
Living Space
Vehicle Space
End to End Application Space
HPL Streaming Media Technologies
Rate Changing
Object-Based
Video Processing
Smart
Recoding
Streaming
Media
Industry
Splicing
Digital
VCR
HP Technologies
MPEG4
to MPEG2
Independent
Region
Coding
Wireless
Video
Digital
Television
Industry
Secure
Video
Streaming
Low-Power
Video
Compression
MPEG2
to H.263
Error-Resilient
Video Coding
Problem Statement
How do we process compressed media streams?
2,000 MOPS
Decode
20 Mbps
•
1 Gbps
20,000 MOPS
Process
1 Gbps
Encode
20 Mbps
Difficulties:
– Computational requirements: 22,000 MOP overhead
from decode and encode operations (plus additional
processing)
– Bandwidth requirements: Need to process
uncompressed data at 1 Gbps
– Quality issues: Even without processing, the
decode/encode cycle causes quality degradation.
•
Compressed
Domain
Processing
In the digital world, media
streams are stored,
transported, and
distributed in compressed
form.
How do we process compressed
video streams?
2,000 MOPS
100110101011100010
Decode
20 Mbps
100110101011100010
20,000 MOPS
Process
Encode
010001101001101101
20 Mbps
1 Gbps
Use efficient compressed-domain
processing algorithms.
1 Gbps
CDP
010001101001101101
CITMO 2002
Streaming Media System
Full resolution
Decode and Display
Transcoder
High-bandwidth
LAN
Full resolution
Capture and
Encode
Low resolution
Decode and Display
HP Smart
Stream Router
Low-bandwidth
wireless link
HP Streaming System
Media streaming and
transcoding over packet
networks.
Secure Delivery and Transcoding of
Compressed Video Streams
Full resolution Decrypt,
Decode, and Display
Secure
Transcoding
Secure
Transcoder
High-bandwidth
LAN
Capture,
Encode,
and Encrypt
Low resolution Decrypt,
Decode, and Display
HP Smart
Stream Router
Low-bandwidth
wireless link
HP Secure Streaming System
Secure media streaming and
transcoding over packet
networks.
HP CapShare 910 information appliance
One of the most DSP intensive things I ever worked on!
Scanning motion up to 1 mph
Image processing path
Path Reconstruction
and Page Orientation
Image Rectification
Image Stitching
2D optical navigator
•
Image X-correlation processor
• 2D motion & contact detection
• Derivative used in optical mice
• Prototype:
• 600 dpi resolution
• 32x64 pixels
• 1.5 GOPS computation rate
• 25K frames/second (1 mph)
• 180 mW power dissipation
•
Commercial versions:
• 18x18 pixels
• 2.3K frames/sec
• 75 mW power dissipation
Device Operation
9 Nearest Pixel
Correlations
Surface Texture Image
Optics
2D Minima Interpolation
9 Local
Pixels
Grazing Illumination
Object surface Texture
Compute Minima Translation
Final navigator design
SecurePAD—
Dynamic Security
Goal of providing access,
security and accountability
based on who, where and
how a resource is used.
Biomedical, Health,
Sensors and Algorithms
Aggregate sensor data to give
physical meaning.
Used by services to provide:
Personalization
Privacy assurance
Connectivity optimization
Current Efforts:
Device Possession
Biometric physiological
state
Context based Privacy
Algorithms
Sensor enhanced Security
Algorithms
In the watch band
The Djammer: DSP for interactive MP3
Wearable, grab and shake design
USB sensor interface allows multiple
devices and use with BlueTooth earpiece
Music interaction and scratching is done
using an optical sensor and accelerometers
Prototype used 802.11b for the wireless
link. Bluetooth is also possible.
DJammer = DJ +
jam session
What is new about the DJammer
is:
1.
2.
3.
4.
It allows mobile users to
interact with the digital music
presented to them.
It allows users to augment
music contents through a
digital realization of music
scratching (pitch control).
It permits the resulting
augmented music to be
communicated by streaming
to other DJammer users
allowing further augmentation
and streaming.
It is capable of being
aggregated into sessions with
other dissimilar devices
allowing sharing of music.
MEMS
Micropositioners
– Folded flexures provide
x and y axis motion,
while limiting z
displacement
– Motor fabrication
integrated with storage
mechanism fabrication
A
D.A. Horsley et al., Proc. of the 2000 Solid State Sensor & Actuator Workshop,Hilton Head, SC, June 4-8, 2000.
A’
Atomic
Resolution
Storage
Probe-based storage using
phase-change medium,
array of electron emitters,
MEMS micropositioner
– Write / Erase
> apply strong electron beam
from tip
> heating/cooling profile
determines write / erase.
– Read
> apply weak electron beam
from tip
> detect resistance of written
bits
> Optical reading techniques
> SEM , others...
”Avoiding a Data Crunch”, J. W. Toigo, Scientific American 282, 58 (2000)
ARS Medium working!
•
HP Logo
– written on phase
change material
– conductivity
image, not
topological
image
Molecular Electronics
Pushing the Limits
Scaling of electronic devices
Number of chip components
1018
Classical Age
1016
Quantum Age
77oK
1014
1012
SIA Roadmap 2005
2000
1995
1010
108
106
104
295oK
Historical Trend 1990
2010
4oK
Quantum State Switch
CMOS
1980
1970
102
101
100
10-1
Feature size (microns)
10-2
10-3
Silicide Nanowires on Si
parallel quantum wires –
nine nanometer separation
Atomic Image
HP Company Technology Model
radical ideas
Internet
molecular
Appliance
electronics
s
Streaming
Compression
Computer
Media
Security
Communication
Wireless Architectures
Algorithms
and Internetworking
Printing Software
Networking
Engineering
CoolTown
Computer
bistable displays Storage
breakthrough technology
Systems
Graphics
technologies advancements
labs
atomic
resolution
storage
transfers
next generation
products
product
organizations
current products
Imaging
labs