Transcript Digital Cameras Engineering Math Physics (EMP) Jennifer Rexford

Digital Cameras
Engineering Math Physics (EMP)
Jennifer Rexford
http://www.cs.princeton.edu/~jrex
1
Image Transmission Over Wireless Networks
• Image capture and compression
– Inner-workings of a digital camera
– Manipulating & transforming a matrix of pixels
– Implementing a variant of JPEG compression
• Wireless networks
– Wireless technology
– Acoustic waves and electrical signals
– Radios
• Video over wireless networks
– Video compression and quality
– Transmitting video over wireless
– Controlling a car over a radio link
2
Traditional Photography
• A chemical process, little
changed from 1826
• Taken in France on a
pewter plate
• … with 8-hour exposure
The world's first photograph
3
Digital Photography
• Digital photography is an
electronic process
• Only widely available in the
last ten years
• Digital cameras now surpass
film cameras in sales
4
Image Formation
Digital Camera
Film
Eye
5
Aperture and Exposure
• Aperture
– Diameter of the hole allowing light to enter
– E.g., the pupil of the eye
– Higher aperture leads to more light entering
– … though poorer focus across a wider depth of field
• Shutter speed
– Time for light to enter the camera
– Longer times lead to more light
– … though blurring of moving subjects
• Together, determine the exposure
– The amount of light allowed to enter the camera
6
Image Formation in a Pinhole Camera
• Light enters a darkened chamber through pinhole
opening and forms an image on the further surface
7
Image Formation in a Digital Camera
+10V
Photon
++++++







+
• Array of sensors
CCD sensor
– Light-sensitive diodes that convert photons to electrons
– Each cell corresponds to a picture element (pixel)
• Sensor technologies
– Charge Coupled Device (CCD)
– Complementary Metal Oxide Semiconductor (CMOS)
8
Sensor Array: Image Sampling
9
Sensor Array: Reading Out the Pixels
• Transfer the charge from
one row to the next
• Transfer charge in the serial
register one cell at a time
• Perform digital to analog
conversion one cell at a time
• Store digital representation
Digital-to-analog
conversion
10
Sensor Array: Reading Out the Pixels
11
More Pixels Mean More Detail
1600 x 1400
1280 x 960
640 x 480
12
The
2272 x 1704
hand
The
320 x 240
hand
13
Representing Color
• Light receptors in the human eye
– Rods: sensitive in low light, mostly at periphery of eye
– Cones: only at higher light levels, provide color vision
– Different types of cones for red, green, and blue
• RGB color model
– A color is some combination of red, green, and blue
– E.g., eight bits for each color
 With 28 = 256 values
 Corresponding to intensity
– Leading to 24 bits per pixel
 Red: 255, 0, 0
 Green: 0, 255, 0
 Yellow: 255, 255, 0
14
Number of Bits Per Pixel
• Number of bits per pixel
– More bits can represent a wider range of colors
– 24 bits can capture 224 = 16,777,216 colors
– Most humans can distinguish around 10 million colors
8 bits / pixel / color
4 bits / pixel / color
15
Separate Sensors Per Color
• Expensive cameras
– A prism to split the light into three colors
– Three CCD arrays, one per RGB color
16
Practical Color Sensing: Bayer Grid
• Place a small color filter
over each sensor
• Each cell captures
intensity of a single color
• More green pixels, since
human eye is better at
resolving green
17
Practical Color Sensing: Interpolating
• Challenge: estimating pixels
we do not know for certain
• For a non-green cell, look at
the neighboring green cells
– And, interpolate the value
• Accuracy of interpolation
Estimate “RGB” at
the “G” cells from
neighboring values
– Good in low-contrast areas
– Poor with sharp edges (e.g., text)
18
Digital Images Require a Lot of Storage
• Three dimensional object
– Width (e.g., 640 pixels)
– Height (e.g., 480 pixels)
– Bits per pixel (e.g., 24-bit color)
• Storage is the product
– Pixel width * pixel height * bits/pixel
– Divided by 8 to convert from bits to bytes
• Common sizes
– 640 x 480: 1 Megabyte
– 800 x 600: 1.5 Megabytes
– 1600 x 1200: 6 Megabytes
19
Compression
• Benefits of reducing the size
– Consume less storage space and network bandwidth
– Reduce the time to load, store, and transmit the image
• Redundancy in the image
– Neighboring pixels often the
same, or at least similar
– E.g., the blue sky
• Human perception factors
– Human eye is not sensitive
to high frequencies
20
Contrast Sensitivity Curve
21
Lossy vs. Lossless Compression
• Lossless
– Only exploits redundancy in the data
– So, the data can be reconstructed exactly
– Necessary for most text documents (e.g., legal
documents, computer programs, and books)
• Lossy
– Exploits both data redundancy and human perception
– So, some of the information is lost forever
– Acceptable for digital audio, images, and video
22
Examples of Lossless Compression
• Huffman encoding
– Assign fewer bits to less-popular symbols
– E.g., “a” occurs more often than “i”
– … so encode “a” as “000” and “i” as “00111”
– Efficient when probabilities vary widely
• Run-length encoding
– Identify repeated occurrences of the same symbol
– Capture the symbol and the number of repetitions
– E.g., “eeeeeee”  “@e7”
– E.g., “eeeeetnnnnnn”  “@e5t@n6”
23
Joint Photographic Experts Group
• Lossy compression of images
– Starts with an array of pixels in RGB format
 With one number per pixel for each of the three colors
– Outputs a smaller file with some loss in quality
– Exploits both redundancy and human perception
 Transforms the data to identify parts that humans notice less
 More about transforming the data in Wednesday’s class
Uncompressed: 167 KB
Good quality: 46 KB
Poor quality: 9 KB24
Conclusion
• Digital cameras
– Light and a optical lens
– Charge and electronic devices
– Pixels and a digital computer
• Digital images
– A two-dimensional array of pixels
– Red, green, and blue intensities for each picture
• Image compression
– Raw images are very large
– Compression reduces the image size substantially
– By exploiting redundancy and human perception
25