View
221
Download
0
Tags:
Embed Size (px)
Citation preview
2
Image Transmission Over Wireless Networks
• Image capture and compression– Inner-workings of a digital camera– Manipulating & transforming a matrix of pixels– Implementing a variant of JPEG compression
• Wireless networks– Wireless technology– Acoustic waves and electrical signals– Radios
• Video over wireless networks– Video compression and quality– Transmitting video over wireless– Controlling a car over a radio link
3
Traditional Photography
• A chemical process, little changed from 1826
• Taken in France on a pewter plate
• … with 8-hour exposure The world's first photograph
5
Image Formation in a Pinhole Camera
• Light enters a darkened chamber through pinhole opening and forms an image on the further surface
6
Aperture
• Hole or opening where light enters–Or, the diameter of that hole or opening
• Pupil of the human eye–Bright light: 1.5 mm diameter–Average light: 3-4 mm diameter–Dim light: 8 mm diameter
• Camera–Wider aperture admits more light–Though leads to blurriness in the
objects away from point of focus
7
Shutter Speed
• Time for light to enter camera–Longer times lead to more light–… though blurs moving subjects
• Exposure–Total light entering
the camera–Depends on aperture
and shutter speed
8
Digital Photography
• Digital photography is an electronic process
• Only widely available in the last ten years
• Digital cameras now surpass film cameras in sales
• Polaroid closing its film plants this year…
9
Image Formation in a Digital Camera
• Array of sensors– Light-sensitive diodes convert photons to electrons– Buckets that collect charge in proportion to light– Each bucket corresponds to a picture element (pixel)
+10V+10V
+ + + + + ++ + + + + +
PhotonPhoton
++
A sensor converts one kind of energy to another
10
CCD: Charge Coupled Device
• Common sensor array used in digital cameras– Each capacitor accumulates charge in response to light
• Responds to about 70% of the incident light– In contrast, photographic film captures only about 2%
• Also widely used in astronomy telescopes
CCD sensor
12
Sensor Array: Reading Out the Pixels
• Transfer the charge from one row to the next
• Transfer charge in the serial register one cell at a time
• Perform digital to analog conversion one cell at a time
• Store digital representation
Digital-to-analog conversion
16
Representing Color
• Light receptors in the human eye– Rods: sensitive in low light, mostly at periphery of eye– Cones: only at higher light levels, provide color vision– Different types of cones for red, green, and blue
• RGB color model– A color is some combination of red, green, and blue– Intensity value for each color
0 for no intensity 1 for high intensity
– Examples Red: 1, 0, 0 Green: 0, 1, 0 Yellow: 1, 1, 0
17
Representing Image as a 3D Matrix
• In the lab this week…– Matlab experiments with digital images
• Matrix storing color intensities per pixel– Row: from top to bottom– Column: from left to right– Color: red, green, blue
• Examples– M(3,2,1): third row, second column, red intensity– M(4,3,2): fourth row, third column, green intensity
123
21
18
Limited Granularity of Color
• Three intensities, one per color– Any value between 0 and 1
• Storing all possible values take a lot of bits– E.g., storing 0.368491029692069439604504560106– Can a person really differentiate from 0.36849?
• Limiting the number of intensity settings– Eight bits for each color– From 00000000 to 11111111– With 28 = 256 values
• Leading to 24 bits per pixel Red: 255, 0, 0 Green: 0, 255, 0 Yellow: 255, 255, 0
19
Number of Bits Per Pixel
• Number of bits per pixel– More bits can represent a wider range of colors– 24 bits can capture 224 = 16,777,216 colors– Most humans can distinguish around 10 million colors
8 bits / pixel / color8 bits / pixel / color 4 bits / pixel / color4 bits / pixel / color
20
Separate Sensors Per Color
• Expensive cameras– A prism to split the light into three colors– Three CCD arrays, one per RGB color
21
Practical Color Sensing: Bayer Grid
• Place a small color filter over each sensor
• Each cell captures intensity of a single color
• More green pixels, since human eye is better at resolving green
22
Practical Color Sensing: Interpolating
• Challenge: inferring what we can’t see– Estimating pixels we do not know
• Solution: estimate based on neighboring pixels– E.g., red for non-red cell averaged from red neighbors– E.g., blue for non-blue cell averaged from blue neighbors
Estimate “R” and “B” at the “G” cells from
neighboring values
23
Interpolation
• Examples of interpolation
• Accuracy of interpolation– Good in low-contrast areas (neighbors mostly the same)– Poor with sharp edges (e.g., text)
and makes
and makes
and makes
24
Are More Pixels Always Better?
• Generally more is better– Better resolution of the picture– Though at some point humans can’t tell the difference
• But, other factors matter as well– Sensor size– Lens quality– Whether Bayer grid is used
• Problem with too many pixels– Very small sensors catch fewer photons– Much higher signal-to-noise ratio
• Plus, more pixels means more storage…
25
Digital Images Require a Lot of Storage
• Three dimensional object– Width (e.g., 640 pixels)– Height (e.g., 480 pixels)– Bits per pixel (e.g., 24-bit color)
• Storage is the product– Pixel width * pixel height * bits/pixel– Divided by 8 to convert from bits to bytes
• Example sizes– 640 x 480: 1 Megabyte– 800 x 600: 1.5 Megabytes– 1600 x 1200: 6 Megabytes
26
Compression
• Benefits of reducing the size– Consume less storage space and network bandwidth– Reduce the time to load, store, and transmit the image
• Redundancy in the image– Neighboring pixels often the
same, or at least similar– E.g., the blue sky
• Human perception factors– Human eye is not sensitive
to high frequencies
2727
Compression Pipeline
• Sender and receiver must agree– Sender/writer compresses the raw data– Receiver/reader un-compresses the compressed data
• Example: digital photography
compress uncompress
compress uncompress
2828
Two Kinds of Compression
• Lossless– Only exploits redundancy in the data– So, the data can be reconstructed exactly– Necessary for most text documents (e.g., legal
documents, computer programs, and books)
• Lossy– Exploits both data redundancy and human perception– So, some of the information is lost forever– Acceptable for digital audio, images, and video
2929
Lossless: Huffman Encoding
• Normal encoding of text– Fixed number of bits for each character
• ASCII with seven bits for each character– Allows representation of 27=128 characters– Use 97 for ‘a’, 98 for ‘b’, …, 122 for ‘z’
• But, some characters occur more often than others– Letter ‘a’ occurs much more often than ‘x’
• Idea: assign fewer bits to more-popular symbols – Encode ‘a’ as “000”– Encode ‘x’ as “11010111”
3030
Lossless: Huffman Encoding
• Challenge: generating an efficient encoding– Smaller codes for popular characters– Longer codes for unpopular characters
English Text: frequency distribution
Morse code
3131
Lossless: Run-Length Encoding
• Sometimes the same symbol repeats– Such as “eeeeeee” or “eeeeetnnnnnn”– That is, a run of “e” symbols or a run of “n” symbols
• Idea: capture the symbol only once– Count the number of times the symbol occurs– Record the symbol and the number of occurrences
• Examples– So, “eeeeeee” becomes “@e7”– So, “eeeeetnnnnnn” becomes “@e5t@n6”
• Useful for fax machines– Lots of white, separate by occasional black
32
Joint Photographic Experts Group
• Starts with an array of pixels in RGB format– With one number per pixel for each of the three colors– And outputs a smaller file with some loss in quality
• Exploits both redundancy and human perception– Transforms data to identify parts that humans notice less– More about transforming the data in Wednesday’s class
Uncompressed: 167 KB Good quality: 46 KB Poor quality: 9 KB
34
New Era of Computational Photography
• Beyond manual editing of photographs– E.g., to crop, lighten/darken, sharpen edges, etc.
• Post-processing of a single photography– De-blur photos marred by camera shake– Changing the depth of field or vantage point
• Combining multiple pictures– Creating three-dimensional representations– Stitching together photos for a larger view– Creating a higher-resolution picture of a single scene
• <Insert your idea here>
http://www.news.com/8301-13580_3-9882019-39.html
35
Conclusion
• Conversion of information– Light (photons) and a optical lens– Charge (electrons) and electronic devices– Bits (0s and 1s) and a digital computer
• Combines many disciplines– Physics: lenses and light– Electrical engineering: charge coupled device– Computer science: manipulating digital representations– Mathematics: compression algorithms– Psychology/biology: human perception
• Next class: compression algorithms