View
13.767
Download
1
Category
Tags:
Preview:
DESCRIPTION
The enormous possibilities and widespread connectivity offered by the Internet and the World Wide Web has spawned multiple ways of exchanging and communicating color images. The Internet is an evolving communication system, where uses, technologies, and applications are continuously introduced by a plethora of players. Its functionality, reliability, scaling properties, and performance limits are largely unknown—albeit they span wide gamuts from optic fiber to wireless connections and from game consoles to palmtop devices, etc. To be successful in Internet imaging, users and developers must design systems in a top-down approach. The goal of this tutorial is to sort out the available standard methods so that attendees will become familiar with the different possibilities for Internet imaging; the trade-offs, issues and dependencies of each; how and when each is used; and their system implications. To this end, we systematically present the standard methods for color encoding, image compression, file formatting, protocols, and applications.
Citation preview
w w w
Visual Communications and Image Processing 2003
Color Imaging on the Internet
Robert Buckley
Xerox Innovation Group
Giordano Beretta
Hewlett-Packard Laboratories
http://www.inventoland.net/imaging/cii/
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1Course objectives
• List and describe the current and emerging methods for Internet image exchange
• Develop a systematic understanding of the principles of color encoding, image compression, file formatting, protocols, and Internet imaging applications
• Understand the differences between the various methods for each imaging function
• Develop an intuition for specifying well-balanced scalable architectures for Internet imaging
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2Rationale
Why a course on color imaging specifically for the Internet?
• A picture is worth ten thousand words…
• A screenful of text requires 24
×
80 = 1,920 bytes
• A VGA size image requires 640
×
480
×
3 = 921,600 bytes
• …but requires almost 500 times as much bandwidth…
• data compression is essential for images on the Internet
• which compression is best for my image?
• …and the server and client are unknown a priori
• which color representation is suitable to both?
• which file format can be understood by both?
• how can they negotiate the above?
• how can we provide for unknown viewing conditions?
To be successful, systems must be designed in a top-down approach
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
3Course roadmap
• Systematic bottom-up presentation and comparison of methods
• Intended for top-down system design
Application
Protocol
Format
Compression
Color image
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
41 Basics
Evolution of Internet imaging
• Internet developed over 30 years, now mature and in incremental engineering mode
• Although the Internet has been used for scientific visualization from the beginning, it has become a visual medium only since the advent of the free Mosaic browser in 1993
• Outline of this module:
• the Internet
• protocols
• media types
• intelligent image processing
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
51.1 The Internet
Born Arpanet, Fall 1969
• Originally a high-speed packet-switching network connecting research super-computers
• packet switching allows building a reliable system that is based on an infrastructure assumed at all times to be unreliable
• each packet is individually addressed and each node just forwards packets not addressed to itself
• the routing of packets is irrelevant
• based on TCP/IP
• Today the Internet is the communications medium for
• individuals
• businesses
• communities of practice (extended knowledge networks)
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
6
1.1.1 The communication process
Claude Shannon, 1941
1. Information
source
: person or thing generating original message
2.
Transmitter
: intrument that transforms the message into a signal suitable for transmission
3. Communication
channel
: medium that conducts the signal4.
Receiver
: instrument that takes the signal and tries to reconstruct the message
5.
Destination
: person or thing the message is intended for
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
7
1.1.2 Multi-layer models for networking
Network AccessEthernet, FDDI Physical
Data Link
Network
Transport
Session
Presentation
Application
DoD 4-layer model OSI 7-layer model
Used in newer designsUsed in the originaldevelopment ofthe Internet
InternetIP, IPv6
Host-to-HostTCP, UDP
ProcessFTP, SMTP, HTTP
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
8
1.1.3 IETF standards development
0
500
1000
1500
2000
2500
3000
3500
4000
RFC
No
.
2000199519901985198019751970
3559
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet 9
1.1.4 The WWW
Born World Wide Web, March 1989
• Using hypertext links to connect chunks of information on the Internet
• The WWW is a set of three specifications• URL, Uniform Resource Locator, to locate information
• HTML, Hypertext Markup Language, to write simple documents
• HTTP, Hypertext Transfer Protocol, to transfer HTML files
• The WWW became popular when• the Internet became commercialized
• fast data connections became pervasive
• graphical browsers made navigation easy and appealing
• early adopters understood the value of the new communication medium and invented disruptive technologies
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
101.2 Protocols
• A protocol is a set of conventions or rules governing communications
• Protocols allow networks to interconnect and ensure compatibility between devices of different manufacturers
• Examples:• FTP — file transfer protocol
• HTTP — hypertext transfer protocol
• IIP — Internet imaging protocol
• IPP — Internet printing protocol
• SMTP — Simple Mail Transfer Protocol
• Protocols become standards when signed off by an official body like IETF, W3C, ITU-T, ISO, or IEEE• de facto and de jure standards
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
111.2.1 FTP — File Transfer Protocol
Transfer files from one machine to another
• Based on TCP/IP• TCP (transmission control protocol) converts messages into streams of
packets at the source, then reassembles them back into messages at the destination
• IP (Internet protocol) handles addressing, seeing that packets are routed across multiple nodes and even across multiple networks with multiple standards
• Requires explicit directory navigation both at the source and the destination
• Allows anonymous login
• Can perform end-of-line conversion in ASCII files
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
121.2.2 HTTP — HyperText Transfer Protocol
Transfer compound documents with links
• Application-level protocol for distributed, collaborative, hypermedia information systems
• Requires a reliable transport such as TCP/IP
• Request the components of a document identified by hypertext links
• Provides support for HTML forms
• Typing and negotiation of data representation allows systems to be built independently of the data being transferred• example: color images
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
131.2.3 Protocols for wireless applications
The PC is no longer at the center of the world
• WASP — wireless application service providers
• WAP — wireless application protocol
• WML — wireless mark-up language for WAP• existing commercial applications for automatic translation from HTML to
WML by AvantGo and Phone.com
• iMode — uses HTML, but the screen is still small• created by NTT DoCoMo
• does not require translation from HTML
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
141.2.4 Protocol evolution for services
remote messages remote procedures distributed objects web services
1960 1970 1980 1990 2000
LPC
UDP/TCP
DDE
IPC
DLL
OLE 1.0
MSRPC
NCS
ONC
SHL
DCE
ONC+
RRBC
CLR
COM DCOM
COM/CORBA
RPC
RPC
CORBA
JAVA/RMI
J2EE
SOAP
.NET
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
151.3 Internet media types
• Identify type and encoding of transmitted data• type/subtype
• used by Multipurpose Internet Mail Extensions (MIME) and others
• used to be called MIME types• standard types registered with Internet Assigned Numbers Authority
(IANA)
• Standard types, sample subtypes
text plain, html
multipart mixed, related
message rfc822, http
application pdf, vnd.ms-powerpoint, ipp
image tiff, jpeg, png, gif, vnd.fpx
audio basic, 32kadpcm
video mpeg, quicktime
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
161.4 New trends in image processingUser’s expectations
• Many users access the Internet in the office on fast workstations connected over fast links to the Internet
• At home users often have fast graphics controllers for playing realistic computer games
• Increasingly, private homes are equipped with fast connections over DSL, cable modem, 802.11g, FTTH, …
• The latest video game machines are very powerful graphic workstations
These user experiences set very high expectations for color imaging on the Internet
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
171.4.1 Polarization of devices
The nomadic workforce
• The new generation grew up on video games & WWW
• At work, they expect concise answers immediately on multiple media
• The new working world is mobile and wireless• a comprehensive fast fiber optics network provides a global backbone
• the “last mile” is wireless
• computers are wearable
• An appropriate viewing device has not yet been invented• but it will not be printed paper
• the viewing conditions will be unpredictable
• likely, a plethora of viewing devices will be in use
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
181.4.1.1 Global Crossing’s peak network
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
191.4.2 How fast is the Internet?
It is both fast and slow
• There is a lot of global fiber• example: Global Crossing planned to circumscribe all continents
• Backbones will have ample bandwidth• oversupply: a large amount of fiber is dark
• competition is fierce
• movies on demand and telepresence will consume this bandwidth
• Most users will access the data wireless• color imaging over the Internet must be efficient
• Today’s game machines have much more processing power than desktop machines• trade-off data for computation on the client
Trend: separation of data from control
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
201.4.3 Leveraging on vision theory
• To conserve power, wireless devices will have low effective transmission bandwidth and small display areas
• Concomitantly the new users are impatient
• Progressive encoding based on region of interest will be crucial• JPEG 2000 and MPEG-21 provide the frameworks
• algorithms are required
• Automatic cropping based on region of interest is a necessary capability for major commercial sites
• Leverage on vision theory for Internet imaging• Intelligent image processing technologies
• Lawrence Stark & Claudio Privitera, UC Berkeley
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
211.5 Anatomy of a Web page
computer graphics
plain text
full color image
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
221.5.1 Web page elements
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
232 Color representations
Application
Protocol
Format
Compression
Color image
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
242.1 Requirements
• Color must be encoded in standards that• support communication over the Internet
• the total size of a page should be such it can be transferred quickly• hence the color space must compress well
• are suitable for heterogeneous environments
• there is no a priori knowledge of the user platform• the Internet is more like a bazaar than a cathedral
• can easily be implemented efficiently and robustly
• Internet imaging applications are not implemented by color scientists• images must be displayed reliably (no unexpected rendering)• there is no a priori knowledge of the user’s machine power
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
252.2 Viewing condition issuessee also §2.6.2, slide 40
• There is no control over the user’s viewing conditions• users often work in poor viewing conditions
• viewing conditions can change during a session
• there is a plethora of viewing devices
• an applications implementation may not be aware of the difference, e.g. between colorimetric RGB and device RGB
• Issues too complex to expect users controlling their viewing conditions
• Color integrity is more important than color fidelity• Ralph Evans: consistency principle
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
262.3 Color representationsColor model operators
• XYZ• basis for all colorimetry
• defined by CIE for 1931 2˚ and 1964 10˚ Standard Observers
• most applications refer to 2˚ Observer
• RGB• scanners and digital cameras — linear, non-CIE
• monitors and displays — non-linear, CIE-based
• Luma-chroma• luminance (lightness) and 2 opponent color signals
• color television — luminance-chrominance YIQ, YUV, YCbCr
• uniform color spaces — CIELAB, CIELUV
• color fax uses CIELAB
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
272.4 Luma-chroma spaces
LC1
C2
A
fR R( )
fG G( )
fB B( )
⋅=
XYZNTSCRGB
EBURGB
SMPTERGB
CCIR709
CIELABPhotoYCC
YIQ
YES
YUV
sRGB
YC1C2
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
282.4.1 RGB separations
R
G B
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
292.4.2 CIELAB separations
a* b*
L*
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
302.4.3 Chroma subsampling
L*
a*b*
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
312.5 Some popular schemesto represent color on the Internet
• sRGB is a colorimetric standard based on common CRTs• gamma function is built-in for efficient display
• does not require computations in most cases
• viewing conditions are part of the standard, but are not realistic for casual users on the Internet
• extended sRGB color spaces are under development
• CIELAB and YUV are opponent color spaces that compress well in the case of pictorial images
• YCbCr is an opponent color space that was used extensively in developing the JPEG standard• Y is the same as in YUV
• U and V are scaled and zero-shifted so that Cb and Cr are in [0, 1]; then they are scaled by 255 to be represented by a byte
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
322.5.1 RGB specification
• Primaries• red: (xR, yR)
• green: (xG, yG)
• blue: (xB, yB)
• White point: (xN, yN)
• Non-linearity (gamma)
• Example: sRGB• IEC 61966-2-1
• (xR, yR) = (0.64, 0.33)
• (xG, yG) = (0.30, 0.60)
• (xB, yB) = (0.15, 0.06)
• (xN, yN) = (0.3127, 0.3290)
• same as ITU-R BT.709-2
Transformation from sRGB to 1931 CIE XYZ values
R’sRGB = R8bit / 255.0
G’sRGB = G8bit / 255.0
B’sRGB = B8bit / 255.0
If R’sRGB, G’sRGB, B’sRGB ≤ 0.04045
RsRGB = R’sRGB / 12.92
GsRGB = G’sRGB / 12.92
BsRGB = B’sRGB / 12.92
else R’sRGB, G’sRGB, B’sRGB > 0.04045
RsRGB = [(R’sRGB + 0.055) / 1.055]2.4
GsRGB = [(G’sRGB + 0.055) / 1.055]2.4
BsRGB = [(B’sRGB + 0.055) / 1.055]2.4
and
XYZ
0.4124 0.3576 0.18060.2126 0.7152 0.07220.0193 0.1192 0.9505
RsRGB
GsRGB
BsRGB
=
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
332.5.1.1 sRGB viewing conditionsReference display conditions
Display parameter Reference condition
luminance level 80 cd/m2
white point D65
gamma 2.2
Reference viewing conditions
Viewing parameter Reference condition
screen background 20% of reference display area
surround 20% of ref. ambient illuminance level
proximal field 20% of ref. display luminance level
ambient illuminance level 64 Lux
ambient white point D50
veiling glare 1%
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
342.5.2 CIELAB
• CIE standard for color difference evaluation• uniform color space
• illuminant Xn, Yn, Zn
• L* range: [0, 100]
• Xn, Yn, Zn: reference white• D50: 96.422, 100, 82.521; D65: 95.047, 100, 108.883
• von Kries type adaptation
• Color fax, ICC Profile Connection Space
L* 116 Y Yn⁄3⋅ 16–=
a* 500 X Xn⁄3 Y Yn⁄3–{ }⋅=
b* 200 Y Yn⁄3 Z Zn⁄3–{ }⋅=
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
352.5.2.1 8-bit CIELAB encodings
• CIE encoding (TIFF)• scale L* = [0, 100] to [0, 255]
• limit a* and b* to [-128, 127]
• ICC encoding• scale L* = [0, 100] to [0, 255]
• add offset 128 and limit a* and b* to [0, 255]
• white point: D50
• ITU-T encoding• scale L* = [0, 100] to [0, 255]
• apply scale/offset so a* = [-85, 85] maps to [0, 255]
• apply scale/offset so b* = [-75, 120] maps to [0, 255]
• white point: D50
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
362.5.3 The YUV color space
Used in the PAL television system
or
Reference: Bhaskaran & Konstantinides
YUV
0.299 0.587 0.1140.148– 0.289– 0.4370.615 0.515– 0.100–
RGB
⋅=
Y 0.299 R G–( ) G 0.114 B G–( )+ +=
U 0.493 B Y–( )=
V 0.877 R Y–( )=
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
372.5.4 The YCbCr color space
Popular for JPEG
• From ITU-R BT.601-2 for color television
• 8-bit encoding in digital files
YCbCr
0.299 0.587 0.1140.169– 0.331– 0.5000.500 0.419– 0.081–
RGB
⋅=
YCbCr
0.299 0.587 0.1140.169– 0.331– 0.5000.500 0.419– 0.081–
RGB
0128128
+⋅=
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
382.6 More color representations
• CMYK• color print separations — device specific, non-linear
• example: SWOP printing process specification
• Palette• color map or lookup table
• color represented by an index into a table of N colors
• see §3.3.1, slide 70
• ICC profiles• profile is a transform between a given color space and a Profile
Connection Space (PCS)
• defines color explicitly in terms of its transform to PCS
• PCS is XYZ or CIELAB• ICC has defined standard formats for profiles
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
392.6.1 ICC profile concepts
• Profile classes• input devices (scanners, digital cameras)
• display devices (monitors, LCD projectors)
• output devices (printers, film recorders)
• DeviceLink (dedicated device to device)
• ColorSpace
• Abstract (PCS-to-PCS, effects, e.g., contrast adjustment)
• NamedColor (Pantone®, Trumatch®)
• Rendering intents• colorimetric: absolute, relative; perceptual; saturation
• Models• shaper/matrix (shaper is a 1-D LUT)
• shaper/multi-D LUT
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
402.6.2 ICC profile based color reproduction
• One cannot assume that a casual Web User works in a controlled environment
• sRGB is considered a safe bet for “average” situations
• Tools are available to control color rendering on the Internet server side
• It is imperative that the entire workflow is characterized and ICC profiles be always embedded in images, instead of assumed
• For an example on how to set up an ICC based environment see//www.hpl.hp.com/techreports/1999/HPL-1999-110.pdf
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
412.7 Color interchange models
color values: S = source, R = reference, D = destinationTi = color conversion
colordestinationT2T1
colorsource
destination parameters
DS R
colordestinationT2
colorsource
DS RT1
colordestinationT2
DS RT1
colorsource
source parameters
Type I
Type III
Type II
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
422.7.1 Color interchange model types
• Type I• interchange uses device color values
• source prepares color data for known destination
• example: traditional graphic arts CMYK workflow
• Type II• interchange uses device-neutral, reference color space
• examples: color TV broadcasting, color facsimile
• Type III• source transmits source values + source characteristics
• similar to type II, but with delayed conversion
• examples: PDF CIE-based color spaces, ICC workflow
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
432.8 Server side color management
• On the client side, a set of filters is used to create visually an ICC profile with an applet running in the browser
• High-end systems are based on spectroradiometry & compensate for brightness level differences among monitors
1. On the server side, a servelet pushes each image through a color management system before it is sent to the client• E-Color True Internet Color, Imation Verifi
2. …or servelet sends applet that does correction at browser• Gretag-Macbeth WebSync
3. …or the HTML page is tagged with a trigger• WayTech Coloreal, Praxisoft RealNetColor
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
442.8.1 True Internet Color architecture
Browser
Merchant’sServer
E-Color’sServer
1request
page
2send page
withimage URL
3image URLpoints toE-Color’s
server
4send color-corrected
image
send uncorrected images
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
452.8.2 Verifi Accurate Web Color architecture
Browser
Merchant’sImage Server
Imation’sProfile Server
1request
page
5send profile
fromcookie
6send color-corrected
image
3requestprofile
4send cookie
withprofile
2send raw
uncorrectedimage
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
462.8.3 WebSync architecture
Browser
Merchant’sServer
1request
page
5send page
andcolor-matching
applet
4replaceimagetags
WebSyncSoftware
2forwardrequest3
sendpage
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
472.8.4 Coloreal architecture
• Polynomial for display monitor’s gamma curve is stored in monitor’s EDID chip
• On merchant’s Web server all images are encoded in sRGB
• Web server adds a Coloreal tag to each HTML file
• When monitor is first connected, installer reads gamma curve from EDID chip to create an ICC profile for the monitor
• When an HTML page contains the Coloreal tag, Windows ICM is invoked to use the IC profile to compute device RGB counts
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
482.8.5 RealNetColor architecture
• On Web server all images are encoded in sRGB or are tagged with an ICC profile
• Web server adds a RealNetColor tag to each HTML file
• Each use of the RealNetColor tag triggers a payment from the Web retailer to Praxisoft
• When an HTML page contains the RealNetColor tag, a plug-in converts the color using the ICC profile or assuming sRGB values
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
492.9 Display trendsLCDs are not CRTs
• CRT displays are being replaced by LCD displays
• LCDs are brighter, smaller, and use less power
• However, the colorimetry can be quite different• with careful calibration,
characterization & color management, an LCD can be made to perform close to a CRT in terms of linearity, gamma, and white point
• the color gamut can be very different
• today’s LCDs can outperform CRTs (monitors above are from 1995, 1998)
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
502.9.1 Display gamuts, the whole story
• Gamut renderings in chromaticity diagrams are misleading, because of colorfulness and appearance mode
21” studio CRT 23” LCD
reproduced with permission, © Apple
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
512.9.2 Future display technologies — OLED
Organic light-emitting-diode displays
• LCD displays use absorption filters and polarizers, limiting the gamut in the blues and the brightness
• OLED displays are emissive and are brighter• no filters nor polarizers
• Current limited lifetime of blue OLEDs limits the gamut in the blues even more than for LCDs
• Wafer size still limited• today’s applications: car stereo, portable DVD players
• largest prototype display shown: 13”
• manufacturing process more expensive than LCD
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
522.9.3 Future display technologies — MEMS
• MEMS (Micro-ElectroMechanical System) technology makes is possible to build displays based on interference
• Voltage between thin film stack and metallic membrane controls their gap and therefore the pixel’s color
• Luminance by flickering or dithering
• Typical resolutions: 400–1000 dpi
• Bistable, only draws power during switching
glass substratethin film stack
metallic membraneair
V
reproduced with permission, © Iridigm
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
532.10 Appearance mode
• CRT at 80 cd/m2 is darker than surroundings• perceived as object in field of view
• viewing conditions must be controlled
• color fidelity is important
• LCD at 300 cd/m2 is brighter than indoor surroundings• similar to illuminator viewing condition
• visual system adapts to white point, memory colors
• OLED achieves 30,000 cd/m2 in military applications• expect 1,000 cd/m2 in consumer applications
• MEMS interference displays can be brighter than any surroundings
• Consistency principle (Evans)• reproduction of relation among colors more important than absolute
colorimetry
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
542.11 Rendering state
• Stock photo agency images are rendered to a normalized intent
• Typical consumer images are the raw output of digital cameras or scanners
• Many CBIR algorithms rely on color histograms
• Need to specify when images are unrendered
• RIMM/ROMM RGB
• Need algorithms to perform automatic rendering operation
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
552.11.1 Digital color image flow
sensorspace
unrenderedspace
renderedspace
devicespace
input devicespecific RGB
colorimetricestimate of
original scene
colorimetricestimate of areproduction
output devicespecific RGB
or CMYK
device and/orimage specifictransformation
device and/orimage specifictransformation
image specifictransformation
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
56
2.11.2 Rendering in Photoshop
printer(CMYK counts)
connection space(XYZ)
input(RGB counts)
display(RGB counts)
connection space(CIELAB or XYZ)
connection space(CIELAB or XYZ)
working color space(Adobe RGB, or …) rendered image
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
573 Data compression
Application
Protocol
Format
Compression
Color image
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
58
3.0.1 Approach
• Waveform coding of color images
• with “waveform” we put the emphasis on the signal, as opposed to its
meaning
• “Avoiding the transmission of information which the eye cannot use” A.V. Bedford, 1950
• Reducing statistical or visual redundancy
• source vs. sink coding
• lossless vs. lossy (visually lossless) coding
• lossless: decompressed image identical to original• lossy: decompressed image tolerably different
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
59
3.0.2 General compression system
• 3 stages: transform — quantize — code
• quantize — lossy
• code — lossless
original compressed
spatial filtercolor transform
spatial transform
scalar quantizervector quantizer
color palette
Huffman codingarithmetic coding
Lempel-Ziv
transform
T
code
C
quantize
Q
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
603.1 Coding methods
• Achieve compression by exploiting statistical redundancy
in the symbol set
• average number of bits cannot be less than the entropy H
• H = –
∑
p
i
log (p
i
), where
∑
p
i
= 1 (
p
i
is the probability of symbol
i
)
• entropy sets bound on performance
• Not all symbols are equally likely
• use short codewords for more probable symbols
• use long codewords for less probable ones
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
61
3.1.1 Encoding methods (cont.)
• Statistics known
• Huffman coding
• method of constructing the optimum prefix code• Arithmetic coding
• represents a symbol string as a binary fraction• typically 5–10% better than Huffman coding, but more complex
• Statistics not known
• Lempel-Ziv (dictionary methods) in 3 flavors: LZ77, LZ78, LZW
• represent a string in terms of previous occurrences using:
• a pointer to the previous occurrence and its length (LZ77)• a dictionary of previous occurrences (LZ78, LZW)
• Flate
• LZ77 followed by Huffman coding
• in some contents authoring tools, Flate encoding is labelled as ZIP
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
62
3.1.2 Huffman coding
Developed 1952 by D.A. Huffman
• Produces the optimum prefix code
• fixed-length symbols to variable-length codewords
1. Order the symbols according to their probabilities
• frequency of occurrence of each symbol known a priori
• in practice, a training set of data is used
2. Merge the two symbols with the smallest probabilities3. Repeat step 2 until one merged symbol is left
• step 2 can be viewed as construction of a binary tree, since at each recursion we merge two symbols
• at end of recursion, all symbols will be leaf nodes of this tree
• the codeword for each symbol is obtained by traversing the binary tree from root to the leaf node corresponding to that symbol
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
63
3.1.2.1 Huffman coding example
• In most Internet imaging applications the size of the alphabet composing these symbols is restricted to at most 64,000 symbols
• Average number of bits cannot be less than the entropy H pi pi( )log∑–=
0.50
0.30
0.18
0.12
0.06
0.03
1.00piV1 000 0.5
V2 001 0.2
V3 010 0.1
V4 011 0.08
V5 100 0.06
V6 101 0.03
V7 110 0.02
V8 111 0.01
Entropy = 2.16
1
0
1
0
1
0
1
0
1
1
1
0
0
0
1
01
0011
0010
0001
00001
000001
000000
2.19
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
643.1.3 Arithmetic coding
• Achieves higher compression than Huffman by combining several symbols into a single unit• a message is encoded as a whole new symbol instead of as separate
symbols
• geometric interpretation: symbols correspond to subintervals in [0, 1)
• Separates coding from modeling• this allows for the dynamic adaptation of the probability model without
affecting the design of the coder
• Many image compression standards allow to substitute Huffman with arithmetic coding• Huffman coding is often the baseline requirement
• arithmetic coding can be used in critical applications
• Covered by patents from IBM, Mitsubishi, and AT&T
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
653.1.4 LZ coding method
Jacob Ziv and Abraham Lempel, 1977 and 1978
• A sliding window is moved across the data stream
• LZ77:• a string is represented in terms of a pointer to the previous occurrence
and its length
• LZ78:• a string is represented in terms of a pointer into a dictionary of previous
occurrences
• a dictionary is built that maps variable length bit strings from the data stream into fixed length codes
• the decoder parses the code sequence, recursively builds the same dictionary, and reconstructs the data stream
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
663.1.5 LZW coding method
Lossless compression of graphics
• Improvement of LZ proposed by Terry Welch in 1984
• Dictionary is initialized with the character set
• Bytes from the input stream are read and used to progressively form larger and larger sequences until a sequence is formed that is not in the dictionary
• The last known sequence’s encoding is output and the new sequence is added to the dictionary
• Typical compression ratio: 2:1
• Implementing LZW may require licensing USP 4,558,302• see http://www.unisys.com/about__unisys/lzw/
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
673.1.6 Flate and deflate
Proposed in 1996 by L. Peter Deutsch
• L77 cascaded with Huffman• window size up to 32K bytes
• Huffman coding of pointers and lengths
• Performance• substantially better compression than LZW
• considerably slower encoding speed than LZW
• same decoding speed
• Usage• PNG format
• gzip, StuffIt, and ZIP archives
• PDF 1.2 and later to compress text, graphics, and indexed image data
• Specification Ver. 1.3, IETF RFC 1951
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
683.2 Binary image compression
• Group 3 1-d (MH) and 2-d (MR)• ITU-T Rec. T.4
• Group 4 (MMR)• ITU-T Rec. T.6
• JBIG — progressive bi-level image compression• ISO 11544 / ITU-T Rec. T.82
• ITU-T Rec. T.85 — application profile for fax
• ITU-T Rec. T.43 — bit-plane coding for color fax images using JBIG
• JBIG2 — lossy/lossless coding for bi-level images• ISO 14492 / ITU-T Rec. T.88
• text halftone, and generic modes
• add color tags to symbols in text mode
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
693.3 Palette colorCounting colors
• 24-bit pixels can represent 16 million colors
• Humans can distinguish 10 million colors
• A 2×3K image contains 6 million pixels
• A 512×512 image contains 250 thousand pixels
• A “typical” 5122 image has 26 thousand colors
• One byte can represent 256 colors
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
703.3.1 Color palettes (mapped color)
• Represent original colors by indices into a map with reduced set of colors (paint by numbers)• choose N colors (palette)
• image dependent (adaptive) or image independent (fixed)• e.g., median cut
• quantize (map) original to palette colors
• use look-up table to map index to palette color
• may use dither in palettized image
quantize
Qoriginal index
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
713.4 Transform coding
• Represent pixels p(x, y) as linear basis functions ci(x, y)
• Coordinate transformation / spectral decomposition• decorrelating original pixels
• compacting signal energy
• matching quantizer to human visual system
• Quantize and code transform coefficients Ci• emphasis on T step of T-Q-C compression model
p x y,( ) k Cici x y,( )∑=
original compressedtransform
T
code
C
quantize
Q
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
723.4.1 Transform coding (cont.)
• Discrete Karhunen-Loeve Transform (KLT) is optimal• uncorrelated coefficients, best energy packing
• image dependent, no fast implementation
• Discrete Cosine Transform (DCT)• image independent, fast transform exists
• performance approaches KLT
• Baseline JPEG standard uses block DCT• Joint Photographic Experts Group
Y k l,( ) 14---C k( )C l( ) S x y,( )
2x 1+( )kπ16
---------------------------cos2y 1+( )lπ
16--------------------------cos
y 0=
7
∑x 0=
7
∑=
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
733.5 JPEG compression methodISO/IEC 10918–1, ITU-T Rec. T.81
• Lossy compression of images
• Pixels are correlated across space• the compaction efficiency of the Discrete Cosine Transform (DCT) is close
to the optimal transform (KLT)
• DCT is an orthogonal and separable transform
• Transformed data is quantized
• Compression is achieved with cascaded entropy coder
• Typical compression ratios (depends on resolution)• 10:1 in RGB
• 25:1 in opponent color spaces
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
743.5.1 JPEG sequential modes of operation
• Sequential DCT• image blocks are coded in scan-like sequence
• Huffman coding (baseline)
• arithmetic coding
• Sequential lossless• DPCM predictive
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
753.5.1.1 JPEG non-sequential modes of operation
• Progressive DCT• image blocks are processed sequentially, but coding is completed in
multiple scans
• spectral selection: successively more coefficients are coded in zig-zag• successive approximation: DCT coefficients are divided by power of 2
before encoding and slices from MSB to LSB are coded• requires buffering
• Hierarchical coding• each image component is encoded as a sequence of frames
• first frame is a low-resolution version of image• subsequent frames are differential frames between source
components and reference reconstructed components• useful for multi-resolution applications
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
763.5.2 Color in JPEG
Very flexible
• No color space specification
• Baseline JPEG: 4 or less color components• Colorimetric color representation is possible
• Full JPEG: 256 or less color components• Discrete spectral color representation is possible
• Compression can be improved with chroma subsampling
Conclusions:
• JPEG can be used for full color communication
• Find way to solve artifact problem in JPEG
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
773.5.2.1 Examples of problems
The same image: original, GIF, JPEG
• GIF can cause color quantization problems due to palettization before LZW compression
• Just changing the q-factor introduces ringing and blockiness artifacts
Soft colors o n gradients
Hot colors o n gradients
Cool colors o n gradients
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
78
3.5.3 JPEG sequential (baseline) pipeline
compressedstreamoriginal raster to
blocktranslation
DCT entropycodingquantization
Huffmantables
quantizationtables
Q
critical knobfor image quality
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
79
3.5.4 The DCT and its kernels
The 64 kernels of thediscrete cosine transform:
Y k l,( ) 14---C k( )C l( ) S x y,( )
2x 1+( )kπ16
---------------------------cos2y 1+( )lπ
16--------------------------cos
y 0=
7
∑x 0=
7
∑=
C8[ ]mn km
m n 12---+⎝ ⎠
⎛ ⎞ π
8----------------------------cos=
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
80
3.5.5 Classical approach: the q-factor
The same image compressed with the same parameters except for an increasing q-factor
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
81
3.5.6 Perceptually lossy compression
• Internet images often include text
• Readability of text is preserved when small features are preserved
• Optimize quantization tables to preserve typeface parts
USP 5,883,979
i t a gterminal
stem
stress
earbar
serif
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
823.5.7 More than just compression
Image processing in the compressed domain
• Optical shortcomings can be compensated• cost reduction
• Geometric transformations
• Preferred rendering
USP 5,850,484
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
833.5.8 Strategies
to optimize the JPEG method
• A discrete quantization table (DQT) can be used for all images of the same class• text
• business graphics
• maps
• drawings
• gradients in various directions
• etc.
• But: image is created only once, downloaded many times• it can be more efficient to compute custom tables for each image:
adaptive algorithm
• Main goal is system balancing!
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
843.6 JPEG 2000 — overviewISO/IEC 15444, ITU-T Rec. T.800
• Wavelet-based follow-on to JPEG• same committee, different contributors
• Single compression architecture• continuous-tone and binary compression
• lossy, lossless, and lossy-to-lossless coding
• progressive rendering
• by quality or by resolution via order of codestream packets
• Offer better compression (~25%) with more features
• More parts coming with extensions, profiles, file formats & conformance
• Part 1 (core decoder) approved as January 2001
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
853.6.1 JPEG 2000 — applications
• Internet and WWW images• low bandwidth, multiple resolutions, random access
• replacement for FlashPix with multiple, tiled JPEG images?
• Mobile applications• error resilience, rate control, progressive decompression
• low bit rate
• Digital photography
• Facsimile and multi-function products
• Compound images
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
863.6.2 JPEG 2000 — features
• Superior low-bit rate performance
• Random access within compressed image
• Multiple resolutions with multi-level wavelet transform
• Can specify bit rate
• Error resilience• re-synchronization of decoder
• Regions of Interest (ROI)• some parts of the image compressed with higher fidelity
• 1–256 color (spectral) components
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
873.6.3 JPEG 2000 — operation
Component transform• reversible for lossless mode• RGB-to-YCbCr for lossy mode
• nonePixel transform
• 2-D discrete wavelet transform• separable, by tiles, multilevel
Scalar bysub-band
skip for lossless mode
Modeling followed by binary adaptive
arithmetic coder
2-D DWTLL LH
HL HHOriginal
2-D DWT LL LH
HL HH
LL LH
HL HH
original compressedtransform
T
code
C
quantize
Q
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
883.6.4 JPEG 2000 — wavelet transform
• 2-level wavelet transform• with JPEG 2000 9×7 filter
2-D DWTLL LH
HL HHOriginal
2-D DWTLL LH
HL HH
LL LH
HL HH
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
893.6.5 Image compressed with JPEG
0.125 bpp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
90
3.6.5.1 Image compressed with JPEG 2000, no ROI
0.125 bpp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
91
3.6.5.2 JPEG 2000 codestream is packetized
• First few packets are such that you can decompress and obtain an image with more quality in the ROI (face) than in the periphery (surround)
• As more packets arrive, you obtain the data to produce better quality in the surround, so that the entire image is rendered at the same quality
• User can truncate the process anywhere in between
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
92
3.6.5.3 Image compressed with JPEG 2000 @ 0.125 bpp
ROI coding (face)
equivalent to 0.125 bpp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
93
3.6.5.4 Image compressed with JPEG 2000 @ 0.25 bpp
ROI coding
equivalent to 0.25 bpp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
94
3.6.5.5 Image compressed with JPEG 2000 @ 0.5 bpp
ROI coding
equivalent to 0.5 bpp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
95
3.6.5.6 Image compressed with JPEG 2000 @ 1 bpp
ROI coding
equivalent to 1 bpp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
96
3.6.5.7 Image compressed with JPEG 2000 @ 2 bpp
ROI coding
equivalent to 2 bpp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
97
3.6.5.8 Image compressed with JPEG 2000 @ 4 bpp
ROI coding
equivalent to 4 bpp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
983.7 Mixed Raster Content — background
in
1
in
2
out
black-and-whitetext and linediagrams
in
1
in
2
out
black-and-whitetext and linediagrams
black-and-whitetext, halftones,stipples, line art,and so on
PSTN
T.4
MH
T.6
MMR
T.85
JBIG
T.42
JPEGCIELAB
T.43
JBIGCIELAB
Multiple, independent compression methods—each optimized for one kind of image content
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
99
3.7.1 Mixed Raster Content — solution
black-and-whitetext & digramsas before,
interchange
T.44
MixedRaster
Content
MRC is a method for using multiple compression methods in raster documents that contain multiple kinds of content
color text andgraphics
black-and-whitetext, halftones,stipples, line art,and so on
in
1
in
2
out
black-and-whitetext and linediagrams
coloredtexttoo
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
100
3.7.2 Fax implementation tree
in
1
in
2
out
black-and-whitetext and linediagrams
in
1
in
2
out
black-and-whitetext and linediagrams
black-and-whitetext, halftones,stipples, line art,and so on
T.4
MH
T.6
MMR
T.85
JBIG
T.42
JPEGCIELAB
T.43
JBIGCIELAB
black-and-whitetext & digramsas before,
T.44
MixedRaster
Content
coloredtexttoo
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
101
3.7.3 Mixed Raster Content — overview
• MRC = Mixed Raster Content
• multi-layer model for representing compound images
• described in ITU-T Recommendation T.44
• originally proposed in joint Xerox/HP contribution• efficient processing, interchange and archiving of raster-oriented pages
with a mixture of multilevel and bilevel images
• Technical approach
• segmentation of an image into multiple layers (planes), by image content
• use spatial resolution, color representation and compression method matched to the content of each layer
• Compound image architecture
• framework for using compression methods
• Performance
• can achieve compression ratios of several 100 to 1 on typical documents
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
102
3.7.4 Mixed Raster Content — model
black-and-white
text & digrams
colored text
blackred
black-and-whitetext & digrams
colored text
Image
3-layer model
• Foreground• multilevel, e.g., text color
• JBIG @ 12 bpp, 100 dpi
• Mask• bilevel, e.g., text shape
• MMR @ 1 bpp, 400 dpi
• Background• multilevel, e.g., contone im.
• JPEG @ 24 bpp, 200 dpi
Image = M • FG + M’ • BG
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1033.7.4.1 MRC model — decomposition by stripe
1 strip/page, 3 layers vs. multiple strips/page, 1-3 layers/strip
If we do not make Recommendationstogether, we will surely not make them at all
Did youever get asinkingfeeling?
But it is better to have proposed a Recommendationand failed, than to never have proposed at all.
But better still to propose and see success in bothdocument and marketplace
Time to pulltogether?
Too late!Too late!
Stripe 1: M
Stripe 2: M, B
Stripe 3: M, B, F
Stripe 4: BStripe 5: M, B, F
Stripe 6: B
Stripe 7: M, F
Based on Fig. 3 & 8 in T.44
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1043.7.5 MRC — test
Create the same-sized files using JPEG and using MRC
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1053.7.5.1 MRC test — decomposition by stripe
Stripe 1: Mask image with FG = red
Stripe 2: Mask image only
Stripe 3: Mask image with graphic in FG
Stripe 4: Mask image only
Stripe 5: Mask (white) with image in BG
Unless otherwise noted, FG is defaultedto black and BG is defaulted to white
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1063.7.6 MRC — performance
Original@ 200 dpi
JPEG@200 dpiCR = 95:1
MRCM — MMR @ 400 dpiFG — JPEG @ 200 dpiBG — JPEG @ 200 dpiCR = 382:1 @ 400 dpi
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1073.7.7 MRC specifications
• Standards• ITU-T Rec. T.44
• TIFF-FX Profile M
• JPM (JPEG2000 standard, Part 6)
• Proprietary• ScanSoft PagisPro
• LizardTech Document Express (DjVu)
• Luratech LuraDocument
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1083.8 Which compression method should I use for my images?
• Users do not like plug-ins — avoid them if possible
• Static pictorial image: JPEG
• Image with few colors: LZ (no dithering)• GIF (§4.3, slide 114) is being replaced by PNG; currently there are still
differences in what is native in browsers
• Vector graphics: do not rasterize• SVG file format (see §4.10, slide 135)
• plug-in still new and huge; not available for all platforms
• High concept vector graphics: scripting• Flash file format
• requires plug-in; not available for all platforms
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1093.8.1 Slide 21 revisited in bytes
uncompressed compressed
238,960
1,427
393,448
8,838
56,331
65K bytes total619K bytes total
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1104 File formats
Application
Protocol
Format
Compression
Color image
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1114.1 Color images
• Image: a rectangular array of pixels• a pixel is an array of samples
• image document: an array of page images
• Two things a file format should do• provide sufficient information to decode an image or rendering or
processing
• height, width, samples per pixel, bits per sample, resolution, color space, compression method, associated images
• image structure, bye ordering• provide useful information about the image
• metadata, e.g., image description, OCR data
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1124.1.1 File formats
How is the data and its metadata stored?
• Specify the structure of a file• file consists of
• metadata (e.g., color space, white point, little endian, big endian)• compressed data
• Text, structure, and meta oriented: HTML, XML
• Image oriented: GIF, PNG, JFIF, FlashPix, TIFF-FX
• Compound document oriented: TIFF-FX, PDF
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1134.2 What is metadata?
• Metadata is machine understandable information about file resources
• The architecture is of metadata represented as a set of independent assertions• assertions about resources are attributes of the resource
• this architecture facilitates programming
• The set of valid attribute names for a context are defined by convention in a vocabulary
• Metadata increases the value of information
• See §4.9.1, slide 129
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1144.3 GIF — Graphics Interchange Format
• Developed by CompuServe, Inc. in 1987
• Protocol for the on-line transmission and interchange of raster graphic data.
• Colors specified in uncalibrated device dependent RGB
• Color is palettized & restricted to power of 2 in [0, 7]
• A GIF data stream can contain several raster-based graphics — this can be used for animations• a optional global color map and a local optional map per image
• The raster data is a string compressed with LZW• sliding window is moved across data stream and dictionary is built
• code size is limited to 12 bits per code
• there are special codes for resetting tables and end-of-stream
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1154.4 PNG — Portable Network GraphicsPatent-free replacement for GIF
• Developed within W3C as license-free alternative to GIF
• Supports palettized color, grayscale, and RGB color• extension chunk for sRGB and ICC profiles
• allows for gamma correction for better cross-platform performance
• not supported by all browsers
• Optional 8-bit alpha channel can be used for transparency• not supported by all browsers
• Only supports single images (no animation)• proposed multi-image version is MNG (Multi-image Network Graphics)
• Compression method is flate
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1164.5 JPEG file formatsThe JPEG standard does not specify a file format; several different formats have been proposed
• ANPA/IPTC — newspaper industry
• ITU-T — ITU-T Rec. T.4 Annex E for color fax
• ETSI — photo videotext, video telephony
• EXIF and Exif Print — digital cameras
• TIFF/EP — digital cameras, ISO/DIS 12234–2
• IOCA — IBM Image Object Content Architecture
• NITFS — intelligence community, DoD
• TIFF — Tag Image File Format rev. 6.0 and later
• PDF — Portable Document Format
• JFIF — JPEG File Interchange Format
• SPIFF — ISO 10918 Part 3
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1174.5.1 JFIF — JPEG File Interchange Format
• Developed by C-Cube Microsystems as a simple file format to exchange JPEG bitstreams• just adds APP0 marker segment with application specific information to a
JPEG datastream, as defined in ISO 10918
• baseline or progressive JPEG
• Simpler than TIFF, but for JPEG only• single codestream, with thumbnail in APP0 marker segment
• quantization and Huffman tables in codestream
• Allows for additional attributes over those of JPEG
• The color space is YCbCr• no provisions for gamma correction
• an offset is applied to turn CbCr into non-negative numbers (see §2.5.4, slide 37)
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1184.5.2 Exif and Exif Print
Exchangeable Image File
• Exif being revised from V2.1 to V2.2, called Exif Print
• New features:• enhanced metadata
• scene modes (portrait, landscape, etc)• more manditory camera data• user / image preference data (sharpness, chroma, …)
• formalized use of sYCC for larger than sRGB color gamut
• ExifPrint sYCC is display referenced for a display with sRGB properties but with no gamut limitations
• ExifPrint sYCC can include ICC profile
• Exif Print was adoped April 2002
• Often used with Digital Print Order Form (DPOF)
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1194.6 TIFF — Tagged Image File Format
• Originally developed by Aldus, acquired by Adobe• current version: TIFF Rev. 6.0
• easily extensible, supports private fields
• several published extensions (notes) and derivatives
• Supports single images and multi-page image documents• images can be striped or tiles
• Supports multiple color spaces• Gray, RGB, palette RGB, CMYK, YCbCr, CIELAB, …
• ICC defines TIFF field for ICC profiles; extension for ICCLAB
• Supports multiple compression types• Group 3 (2 ways: use Compresion=3), Group 4, JPEG (but use TIFF
Technical Note 2 with optional JPEG tables), LZW, Packbits
• Most popular image format, not supported by browsers
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1204.6.1 FlashPix
• Images stored in resolution pyramid• color spaces are PhotoYCC and NIF RGB ≈ sRGB
• Each plane is tiled (64×64 pixels)• each tile is compressed with JPEG
• IIP protocol allows transfer of individual tiles or groups
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1214.6.2 TilePic
• A file format developed by the Berkeley Digital Library Project
• Designed to store tiled data of arbitrary type in a hierarchical, indexed format in order to provide fast retrieval
• Influenced by the FlashPix format
• Based on the GridPix format developed by the Tertiary Disk Project at UC Berkeley
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1224.7 PDF — Portable Document Format
• A PDF file contains a PDF document, a version number, and a directory of important structures in the file
• A PDF document consists of a number of pages• each one is a page description (PostScript imaging model)
• preserves all of the fonts, formatting, colors, and graphics of any source document
• can contain also vector graphics, images, hypertext links, sound, movies
• supported compression methods: JPEG, G-3, G-4, LZW, flate, and run length encoding
• supports ICC profiles
• PDF/X is emerging ANSI standard for digital publication• currently there are no tools, but Adobe’s Acrobat tools can be used in
conjunction with PDF 1.3, which is superset of PDF/X
• Use when layout is important
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1234.7.1 Optimized PDF
A PDF file can be optimized for transmission over the Internet
• All data for first page is at the beginning of the file
• Embedded fonts are subsetted• recommended setting: subset if less than 99% used
• Common page elements are stored only once
• Text and vector graphics are flate compressed
• All uncompressed images are compressed with flate• already compressed images are left intact
• flate is lossless
• Consider down-sampling images in the Distiller
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1244.7.2 PDF for images on the Internet
• PDF has excellent provisions for compression• in PDF the file size for an image is typically 3–5 times smaller than in EPS
• PDF avoids platform dependency of EPS
• Acrobat Exchange can import most file formats and convert them to PDF
• Several graphic programs can save as PDF without Distiller• CorelDraw, Freehand, Illustrator, Photoshop, …
• see also §6.3.1, slide 197
• PDF libraries can be used to generate PDF files from custom programs
• Most text editing programs can import images in PDF files
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1254.7.3 Marking up PDF
• One of the key WWW features is linking
• As a file format for the Internet, PDF has extensive hypertext provisions
• Some authoring tools (e.g. FrameMaker) also have powerful hypertext capabilities
• Hypertext information can be passed from the authoring tool to the Distiller via the pdfmark operator• manual: click help in the Distiller, select pdfmark Guide
• With a PDF library, it is possible to create applications that mark up PDF images for the Internet with metadata• examples: licensing data, copyrights, author, keywords,…
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1264.7.4 PDF short-comings
• Requires a plug-in on Windows OS
• Designed before XML
• Contents is static — it is hard to customize content on the fly for personal experiences• consider SVG as an alternative (see §4.10, slide 135)
• Poorly reflows documents for palm-top devices• conduit only for Windows platform
• easily fails on non-tagged PDF files; cannot find all drawing elements
• crashes on complex files like the course material you are reading
• It is challenging to write software to manipulate PDF documents
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1274.8 HTML, with XHTML comingHypertext Markup Language 4.0
• A subset of SGML format for hypertext documents• originally developed within the IETF, now in W3C
• current version: HTML 4
• XHTML reformulates HTML as an XML application• brings the rigor of XML
• better support of small portable devices
• modularity allows better support for vector graphics, math, etc.
• Images in HTML or XHTML• use the IMG or OBJECT element
• image formats: GIF, JPEG, PNG
• has more to do with the browser than the format
• XHTML is the image format for UPnP Ver. 1.0
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1284.9 XML — eXtensible Markup LanguageFramework for markup languages
• Meta-grammar (data type dictionary) allows to describe any data (users can add new attribute names and tags)
• Document’s flow can be nested (tree instead of list)
• Extensible Style Language (XSL) for appearance
• Grammar can be supplied for structural validation
• Preserve all semantic information on data
• Allows to bring database applications to the WWW
• Use when contents and structure are most important
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1294.9.1 XML for metadata
• Most computer applications manage some metadata in property lists
• XML has the advantage of many available tools• lower implementation cost
• faster implementation
• better interoperability
• Using XML for metadata allows to enforce a grammar• increases quality of metadata, thus the value of the information
• facilitates application to application communication
• As XML is a computer language with a well-defined syntax, it is possible to build equivalence classes and aggregate (merge) information repositories
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1304.9.1.1 RDF — Resource Description Framework
• Early effort to represent metadata for the web
• Simple descriptive data• card index information (Dublin Core)
• privacy information (P3P)
• association of style sheets with documents
• intellectual property rights labeling
• Adobe’s XMP (a.k.a. XAP) is based on this early RDF• will be supported in all Adobe products and their file formats
• open source SDK lowers the entry bar for developers
• supporting XMP is a business and development issue, not a research issue
• Today the research focus for RDF has shifted towards the semantic web (semweb)
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1314.9.1.2 RDF problem 1: DTD vs. schema
• XML DTD does not explicitly support name spaces
• XML schema solves this problem but provides little semantic information
• RDF schema proposal (MetaNet ontology) provides support for richer semantic definitions…
• …however, it provides limited support for local usage constraints, like • closed vocabularies
• occurrence or formatting constraints
• Example for the vocabulary problem• Library of Congress: author
• British Library: creator
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1324.9.1.3 RDF problem 2: ontology merging
• Semantic knowledge is represented in the form of an ontology
• There is no universal ontology representing all human knowledge — we can only create local ontologies
• Today there are no algorithms for automatically aggregating and merging ontologies• example: cook A has an ontology for Mandarin cuisine, cook B has an
ontology for Cantonese cuisine — build an ontology for Chinese cuisine
• Bioinformatics has developed very powerful structure matching and aggregation algorithms for the human genome project• can they provide a solution to the semweb problem?
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1334.9.2 DOM — Document Object Model
For writing XML applications
• Generally, a parser is used to transform an XML file into a data structure — the parse tree
• Applications operate on the parse tree, not on the original file
• Libraries are provided, exploiting the DOM to manipulate the data structure generated by an XML parser• an interface that allows programs and scripts to dynamically access and
update the content, structure and style of documents
• Working Draft released for public review 29 September 2000
• XML is not only for documents• cfr. the role of the scheme language in the late 80s
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1344.9.3 XML for presentation
• Original idea of the WWW is that authors determine contents and its structure, while readers determine the appearance
• Graphic artists need to control appearance
• This dichotomy explains why so much text on the WWW is communicated as images
• XML can be used to define languages for look• XSL is an example of such a styling language
• style for example maps emphasis into an oblique font
• XML can be used to define languages for format• format specifies the placement of individual elements on a page
• the final form can be expressed in an XML language like SVG
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1354.10 SVG — Scalable Vector GraphicsAn XML language for final presentation
• Language for describing 2-dimensional graphics in XML• XML encoding of PostScript imaging operations
• Web-based display of vector data, as well as images and text
• supports styles, scripting, searching and linking
• supported by Illustrator 9.0 and 10 — free viewer plug-in from Adobe
• ‘image’ element• conforming SVG viewers must support JPEG and PNG files
• result of processing is 4-channel RGBA image (A = α-channel)
• Color representations• sRGB or ICC-profile-based color
• profile embedded or accessed via a URI (Uniform Resource Identifiers)
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1364.10.1 SVG — example<?xml version="1.0" encoding="iso-8859-1"?>
<!-- Generator: Adobe Illustrator 10, SVG Export Plug-In . SVG Version: 3.0.0 Build 76) -->
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN"
"http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd" [
<!ENTITY ns_flows "http://ns.adobe.com/Flows/1.0/">
<!ENTITY ns_svg "http://www.w3.org/2000/svg">
<!ENTITY ns_xlink "http://www.w3.org/1999/xlink">
]>
<svg xmlns="&ns_svg;" xmlns:xlink="&ns_xlink;"
xmlns:a="http://ns.adobe.com/AdobeSVGViewerExtensions/3.0/"
width="113.964" height="116.656" viewBox="0 0 113.964 116.656" overflow="visible"
enable-background="new 0 0 113.964 116.656"
xml:space="preserve">
<g id="Layer_1">
<path fill="#FF00FF" stroke="#0000FF" stroke-width="3"
d="M105.362,97.903l-37.386-9.682l-29.169,25.309l-2.345-38.548
L3.377,55.062L39.314,40.92l8.722-37.621l24.555,29.807l38.475-3.33
L90.306,62.34L105.362,97.903z"/>
</g>
</svg>
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1374.11 SMILSynchronized Multimedia Integration Language
• Pronounced “smile”
• Enables simple authoring of interactive audiovisual presentations
• SMIL is typically used for “rich media” (multimedia) presentations which integrate streaming audio and video with images, text or any other media type
• Main feature is capability to synchronize multiple streams
• SMIL is an easy-to-learn HTML-like language, and many SMIL presentations are written using a simple text-editor
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1384.12 MRMLMultimedia Retrieval Markup Language
• XML-based markup language
• Basis for an open communication protocol for content-based image retrieval systems (CBIRSs)
• Separates CBIR engines from their user interfaces• i.e., query formulation from actual query
• Essential for the formulation and implementation of common benchmarks for CBIR
• http://mrml.net/
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1394.13 XML vs. PDF for Web pagesShould I use PDF or XML?
• PDF workflow gives authors full control of contents, structure, and form (layout)
• XML workflow gives authors full control of contents and structure, while the form can be controlled by reader
• In the case of text documents it is easy to reflow the contents algorithmically• a necessity for PDAs and eBooks
• For rich multimedia documents, automatic layout algorithms my produce unacceptable results• the re-incarnation of documents
• XML and PDF may converge
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1404.14 VRML — ISO/IEC 14772-1:1997Virtual Reality Modeling Language
• File format for describing interactive 3D multimedia on the Internet• structured graphics and extra dimensions (z and time)
• application example: color device gamut visualization
• VRML 1.0 specification created by Silicon Graphics, Inc.• based on OpenInventor file format
• Rikk Carey, Gavin Bell, and Chris Marrin
• In December 1997 VRML 2.0 became VRML97
• Web3D Consortium formed 1994
• Requires plug-in
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1414.15 QuickTime
• Platform-independent suite of files, applications, and plug-ins for playing or interacting with a wide range of popular media formats• imported formats: AVI, Flash, MOV, PICT, BMP, GIF, JPEG/JFIF, Photoshop,
PNG, Targa, TIFF, FlashPix, QuickTime Image, DV, MPEG, AIFF, Audio CD, Karaoke, MIDI, MP3, AAC, WAV, Text
• exported formats: AVI, DV Stream, MOV, BMP, JPEG, Photoshop, PNG, Targa, TIFF, QuickTime Image, AIFF, MIDI, MP3, AAC, WAV and Text
• Comprises two managers: the Movie Toolbox and the Image Compression Manager• Movie Toolbox allows to store, retrieve, and manipulate time-based data
that is stored in QuickTime movies
• a single movie may contain several types of data• Image Compression Manager comprises a set of functions that compress
and decompress images or sequences of graphic images
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1424.15.1 QuickTime concepts
media data
metadatavideo
audio
VR
3D
MIDI
graphics
text
…
types of tracks
media index
user datacopyright information
compression formatedit information
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1434.15.2 QuickTime file format
• Metadata part is called movie• index, number of tracks, compression method, timing,…
• Image part is called media data• video frames, audio samples,…
• Movie and media data can be in different files• example: slide shows
• Basic data unit is called atom• atom contains size & type info plus (big-endian) data
• atoms can be nested—containment hierarchy, tree-structure
• QuickTime files consist of atoms
• QuickTime Image File Format• provides container for QuickTime-compressed still images
• supports ColorSync
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1444.16 JPEG 2000
• JPEG 2000 standard will define optional file formats• unique format for branding, syntax based on QuickTime format
• file format is a sequence of boxes (atoms)
• each box has an identifier, length and data• different approach than JPEG
• JPEG standard defined codestream syntax, but not file format
• JPEG 2000 File Format Family (ISO 15444–N)• Part 1 — includes minimal file format: JP2
• Part 2 — includes JP2 extensions: JPX
• Part 3 — Motion JPEG 2000: MJ2
• Part 6 — Compound Images: JPM
• Usage• coming, early adopters: digital cameras
• JP2 will replace FlashPix
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1454.16.1 JPEG 2000 File Format Family
• JP2 (JPEG2000)• single image
• contiguous codestream
• gray, sRGB, restricted ICC profiles, palette, sYCC
• JPX (JPEG2000 EXtensions)• multiple code streams, possibly fragmented
• other color spaces and compression types
• MJ2 (Motion JPEG2000)• timed sequences of JPEG 2000 images
• uses many of the same boxes/atoms as MPEG-4
• JPM (JPEG2000 Multi-layer)• MRC model for JPEG2000 (and other) compressed images
• represents page as a sequence of (Mask, FG) layout objects
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1464.17 TIFF-FX
• Developed within IETF as file format for Internet fax• TIFF representation of image data generated by suite of ITU-T standards
for black-and-white and color facsimile
• TIFF-FX = TIFF for Fax eXtended
• IETF Proposed Standard (RFC 2301)• required by ITU-T Rec. T.37 and IETF RFC 2532 Internet Fax standards
• extension proposed for JBIG2
• Color representation: CIELAB with ITU encoding
B & W colorProfile S — Group 3 1-D (MH) Profile C — JPEG
Profile F — Group 3, 2-D (MR),Group 4 (MMR)
Profile L — JBIG color (T.82)
Profile J — JBIG (T.85) Profile M — Mixed Raster Content
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1474.17.1 Relationship among TIFF-FX profiles
S
ML
CJ F
MHT.4
MH, MR, MMRT.4, T.6
JBIGT.43, T.82
JBIGT.5, T.82
MRCT.44
JPEGT.42, T.81
B&W COLOR
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1484.18 File format overview
• Single image• GIF, PNG, VRML, JFIF, TIFF-FX, JPEG 2000, PDF, QuickTime
• Multiple images• animated GIF, MNG, FlashPix, QuickTime
• Multi-page images• TIFF-FX, PDF
• Compound images• TIFF-FX Profile M, PDF
• Compound documents — vectors, text, images• XML formats, HTML, SVG, PDF, SMIL
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1494.18.1 File format summary
Color spaces supported with the compression methods
• LZW, flate for text, graphics, and indexed images
• JPEG for images
LZW flate JPEG
GIF device RGB n/a n/a
PNG n/a device RGB, sRGB n/a
JFIF n/a n/a YCbCr
FlashPix n/a n/a PhotoYCC, sRGB
TIFF-FX Profile C n/a n/a CIELAB
PDF dev. RGB, dev. CMYK, cal. RGB, CIELAB, XYZ, ICC profiles
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1504.18.2 Internet media types and file extensions
File Format Internet Media Type File Extension
GIF image/gif .gif
PNG image/png .png
JFIF image/jpeg .jpg
FlashPix image/vnd.fpx .fpx
JPEG 2000 image/jpeg2000* .jp2
image/jpeg2000; extended* .jpx
image/jpeg2000-mrc* .jpm
TIFF-FX image/tiff .tif
PDF application/pdf .pdf
HTML text/html .htm, .html
XML text/xml, application/xml .xml†
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
151SVG image/svg-xml* .svg†
Flash application/x-shockwave-flash .swf
VRML model/vrml .wrl
QuickTime movie video/quicktime .mov
QuickTime image .qif
HTTP print job application/ipp
* proposed, not yet registered† generic XML files often have the extension .xml, files for particular XMLapplications or DTDs have specific extensions, such as .svg for SVG or, for ex-ample, .cdf for the Channel Definition Format
File Format Internet Media Type File Extension
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1525 Protocols
Application
Protocol
Format
Compression
Color image
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1535.1 Internet faxWhat is it?
• Store-and-forward Internet fax• scanned document transmission using e-mail attachments
• ITU-T standards and IETF protocols
• uses ESMTP with delivery confirmation and capabilities exchange
• ITU-T Recommendation T.37 — approved September 1999• references IETF standards
• requires use of TIFF-FX
• Simple Mode — TIFF-FX Profile S: April 1999
• minimal b&w with no delivery confirmation or capability exchange• Full Mode — TIFF-FX all profiles: September 1999
• range of b&w and color with delivery confirmation and capability exchange
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1545.1.1 Internet fax
• Advantages• higher bandwidth for color and high resolution
• high confidence SMTP messaging
• Status• B&W Internet fax products available now
• Brooktrout Technology• Daewoo Telecom and Connect One• Dialogic (Intel)• Internet Magic• NetCentric and Cisco• Omtool• Panasonic• Xerox
• Working implementations: Canon, Quality Logic, Interstar Technologies, iReady, KDD, Matsushita, Metasoft, Natural Microsystems, Open Port Technology, Optus Software, Ricoh, WIDE Project, …
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1555.1.2 Internet fax — configurations
Internet
PSTN
workstation
all-in-one
faxon/off ramp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1565.1.3 Internet fax — operation
MUA — Mail User AgentMTA — Mail Transfer AgentMCF — Message ConfirmationMDN — Message Disposition NotificationDSN — Delivery Status Notification
Group 3 fax S&F Internet fax
Image format ITU-T Rec. T.4 TIFF-FX
Addressing +120227653000 recipient@name.org
Content capabilities
ITU-T Rec. T.30 Internet fax schema(RFC 2531)
Transmission point-to-point multi-point
Notification & confirmation
in-bandMCF
out-of-bandMDN, DSN
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1575.2 IPP — Internet Printing ProtocolWhat is it?
• IETF standard developed with help from the Printer Working Group
• Client-server protocol for distributed printing on the Internet• intended to replace LPR/LPD
• Uses HTTP 1.1 POST application protocol• Internet media type: application/ipp
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1585.2.1 IPP — Internet Printing Protocol
Functions
• Get a list of capabilities for a particular printer
• Send a print job to a selected printer
• Check on the progress of a particular print job
• Cancel a previously submitted print job
• Get status from the printer
• Use modern PDLs
• Options for multi-document jobs and print-by-reference
• Event notification
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1595.2.2 IPP — Internet Printing Protocol
Sample configurations
client IPP object
IPP
IPP
client IPP object
Client to printer
Client to server
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1605.2.3 Internet Printing Protocol — model
Printer and job objects
• printer object / printer description attributes• document-format-supported
• Internet media type• compression-supported
• around document, not within document• e.g., none, deflate, gzip, compress
• color supported
printerobject
outputdeviceclient
status
jobsjob control
requests
responses
IPPIPP ordirect connect ornetwork connection
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1615.2.4 Internet Printing Protocol — status
• Current version: IPP 1.1
• Newly published RFCs, September 2002• RFC 3380 IPP: Job and Printer Set Operations
• RFC 3381 IPP: Job Progress Attributes
• RFC 3382 IPP: The ‘collection’ attribute syntax
• Many IPP products available• http://www.pwg.org/ipp/IPP-Products.html
• clients
• small and large print servers
• printers with embedded IPP
• network cards
• software
• test tools
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1625.2.5 Internet Printing Protocol — fax
Internet facsimile status
Internet fax over IPP, used to be Qualdocs
• Real-time alternative to store-and-forward Internet fax• IPP offers negotiation (limited) and delivery receipts
• firewalls will be an issue
• TIFF-FX support mandatory
• PDFax is PDF with:
• image-only • streamable • supports encryption
• synchronous, negotiated image transmission
• use an extended version of IPP 1.1
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1635.2.6 Remote printing and proofing
client Rom
job server
Internet
client Seul
publisher Rio
agency NY
capabilities
RIP server
printerNew Dehli
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1645.2.7 Canon’s approach
Educate user & parametrize ambient conditions
• User is given clear instructions on how to set up equipment• background and glare for soft copy
• illuminance for hard copy
• Low cost sensor is used to assess ambient conditions
• Color appearance model is used to compute a color transformation for the current ambient conditions
• USP 5,521,708; 5,532,848; 5,831,686; 5,900,932; 5,901,243; 6,078,732
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1655.2.8 Electronic Color Proof (ECP)
Imagicolor — Richard Holub
• ECP insures consistent color in a network• resides in a network linking production nodes
• Mediates the• sharing of information about the capabilities of nodal color devices
• interpretation of color image data to the devices
• control of color reproduction
by the devices to a common or a negotiated criterion
• Separable from image data
• Special emphasis on gamut data
• USP 6,043,909; 6,157,735
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1665.3 Digital sendingInternet scanning for the office
• Scanners connected to Ethernet instead of computer
• Documents distributed via e-mail, fax servers, remote printers, or ISV applications
HP 9100C WindowsServer
ImagingApplication
Digital Sender
HP 9100CService
TCP/IP
ApplicationServer
SharedDisk
write read
NOTIFY.DAT
image +metadata
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1675.4 IIP — Internet Imaging ProtocolTransfer tiled images compressed with JPEG
• An HTTP client-server protocol to request FlashPix data from an Internet server
• Motivation:• execution efficiency
• special commands to request image attributes and metadata• supports sequences of tiles
• provisions for security and e-commerce
• locking at the tile level• intelligent caching on proxy servers
• Joint initiative by Hewlett-Packard Company and Live Picture Inc.
• Now a consortium: DIG — Digital Imaging Group
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1685.4.1 JPIP Protocol
• JPIP defines the interactive protocol to achieve the efficient exchange of JPEG 2000 imagery and imagery related data
• The protocol defines the client-server interactions based on a client request and server response
cache model
image
metadata
server capabilities
JPP or JPT streams
index tables
cache
display
metadata
client capabilities
data limits
x-path
client request
server response
HTTP, TCP, UDP
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1695.5 WebDAVWWW Distributed Authoring and Versioning
• Set of extensions to the HTTP protocol which allows users to collaboratively edit and manage files on remote web servers
• Very lightweight, works well on slow networks
• OS support in Windows XP, MacOS X, NetWare 6
• Application support in Office XP, Acrobat, Oracle 9iAS• in Acrobat a user viewing a PDF file can upload comments and edits to a
shared data repository, which can be tapped by and added to by other workers connected to the Web server
• Apple’s iDrive is implemented in WebDAV
• other Microsoft WebDAV application: SharePoint Portal Server
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1705.6 Service discoveryprovide positive user experiences
• “Self-configuring computational infrastructure”• avoid driver installation and finding networked devices
• Automatic service discovery (brokering, agents, avatars)• user specifies a desired service (e.g., print this image in color)
• network finds a resource to fulfill the request
• Proposed architectures either address a few layers of the protocol stack (see §1.1.2, slide 7) or all
•Three major players:•Microsoft — UPnP
•Sun — Jini
•Apple — Rendezvous= IETF zeroconf
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1715.6.1 UPnP — Universal Plug and Play
Device discovery in home networks
• Architecture for pervasive peer-to-peer network connectivity of PCs of all form factors, intelligent appliances, and wireless devices
• Distributed, open networking architecture that leverages TCP/IP and the Web to enable seamless proximity networking in addition to control and data transfer among networked devices in the home, office, and everywhere in between
• Heavily leverages Internet components, including IP, TCP, UDP, HTTP, and XML
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1725.6.1.1 UPnP network components
statetable
controlserver
eventserver
service
service 2service 1
device
service 2service 1
embedded device
UPnP enabled device
service 2
control point
device
UPnP enabled device
UPnP enabled device
root device
service
control point
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1735.6.1.2 Steps in UPnP networking
1. Addressing: supported by TCP/IP, UDP and DHCP or AutoIP2. Discovery: enables control points to locate interesting
devices on a network and their capabilities; the capabilities are announced with unicast and multicast variants of HTTP
3. Description: detailed description of a sought after device; an XML document
4. Control: control point requests actions to be performed; actions are formatted using SOAP (Simple Object Access protocol)
5. Eventing: services may contain variables reflecting their state; control points can request notification by subscribing to a service
6. Presentation: HTML based user interface for a device
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1745.6.2 Jini
• Architecture for the construction of systems from objects and networks• lets programs use services in a network without knowing anything about
the wire protocol that the service uses
• client is taught by each service how to talk to it
• When a service is plugged into a network of Jini services and/or devices, it advertises itself• client finds services by looking for an object that supports the API
• then it will download any code it needs in order to talk to the service
• The Jini architecture uses objects that move around the network to make each service, as well as the entire network of services, adaptable to new strategies over time
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1756 Applications
Application
Protocol
Format
Compression
Color image
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1766.1 Outline
1. Research issues regarding Internet imaging applications• image retrieval
• regions of interest
2. Commercial applications• browsers
• authoring tools
3. Services• basic tools
• three-tier model
• experimental services
• production services
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1776.1.1 Entropy and the secret for success
• As John von Neuman remarked to Claude Shannon, the formula for the information content of a message is mathematically identical to the formula of entropy
• Second law of conservation of energy
• Entropy and information can only incease
• The human endeavor is to counteract to entropy by creating order
• This is the secret for inventing successful Internet appliations
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1786.2 Imaging research for the Internet
• Where can I find a picture of the Golden Pavilion?
• Is there a different view? With autumn color? Sakura?
• Can I organize my images?
• How can JPEG2000 find the regions of interest?
• If I browse my image collection on a PDA, can they be cropped by the server?
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1796.2.1 Image retrieval
Text or contents based
• Text-based image retrieval: images are annotated and a database management system is used to perform image retrieval on the annotation• drawback 1: labor required to manually annotate the images
• drawback 2: imprecision in the annotation process
• Content-based image retrieval systems (CBIRS) overcome these problems by indexing the images according to their visual content, such as color, texture, etc.
• A goal in CBIR research is to design representations that correlate well with the human visual system
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1806.2.1.1 Retrieval metrics
Exact queries are not possible for images (nor text)
• Recall (Sensitivity) = Number of relevant items retrieved / Number of relevant items in database
• Precision (Specificity) = Number of relevant items retrieved / Number of items retrieved
• Algorithms must make a compromise between these two metrics: broad general vs. narrow specific query formulation
• http://www.benchathlon.net/
• CBIR algorithms tend to be very imprecise…
• …the result of a query requires further manual processing
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1816.2.1.2 Navigation vs. searching
Scalability requirements
• Vetting the result of a query requires considerable effort
• An Internet imaging system must allow users to capture the fruits of their vetting labor
• After a search, a good system must provide functionality to organize the retrieved images so they can subsequently be navigated
• Different users navigate differently a given image set
• Navigation is facilitated by general structures, like taxonomies or ontologies
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1826.2.2 Web photo albums
Organizing pictures
• Amateurs: shoe box
• Professionals:• image storage/retrieval of stock photos
• structured work-flow
• operate on single images
view print
put in wallet put in shoe box
great?
capture
catalog
process
archive
retrieve best
Am
ateu
r
Professional
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1836.2.2.1 User model for a Web photo album
• Hypothesis: picture taking households want simple system that can be used iteratively
• Market rule: customer is willing to buy gadgets when their entertainment value is larger than purchase price
• Web Photo Album design goals• publish and retrieve with joy collections of images on the Web
• scalable: many images
• scalable: wide gamut of user skills
capture & archive
process retrieve setcatalog
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1846.2.2.2 Web photo album
Challenges and opportunities
iconography / semantics
digital assets
digital images
1. structure for navigation• external intelligence, categories
• semantics, context, metadata
• thesaurus, taxonomy, ontology
• iconography, context
2. storytelling
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1856.2.2.3 The indexing problem
• Indexing entails categorization
• Categorization is a difficult cognitive task
• High degree of specialization
• Changes in time as iconography evolves
• Categories implemented as keywords
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1866.2.2.4 The category scaling problem
• A typical consumer photo album requires more than 500 keywords → hard to manage
• Easy solution: hierarchical keywords → too difficult for untrained person
• Classical S&R solution: taxonomy (e.g., decimal classification system) → too bulky
• Moreover:• each image has several keywords
• untrained person needs to reclassify
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1876.2.2.5 Solution part I: heaps
• User groups images on desktop by piling them into heaps
• Desktop has arbitrary number of basketsUser groups images by dragging their icons into baskets
• An image can be in several baskets
• Heaps are transientImages are automatically ungrouped• at the end of a session
• when indexing is committed
• a basket is dissolved
CPL PIC-NIC
EventsISLANDS
PlacesPEOPLE
Subjects
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1886.2.2.6 Solution part II: tickets
• On the desktop there are also tickets
• A ticket represents a set of keywords…
• …or filter operations on images (rendering intent prediction, sharpening, special effects, etc.)
• Tickets can be combined by drag & drop
• User indexes images by dragging ticket on top of basket
Events
De-blur
Angel Island
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1896.2.2.7 Solution part III: ticket editor
• Tickets are built in a separate mode (double-click)
• Keywords: User can consult an extensive built-in taxonomy for help in building keyword hierarchy
• Filters: User can select or disable image operations and parameters of filters
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1906.2.2.8 Solution part IV: sequencing
• User can order images by arranging icons
• The result of a query is a heap
• User can expand a heap and arrange the iconsSystem adds a priority, invisible to the user
• An image can have a different priority in each heap
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1916.2.2.9 Conclusions for Web photo albums
A scalable web photo album
• Size: database
• User skills: tickets
• Workflow: iterative
• Effort: heaps
• Algorithms: display list
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1926.2.3 Commercial Web photo albums
• Adobe ActiveShare.com — http://www.activeshare.com/
• Agfa eLab — http://www.agfanet.com/en/ips/fsub_ips.php3
• HP Cartogra — http://www.cartogra.com/ etc.
• † Fuji Film Picture Your Life
• Kodak PhotoNet — http://www.kodak.com/US/en/consumer/aol/aol.shtml
• Lifetouch, Inc. — http://www.lifetouch.com/
• Ofoto — http://www.ofoto.com/
• PhotoAccess.com — http://www.photoaccess.com/
• Seattle PhotoWorks — http://www.photoworks.com/
with http://www.ememories.com
• † PrintLife
• Shutterfly.com Shutterfly — http://www.shutterfly.com/
• † Zing.com ZingAlbums
• …
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1936.2.3.1 Popular business models
• Prints and merchandising
• Web or printed photo albums
• E-services (custom looks, a.k.a. reskinning)
• Example: Indigo Photo-e-Print
http://www.indigonet.com/photo/index.shtml
Film
Scanner
DigitalCamera
HomeConsumers
Internet
DataManagement
System
Prints
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1946.2.4 Algorithms for ROI
• Human vision collects low resolution overview in the retina’s periphery
• High resolution views in the fovea with each fixation as the eye jumps from ROI to ROI under top-down control
L. Stark and C. Privitera, U.C. Berkeley
3K bytes
3K bytes
ROIs
100K bytes
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1956.2.4.1 Intelligent clipping
Image for desktop browser
Image for wireless browser
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1966.3 BrowsersIE has won the browser war
• Microsoft Internet Explorer, AOL Netscape, Mozilla• support GIF, JFIF and PNG
• differ in support of PNG gamma correction and transparency• W3C site has a link to a page that tests compliance:
http://www.w3.org/Graphics/PNG/
• Independent browsers• Opera, OmniWeb, Camino, KHTML/Safari
• Users do not like to install plug-ins• on the open Internet, avoid encodings that require a plug-in
• in intranets plug-ins are acceptable
• WAP vs. iMode
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1976.3.1 Authoring tools
Beyond vi and xv
• Web site and HTML page production• BBEdit, Dreamweaver, GoLive,…
• Image editing• CorelDraw, Ghostscript, GraphicConverter, PBMPLUS, Photoshop, …
• in Photoshop, saving an image as “Single Image PDF” compresses it using JPEG. To choose between JPEG and Flate, save as “Photoshop PDF”
• Vector based illustrations: Freehand, Illustrator,…
• Animation: Flash, ImageStyler, LiveMotion,…
• Image optimization• GraphicConverter, Fireworks, ImageReady, PBMPLUS, …
• Multimedia: Shockwave
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1986.3.2 Instant Delivery
Custom newspapers — an HP product
• Subscription system for newspapers and magazines
• Every night the server polls the publisher’s sites for new material
• Each user receives a custom publication with the articles on subjects they subscribed
• The custom publication is automatically printed on the user’s local printer, ready for breakfast
• Anyone can publish with Instant Delivery
• Currently supported on Windows and MacOS
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
1996.4 Three tier architectureTechnology trends
• With the transition from programs to Web services, the underlying technology has evolved:
1. Objects — Java, C++, Smalltalk2. Components — CORBA, COM3. Services — XML4. Web services — J2EE and .NET
• How are Web imaging services built?
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2006.4.1 Three tier architecture concepts
databases
business logic
client tier
Web servicecontainer
back-endsystems
firewallWeb-based clients
business partner’s app, Web client
internal clients
UI, demarshalling, error checking, converters
transactions, events
billing, customer management, identity
e-mail, printing
W3C standards
mostly proprietary standards
J2EE and.NET
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2016.4.2 Programming tools
Microsoft vs. non-Microsoft
• Java• extensive classes for various format and ICC profiles
• watch out for existing intellectual property if you plan to distribute your software
• Microsoft: C#
• JavaScript• Netscape’s scripting language (was LiveScript)
• also used in Acrobat
• Microsoft: Visual Basic
• Scripting languages — the duct tape of the Internet• Perl — for CGI scripts and SQL database interface
• PHP — for MySQL database interface
• scripts are not architected, are inefficient, are unreliable — avoid them
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2026.4.3 .net to J2EE model comparison
COM-IPC, SOAP
NDR, XML/SOAP, MIME
CLR
Runtime GC
UDDI, DISCO
MS foundation services
.dll, .exe, .asp, …
VB
secu
rity
SMTP, IIOP, RMI, SOAP
CDR, GIOP, XML
Java
BOA/POA
UDDI, Java Inspection
J2 services
.class, bean (primary), .dll, …
Java reflection
security
packaging
helper classes/services
activation/lifecycle
scripting intrfaces
discovery/registration
object model
marshalling format/message types
communicationprotocol
.net
eve
nts
, MTS
, Pas
spo
rt, D
om
ain
, sh
op
pin
g c
art
EJB even
ts, JND
I, JTS, EJB secu
rity, … , EJB
, JAF, JavaM
ail, JMS
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2036.4.3.1 J2 acronyms
EJB Enterprise Java Beans, transactional server components
JAF JavaBean App. Framework, maps data blobs to manipulators
JidlJSDL
Java IDL, Java to CORBA bridgeJava WSDL, Java to SOAP bridge
JavaMail Mail services
JDBC Database access
JDK Java Developer’s Kit
JMS Events/asynchronous messaging
JNDI Abstraction over DNS, LDAP, Novell Directory, CORBA naming
JSP Java Server Pages, replacement to CGI
JTS Transactions
RMI/IIOP Intra-enterprise communication
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2046.4.3.2 .NET
• Microsoft's Internet-based strategy to wire appliances, Web services, and legacy applications on the Internet
• Comprises Biztalk Server 2000 meant for XML document routing on the Internet in a reliable manner
• The .NET platform also comprises Visual Studio based interfaces to wrap legacy business applications as Web enabled services
• Third component of .NET involves Common Language Runtime (CLR) environment, aimed at unseating Java
• .NET does not address the business conventions required for the automatic business-to-business dynamic interactions required in the e-marketplace
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2056.4.4 Simple image Web site
• Client: any Web browser
• Service: Apache server with PHP or Perl scripts
• Back-end: MySQL database on Linux
• Problems with decent size images:• Linux file system too slow
• MySQL not transaction oriented
• scripts too slow, CGI requires initialize processes
databases
client tier Web-based clients
business logic
back-end systems
Web service container
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2066.4.4.1 Simple solutions
• Use a real Unix workstation with high performance disks and file system
• Use a full database like Oracle when you need transactions and stored procedures
• Use Java servlets instead of CGI scripts
databases
client tier Web-based clients
business logic
back-end systems
Web service container
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2076.4.4.2 Java servlets
• Servlets extend the functionality of Web servers• work with Apache, Netscape, iPlanet, MS IIS, etc.
• Like an applet running on the server side
• Process always runs and services all requests
• Full access to all Java classes, in particular to JDBC for database access
• Can be integrated with Web-enabled application servers• BEA WebLogic, IBM WebSphere, iPlanet, etc.
• JavaServer Pages (JSP) extend servlet technology to combine static HTML template data with dynamic content
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2086.4.4.3 Tomcat JavaServer Pages
• Open-source implementation of Java Servlet and JavaServer Pages technologies
• Developed under the Jakarta project at the Apache Software Foundation• Jakarta is an Apache umbrella project that includes 3 subprojects related
to JSP and servlet technology:
• Tomcat, a JavaServer Pages and Java Servlets implementation • Watchdog, a JSP page and servlet validator • Taglibs, a JSP tag library repository
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2096.4.5 Web services
Web services are modular and reusable software components that are created by wrapping a business application inside a Web service interface
• Adobe — AlterCast
• HP — e-Speak
• IBM — WebSphere
• …
databases
client tier Web-based clients
business logic
back-end systems
Web service container
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2106.4.5.1 AlterCast
customer
APIlayer
AlterCastcore layer
application layer
PSD libraries SVG libraries CoolType etc.
AlterCast core
imaging technologies
JavaPerl COM, COM+, .NET comand line
SOAP server
HTML
XML AlterCast commands
database Content mgt. syst. Asset mgt. syst.
shell scriptPerl scripts Java JSP VIsual Basic, VB.NET, ASP
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2116.4.5.2 e-Speak
• Advanced features provided by e-Speak include discovery, negotiation, and mediation of e-services
• There are two components in the e-Speak platform:• Service Framework Specification (SFS)
• e-Speak Service Engine (SE), which is a high performance software implementation of the SFS
• Based on XML and Java
• Supports TCP/IP, HTTP, WAP
• Compatible with Jini and UPnP
• Compatible with pervasive object models• EJB, CORBA, DCOM
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2126.4.5.3 WebSphere
• IBM’s scalable Internet software platform for e-business
• Technology for dynamic business to business process integration — seamlessly links into existing systems• built on standards like Java and XML
• runs on all major operating systems
• supports applications on 35 different platforms
• Foundation of the platform is the WebSphere Application Server• scales from servelets+JSP+XML to EJB to high-volume transactional
applications integrating EJB and CORBA through JTS
• While WebSphere addresses more business conventions than .NET, it still lacks the power and features of e-Speak's SFS business conventions
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2136.5 Summary 1Color space to applications
Web Browsers Image TransferApplication
Protocol
Format
Compression
Color Space
HTTP *TP
HTML
GIF PNG JFIF
LZW flate JPEG
palette
RGB sRGB YCbCrICC
Profile
Other formatsvia plug-ins
e.g., PDF,TIFF, SVG
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2146.5.1 Summary 2
Color space to applications
Application
Protocol
Format
Compression
Color Space
Image Transfer InternetFax
InternetPrinting
*TP IIP *TP ESMTP IPP
JFIF FlashPix JP2 TIFF-FX
JPEG JPEG JPEG 2000JPEG
MRC
Binary
SupportedDocumentFormatsProfile
CProfile
M
YCbCr sRGB Photo-YCC
sRGB,Gray
SimpleICC profile
CIELAB
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2157 Conclusions
• Take a top-down systems design approach
• There is a lot available• GIF, JFIF, HTML, …
• … And a lot happening• MRC, IPP, Internet Fax,...
• …With more coming• JPEG 2000, SVG, wireless...
• All fueled by the possibilities offered by the Internet and the Web
• The challenge will be delivering the desired color via all these possibilities for Internet color imaging
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2168 Issues and futures
• Will JPEG 2000 replace JPEG?
• The graphic arts industry is accustomed to CMYK; why was CMYK never mentioned?
• Can I use ICC profiles?• variable appearance documents (browsers)
• fixed appearance documents (PDF)
• How do I control the viewing conditions?• color robustness vs. color fidelity
• display calibration widgets
• environment sensors
• How can I create digital assets?• syndication (hitch-hiking), e-services, metadata, digital rights
management, image retrieval
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2179 Acknowledgements
• Carl-Uno Manros for his inputs on IPP
• Ricardo de Queiroz for supplying the wavelet-transformed images
• David McDowell for his insights on standardization activities and clarifications• Standards Update column in IS&T Reporter is an excellent source for
information on current imaging standardization activities
• James King for his insights into XML
• Lawrence Stark and Claudio Privitera for many discussions on intelligent image processing and illustrations
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
2189.1 Acknowledgements (continued)
• Jean Gastinel for showing emerging display technologies
• David Boggs, Neil Gunther, Keith Moore, Peter Schnorf for their insights on communications and Web services
• Mark Gorzynski for his input on color management and file formats
• Gabriel Marcu for illustrations and ColorSync advice
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
21910 Bibliography
Color encoding: Giorgianni & Madden, Digital Color Management, Addison Wesley, Reading, 1998
Compression: Bhaskaran & Konstantinides, Image and Video Compression standards, Kluver, Boston, 1997Held & Marshall, Data and Image Compression, Wiley & Sons, Chichester, 1996
Fax: MCConnell, Bodson, & Schaphorst, FAX: Digital Facsimile Technology and Applications, 2nd ed., Artech House, Boston, 1992
JPEG: Pennebaker & Mitchell, JPEG Still Image Data Compression Standard, Chapman & Hall, 1993
PDF: Thomas Merz, Web Publishing with Acrobat/PDF, Springer Verlag, Berlin, 1998
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
22011 Links to references
Latest version of this list:http://www.inventoland.net/imaging/cii/links.html
Contents based image retrieval (CBIR):http://www.hpl.hp.com/techreports/2000/HPL-2000-162.htmlhttp://www.benchathlon.net/http://www.almaden.ibm.com/almaden/hermitage.htmlhttp://www.virage.com/index.htmlhttp://vrw.excalib.com:8015/csthttp://viper.unige.ch/
DjVu: http://djvu.research.att.com/http://www.lizardtech.com/http://www.luratech.com/
e-Speak:http://www.bluestone.com/products/hp_web_services/
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
221FlashPix: http://www.digitalimaging.org/http://www.Kodak.com/go/flashpixhttp://elib.cs.berkeley.edu/tilepic/http://now.cs.berkeley.edu/Td/GridPix/
Flate: http://www.faqs.org/rfcs/rfc1950.htmlftp://ftp.isi.edu/in-notes/rfc1950.txthttp://www.faqs.org/rfcs/rfc1951.htmlftp://ftp.isi.edu/in-notes/rfc1951.txt
GIF: ftp://ftp.ncsa.uiuc.edu/misc/file.formats/graphics.formats/http://www.unisys.com/unisys/lzw/
Graphics: http://www.w3.org/Graphics/
HTML: http://www.w3.org/MarkUp/
HTTP: http://www.w3.org/Protocols/rfc2616/rfc2616.html
ICC: http://www.color.org/ see also
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
222http://www.hpl.hp.com/techreports/1999/HPL-1999-110.html
IETF (Internet Engineering Task Force): http://www.ietf.org/
IEEE Computer Society Guide to Web Resourceshttp://www.computer.org/internet/links.htm
Instant Delivery: http://www.instant-delivery.com/
Intelligent image processing:http://scan.berkeley.edu/research/eye/eye.stm
Internet fax: http://www.ietf.org/html.charters/fax-charter.htmlhttp://www.ifaxbus.org/http://www.humancomm.com/aboutinetfax.htm
Internet media types:ftp://ftp.isi.edu/in-notes/iana/assignments/media-types/media-types
IIP: http://www.digitalimaging.org/
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
223IPP: http://www.pwg.org/ipp/http://www.pwg.org/ipp/IPP-Products.html
ITU-T standards: http://www.itu.ch/publications/bookstore.html
JBIG: ftp://ftp.informatik.uni-erlangen.de/pub/doc/ISO/JBIG/
JFIF: http://www.w3.org/Graphics/JPEG/jfif3.pdfhttp://icib.igd.fhg.de/icib/it/defacto/research/jfif/read.htmlftp://ftp.ncsa.uiuc.edu/misc/file.formats/graphics.formats
Jini: http://www.sun.com/jini/overview/
JPEG: http://www.w3.org/Graphics/JPEG/http://www.jpeg.org/public/jpeglinks.htmftp://ftp.uu.net/graphics/jpeg/
JPEG 2000: http://www.jpeg.org/JPEG2000.htm
LZW: http://www.unisys.com/unisys/lzw/
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
224MNG: http://www.cdrom.com/pub/mng/
MPEG: http://mpeg.telecomitalialab.com/
MRC: http://www.xerox.com/research/xac/mrc/index.htmhttp://www.scansoft.com/products/pagismill/
MRML: http://mrml.net/
PBMPLUS: http://www.acme.com/software/pbmplus/
PDF: http://partners.adobe.com/asn/developer/PDFS/TN/PDFSPEC.PDF
http://www.ghostscript.com/http://www.pdflib.com/
PDF/X: http://www.ddap.org/solutions/pdf-x_faqs.html
PNG: http://www.cdrom.com/pub/png/ http://www.w3.org/Graphics/PNG/
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
225QUALDOCS: http://www.ietf.org/internet-drafts/draft-moore-qualdocs-protocol-00.txt
QuickTime: http://developer.apple.com/techpubs/quicktime/qtdevdocs/RM/pdfframe.htm
Rendezvous:http://www.apple.com/macosx/jaguar/rendezvous.htmlhttp://www.opensource.apple.com/projects/rendezvoushttp://www.ietf.org/html.charters/zeroconf-charter.html
RFC####: ftp://ftp.isi.edu/in-notes/rfc####.txthttp://www.rfc-editor.org/
Server side color management:http://www.coloreal.com/http://www.ecolor.com/http://www.gretagmacbeth.com/http://www.praxisoft.com/products/internet.htmlhttp://www.verifi.net/
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
226SMIL: http://www.w3.org/AudioVideo/
sRGB: http://www.srgb.com/http://www.w3.org/Graphics/Color/sRGB
SVG: http://www.w3.org/Graphics/SVG/
Test images: ftp://nic.funet.fi/pub/graphics/misc/test-images/
TIFF: http://partners.adobe.com/asn/tech/tiff/index.jsp
TIFF/EP:
TIFF-FX: ftp://ftp.isi.edu/in-notes/rfc2301.txthttp://www.ietf.org/internet-drafts/draft-ietf-fax-tiff-fx-07.txthttp://www.xerox.com/research/xac/tiff-fx/index.htm
Universal Plug and Play (UPnP): http://www.upnp.org/
VRML: http://www.vrml.org/
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
227WebDAV:http://www.ietf.org/html.charters/webdav-charter.htmlhttp://www.webdav.org/
Web photo albums:http://www.activeshare.com/http://www.agfanet.com/en/ips/fsub_ips.php3http://www.apalo.com/http://www.cartogra.com/http://www.pictures.fujifilm.com/pictures/http://www.indigonet.com/photo/index.shtmlhttp://www.kodak.com/go/photonet/http://www.ofoto.com/http://www.photoaccess.com/http://www.photoworks.com/http://www.shutterfly.com/http://www.zing.com/
WebSphere: http://www.ibm.com/websphere
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
228Wireless:WAP: http://www.wap.com/, http://www.wapforum.org/WML: http://www.oasis-open.org/cover/wap-wml.htmliMode: http://www.nttdocomo.com/
W3C (World Wide Web Consortium): http://www.w3c.org/
XHTML: http://www.w3.org/TR/xhtml1/
XML: http://www.w3.org/XML/
Zero Configuration Networking:http://www.ietf.org/html.charters/zeroconf-charter.html
Latest version of this list:http://www.inventoland.net/imaging/cii/links.html
R.R. Buckley & G.B. Beretta VCIP 2003— Lugano, 8 July 2003 T3 — Color Imaging on the Internet
22912 Conference
From January 2000, the IS&T/ SPIE Symposium on Electronic Imaging: Science and Technology at Photonics West has an annual conference on Internet Imaging
http://electronicimaging.org/
Proceedings are available from
http://spie.org/app/Publications/index.cfm?fuseaction=proceedings&type=byconf
Recommended