Lecture09Lecture09

Push & Immediate Mode Push & Immediate Mode ImagingImaging

Push Imaging Model (PIM)Push Imaging Model (PIM) ImageProducerImageProducer is an interface for objects which can is an interface for objects which can

produce the image data for Images.produce the image data for Images. ImageConsumerImageConsumer is an interface for objects expressing is an interface for objects expressing

interest in image data through the ImageProducer interest in image data through the ImageProducer interfaces. When a consumer is added to an image interfaces. When a consumer is added to an image producer, the producer delivers all of the data about the producer, the producer delivers all of the data about the image using the method calls defined in this interface.image using the method calls defined in this interface.

ImageConsumer’sImageConsumer’s ImageProducer’sImageProducer’s

ImageFilterImageFilter

PixelGrabberPixelGrabberFilteredImageSourceFilteredImageSource

MemoryImageSourceMemoryImageSource

RenderableImageProducerRenderableImageProducer

Image Manipulation in Push Imaging Model

Basic single pixel level image Basic single pixel level image manipulation can be done with push manipulation can be done with push imaging model.imaging model.• Image Scaling• Image Filtering

Image Scaling in PIMImage Scaling in PIM

getScaledInstance(int width, int height, int hints) method of the Image class can be used to scale an image and return a new Image object. hints specifies the algorithm to be used for image resampling. width and height specifies the dimensions of the new image.

Image Filtering in PIM

CropImageFilter : Crop out certain region of the image pixels.

RGBImageFilter : This subclass can be used to change individual pixel values. In order to use this class, public int filterRGB(int x, int y, int rgb) method of it must be defined.

Image Filtering The ImageFilter class objects are ImageConsumer objects

which allow them to receive data from an ImageProducer. These filters get wrapped in an ImageProducer called java.awt.Image.FilteredImageSource which sends out the filtered data. The ImageObserver is told the status of the image loading in the final ImageConsumer through calls to its imageUpdate method.

// ...Image originalImg = getImage(url);ImageFilter ifil = new CropImageFilter(0,0,64,64);ImageProducer ip = new

FilteredImageSource(originalImg.getSource(),ifil);Image filteredImg = createImage(ip);// ...

PixelGrabber

PixelGrabber is an ImageConsumer that can collect pixel data into an array such that pixels can be processed in its entirety before sending it out again.

Thus PixelGrabber marks the end of the push model and the beginning of an immediate mode model because the image data is put into an array instead of being passed to another ImageConsumer.

PixelGrabberPixelGrabber grabber = new PixelGrabber(originalImg,

0, 0, -1, -1,true);try {

if (grabber.grabPixels()) {int width = grabber.getWidth();int height = grabber.getHeight();int[] originalPixelArray = (int[]) grabber.getPixels();

} else {System.err.println("Grabbing Failed");

}} catch (InterruptedException e) {

System.err.println("PixelGrabbing interrupted");}

PixelGrabber Once you have pixel data in the originalPixelArray

as in the above code, we can process it resulting in, say, newPixelArray.

This post-processed data can be given to a ImageProducer such as java.awt.image.MemoryImageSource so that the push imaging pipeline can be started again as follows:

MemoryImageSource mis;mis = new MemoryImageSource(width, height,

newPixelArray,arrayOffset,scanLength);Image filteredImage = createImage(mis);

Animation with MemoryImageSource

An Example code:\\ declare variables\\ create the pixels (a pixel array)source = new MemoryImageSource(width,

height, pixels, 0, width);source.setAnimated(true);Image image = createImage(source);\\ do other stuff

\\ Continued next slide

Animation with MemoryImageSource

Thread me = Thread.currentThread( );while (true) {

try {thread.sleep(10);

} catch( InterruptedException e ) { return;}// Modify the values in the pixels array at (x, y, w, h)// Send the new data to the interested // ImageConsumerssource.newPixels(x, y, w, h);

}\\ Do other stuff

Immediate Mode Imaging Model of Java2D

This model is centered around the subclass java.awt.image.BufferedImage of the class Image. This is achieved by having each BufferedImage contain both a Raster and a ColorModel.

Basically, this model provides memory allocation and storage of all image data, thus making it available to the programmer at all the times just as if you collected all the pixel data using a PixelGrabber in the older push model.

Image object into a BufferedImage

1. Make sure that all the image data is loaded.

2. Create a new BufferedImage using the image width, height and image data type (usually BufferedImage.TYPE_INT_ARGB).

3. Obtain the BufferedImage’s Graphics2D object.

4. Using this graphics object, draw the image onto the BufferedImage as done in the double buffering.

Image object into a BufferedImage

static public BufferedImage createBufferedImage(Image imageIn,int imageType) {

Label dummyComponent = new Label();// you can use any // component here

MediaTracker mt = new MediaTracker(dummyComponent);mt.addImage(imageIn,0); // 0 is a given identifier of type inttry {

mt.waitForID(0); // 0 is the identifier} catch (InterruptedException ie) {}BufferedImage bufferedImageOut = new BufferedImage(

imageIn.getWidth(null),imageIn.getHeight(null), imageType);

Graphics g = bufferedImageOut.getGraphics();g.drawImage(imageIn,0,0,null);return bufferedImageOut;

}

Filtering with BufferedImage

When filtering with BufferedImage, you can perform more advanced filtering operations such as convolution and geometric transformations.

Filters for BufferedImages tend to give the alpha chanel special consideration whereas filters for Rasters don’t. This is because BufferedImages contain a ColorModel that allows interpretation of the color components, and with Raster no such interpretation is possible.

BufferedImageOp and RasterOp Interfaces

When performing BufferedImage filtering, much of the functionality of the used filter will be defined by the BufferedImageOp interface.

When performing Raster filtering, much of the functionality of the used filter will be defined by the RasterOp interface.

The following classes all implement both BufferedImageOp and RasterOp interfaces:• AffineTransformOp, RescaleOp, ConvolveOp,

LookupOp• BandCombineOp only implements the

RasterOp interface.

BufferedImageOp

The BufferedImageOp has the method filter to perform filtering:

public BufferedImage filter( BufferedImage src,

BufferedImage dest);• If the color models for the two images do not

match, a color conversion into the destination color model is performed. If the destination image is null, a BufferedImage with an appropriate ColorModel is created. Returns the filtered BufferedImage.

AffineTransformOp

Affine transformations are linear transformations and are made up of rotations, translations, scalings, and shearings. A 2D affine transform is represented by the following equations:

121110

020100

mymxmy

mymxmx

oldoldnew

oldoldnew

AffineTransformOp - An example

// Obtain the sourceBI (the source BufferedImage)AffineTransform at = new AffineTransform();at.scale(2.0,2.0);BufferedImageOp bio;bio = new AffineTransformOp(at,

AffineTransformOp.TYPE_BILINEAR);destinationBI = bio.filter(sourceBI,null);// ...

AffineTransformOp - Methods

// rotate theta radians around the originpublic void rotate(double theta);// rotate theta radians around the point x,ypublic void rotate(double theta, double x, double y);// scale by sx in x direction, and sy in y directionpublic void scale(double sx, double sy);// translate by tx in x direction, and ty in y directionpublic void translate(double tx, double ty);// shear using multipliers of shx and shypublic void shear(double shx, double shy);

ConvolveOp The java.awt.image.ConvolveOp class convolves

a kernel with a source image in order to produce a destination image.

A kernel can be thought of as a two-dimensional array with an origin. During the convolution, the origin of the array is overlaid on each pixel of the source image.

This origin value is multiplied by the pixel value it is over, and all surrounding kernel array values are multiplied by the pixel values that they are over.

Finally, all these values are summed together and the resulting number replaces the pixel corresponding to the kernel centre.

Edge Conditions of ConvolveOpEdge Conditions of ConvolveOp When using convolution algorithms, edge

pixels presents difficulty and hence some special instructions are needed as to how these edge pixels are handled.

There are two such instructions defined in ConvolveOp class:• EDGE_NO_OP - Pixels at the edge of the source

image are copied to the corresponding pixels in the destination without modification.

• EDGE_ZERO_FILL - Pixels at the edge of the destination image are set to zero. This is the default.

ConvolveOpConvolveOp ConvolveOp(Kernel kernel) ConvolveOp(Kernel kernel, int edgeCondition,

RenderingHints hints)

public Kernel(int width, int height, float[] data)• width - width of the kernel • height - height of the kernel • data - kernel data in row major order• The X origin is (width-1)/2 and the Y origin is (height-1)/2.

The RenderingHints class contains rendering hints The RenderingHints class contains rendering hints such as interpolation type that can be used by such as interpolation type that can be used by the Graphics2D class, and classes that implement the Graphics2D class, and classes that implement BufferedImageOp and Raster. BufferedImageOp and Raster.

Example 2D Haar KernelsExample 2D Haar Kernels

11

11

4

1

11

11

4

1

11

11

4

1

11

11

4

1

hhlh

hlll

Where

hhlh

hlll

Other Operations RescaleOP : This class multiplies each pixel by a

scaling factor before adding an offset to it. Mathematically, this can be expressed as follows:

dstSample = (srcSample*scaleFactor) + offset

LookupOp : This provides a means to filter Rasters and BufferedImages using a lookup table(LUT). The LUT is simply an array in which the source pixel samples are treated as array indices.

ColorConvertOp : This class performs a pixel by pixel color conversion of the image source into the image destination. This is done by converting the pixels from the source image’s color space into the destination image’s color space.