Transcript Java Advanced Image

Java and Imaging
Paint
• paint() method
• AWT Coordinate System
• Color & ColorModel
– the java.awt.Color class
– the java.awt.image.ColorModel class
paint() method
Java
Peer
to draw, inherit Canvas, override paint
redraw screen
WINDOW_
paint()
EXPOSED
update()
Event
repaint()
clear component
window
AWT Coordinate System
The java.awt.Color class
• Encapsulate RGB colors
• have static final Color object of frequent-used
colors
public static final Color black;
public Color(int red, int green, int blue);
public Color brighter();
public Color darker();
public int getRed();
java.awt.image.ColorModel
• encapsulate the METHODS of TRANSLATION
from PIXEL VALUES
• get alpha, red, green, blue value from pixel
• implemented by DirectColorModel,
IndexColorModel
public abstract class ColorModel;
public colorModel(int nbits);
public abstract getAlpha(int pixelvalue);
public static ColorModel getRGBdefault();
// return default ColorModel 0xAARRGGBB
Drawing
• The Graphics class
• Simple Drawing
• Working With Text
– Font
– FontMetrics
java.awt.Graphics class
• contain current graphics context and methods
for drawing
• graphics context : fgcolor, bgcolor, font, etc.
• abstract class - actual java implementation and
native library is provided by virtual machine
Drawing Methods of Graphics
Simple Drawing
import java.awt.*;
import java.awt.event.*;
public class DrawingCanvas
extends
Canvas implements
MouseListener {
protected Point pt;
public DrawingCanvas() {
pt = new Point(50, 50);
addMouseListener(this);
}
public Dimension
getPreferredSize() {
return new Dimension(100,
100);
}
public void mouseClicked(MouseEvent e) {
pt = e.getPoint();
repaint();
}
public void mouseEntered(MouseEvent e) {
}
public void mouseExited(MouseEvent e) { }
public void mousePressed(MouseEvent e) {
}
public void mouseReleased(MouseEvent e)
{}
public void paint(Graphics g) {
g.setColor(Color.blue);
g.drawRect(pt.x, pt.y, 10, 10);
}
java.awt.Font
• load Font object from font name
public Font(String name, int style, int size);
public static Font getFont(String nm);
java.awt.FontMetrics
• has methods to get information from Font
object
public FontMetrics(Font fn);
public getAscent();
public getDescent();
Using images
• The Image class
• Loading images
– Loading
– Keep Track of Image Loading
• With the MediaTracker class
• With the ImageObserver interface*
• Displaying images
Using images • Manipulating images*
– imagefilter
java.awt.Image class
• encapsulate image
• programmer can’t know about internal data
structure representaing image
public abstract Graphics getGraphics();
public abstract int getWidth();
public abstract int getHeight();
public Image createImage(ImageProducer);
public Image createImage(int w, int h);
Loading
• automatic loading about JPEG and GIF images
• background loading
• use methods in java.applet.Applet or
java.awt.Toolkit
• load via filename or url
public abstract Image Toolkit.getImage(URL url);
public abstract Image Toolkit.getImage(String file)
Toolkit Component.getToolkit() or static Toolkit
Toolkit.getDefaultToolkit()
java.awt.MediaTracker
• keep track of image loading
public MediaTracker(Component comp);
public addImage(Image img, int id);
public boolean checkAll();
public boolean checkID(int id);
public boolean isErrorAny();
public void waitForAll();
java.awt.MediaTracker MediaTracker tracker;
tracker = new MediaTracker(this);
Toolkit tk = Toolkit.getDefaultToolkit();
for (int i = 1; i <= 10; i++) {
images[i-1] = tk.getImage("image" + i
+ ".gif");
tracker.addImage(images[i-1], i);
}
try {
//Start downloading the images. Wait until
they're loaded.
tracker.waitForAll();
} catch (InterruptedException e) {}
// in paint();
if (!tracker.checkAll()) {
g.clearRect(0, 0, d.width, d.height);
g.drawString("Please wait...", 0,
d.height/2);
}
//If all images are loaded, draw.
else {
...//same code as before...
java.awt.image.ImageObserver interface
loading
image
imageUpdate()
width known
height known
loading
ImageObserver
java.awt.image.ImageObserver
public interface ImageObserver {
public abstract boolean imageUpdate(image img, int infoflags,
int x, int y, int width, int height);
}
If image knows width and height, or loading pixels, call
this methods with infoflags. if return true, no more call
this methods
Component implements ImageObserver
if not completely loaded yet, repaint()
java.awt.image.ImageObserver
• public int Image.getWidth(ImageObserver observer)
– If the width is not known yet then the
ImageObserver will be notified later and -1 will be
returned.
• public boolean drawImage ( Image img, int x, int y, int
width, int height, Color bgColor, ImageObserver
observer )
– return immediately. return ture if image is fully
initialized.if not, return false, and let img notify
observer
Displaying images
• public abstract boolean drawImage(Image img, int x,
int y, ImageObserver observer);
• public abstract boolean drawImage(Image img, int x, int
y, int width, int height, ImageObserver observer);
• public abstract boolean drawImage(Image img, int x, int
y, Color bgcolor, ImageObserver observer);
• public abstract boolean drawImage(Image img, int x, int
y, int width, int height, Color bgcolor, ImageObserver
observer);
ImageFilter -
ImageProducer
ImageConsumer
startProduction() setPixels()
ImageFilter ImageProducer
ImageFilter
ImageConsumer
ImageFilter filter = new RotateFilter(angle);
ImageProducer producer = new
FilteredImageSource(souceImage.getSource(), filter);
Image resultImage = createImage(producer);
Improving Animation
• Eliminating Flashing
–
–
–
–
Overrideing update() method
Clipping
Double Buffering
Speed up Image Loading with Clipping
Overriding update() method
• default update()
– clear all component's Background and call paint()
– Component’s member method
public void update(Graphics g) {
DoActualDrawing();
}
public void paint(Graphics g) {
update(g);
}
Clipping
• public abstract void clipRect(int x, int y, int width, int
height);
– Clipping the Drawing Area
public void update(Graphics g) {
Graphics g2 = g.create(x, y, w, h);
paint(g2);
}
Double Buffering
• update drawing area one time
Dimension offDimension;
Image offImage;
Graphics offGraphics;
public void update(Graphics g) {
if( (offGraphics == null) || ( d.width != offDimension.width) ||
(d.height != offDimension.height) ) {
offDimension = d;
offImage = createImage(d.width, d.height);
offGraphics = offImage.getGraphics();
}
DoActualDrawing(offGraphics);
drawImage(offGraphics, 0, 0, this);
Speed up Image Loading with
Clipping
• when loading images using URLs, most of the time is
taken up by initiating HTTP connections.
– make an image strip
int stripWidth = imageStrip.getWidth(this);
int stripHeight = imageStrip.getHeight(this);
int imageWidth = stripWidth / numImages;
g.clipRect(0, 0, imageWidth, stripHeight);
g.drawImage(imageStrip, -imageNumber*imageWidth, 0, this);
New Features of Java1.1
•
•
•
•
Enhancement on Graphics Clip
Cropping, Scaling, and Flipping
MemoryImageSource
PixelGrabber
Enhancement on Clipping
• New methods of Graphics
Shape getClip();
void setClip(Shape clip);
void setClip(int x, int y, int w, int h);
void getClipBounds();
Enhancement on Clipping
Graphics g2 = g.create();
g2.clipRect(10, 10, 100, 100);
g2.drawImage(img2, 10, 10, this);
g2.dispose();
Shape oldclip = g.getClip();
g.clipRect(10, 10, 100, 100);
g.drawImage(img2, 10, 10, this);
g.setClip(oldclip);
Java Media APIs
•
•
•
•
Java2D
Java Advanced Imaging (JAI)
Java Media Framework (JMF)
Java3D
java.awt.image
• Image is an abstract class
• Image objects are constructed in a platform
specific manner
• Two types of image objects:
– Images created from an image source
– Images created by an AWT component
• Component.createImage()
Processing Images in AWT
ImageProducer
PixelGrabber
MemoryImageSource
MemoryImageSource
JAI models
• The producer/consumer model
– 基本的 AWT 影像 model
Push model
• The immediate mode model
– 進階的 AWT 影像 model
• The pipeline model
Pull model
– The JAI 所使用的影像 model
AWT Push model
Java 2D Immediate mode
Pull Mode
Image
ImageProducer
ImageConsumer
ImageObserver
BufferedImage
Raster
BufferedImageOp
RasterOp
RenderableImage
RenderableImageOp
RenderedOp
RenderableOp
TiledImage
BufferedImage
java.awt.image.BufferedImage
Color Model
Raster
ColorSpace
SampleModel
DataBuffer
Encapsulates the methods for translating a pixel value to color
components (ex. red/green/blue) and an alpha component for rendering
DataBuffer
• java.awt.image.DataBuffer
• DataBuffer stores pixel data as one or more arrays of
primitive data types.
• Java2D buffer types:
–
–
–
–
Byte
Integer
Short
Unsigned short
• JAI buffer types:
– Double
– Float
Sample Model
• SampleMode describes how pixels are stored in
the DataBuffer.
– Packed Models
• Single-pixel packed model
• Multiple-pixel packed sample model
– Component Models
• Component sample model
ColorModel
• Converts raster pixel samples into color
components for displaying
– Pixel samples are packed into a short integer (16
bits). Each sample would be 5 bits.
– If displayed on a color monitor (0-255) per channel,
the resulting image would appear dark.
– Color model handles the conversion.
ColorSpaces
• Device independent/dependent
• Converts between devices
• Supports many color spaces including:
–
–
–
–
–
CMYK
HSV (HSB)
CIEXYZ
sRGB
ICC Color Profile
• Create ColorSpace via factory
Code Example
Creating an Image
Alpha Channel
• A mixing factor to control the linear
interpolation of foreground and background
colors.
• Varies from 0 to 1
– 0 indicates no coverage (clear)
– 1 indicates full coverage
• Each sample is multiplied by the alpha channel
Transformations
• Definitions:
– Transformation is a function that maps an object from one
point to another.
– Affine transform preserves:
• Finiteness
• Parallelism
• Transformations:
•
•
•
•
•
Translation
Scaling
Rotation
Reflection
Shear
Translation
y
x
x' x  dx
y' y  dy
x' x  dx 
     
y' y  dy 
Scaling
y
x
x' sx  x
y' sy  y
x' sx 0 x 
 
  
y' 0 sy y 
Rotation
y
x
x' x cos( ) y sin(  )
y' y sin(  ) y cos( )
x' cos( )  sin(  )x 
  
 
y' sin(  ) cos( ) y 
Reflection
y
x
1

0

0
0
1
0
0x
 
0y
1

1

Shear
y
x
1

0

0
shx
1
0
0x
 
0y
0

1

Transformations
• Transformations can be concatenated:
– C = Translate * Scale * Rotate * Shear
• Java Classes:
– AffineTransform ( java.awt.geom )
– AffineTransformOp ( java.awt.image )
Interpolation
• What is Interpolation?
– Interpolation is a process of generating a value of a pixel
based on its neighbors.
• Types of Interpolation:
–
–
–
–
–
Nearest-neighbor
Linear
Bilinear
Trilinear
Cubic
Interpolation - Linear
d
px
Interpolated Pixel
P(x+1)
Px’
P(x+1)
px
x
Px’
x’
x+1
px'  px [( px1  p) d]
Building Imaging Applications with Java p272
Bilinear Interpolation
P(x,y)
P(x+1,y)
dy
dx
P(x,y+1)
P(x’,y’)
P(x+1,y+1)
p(x ', y')  [ pu (1 dy )]( p1  dy )
Trilinear Interpolation
P(x,y,z+1)
P(x,y,z)
dy
dx
P(x’,y’,z’)
dz
P(x,y+1,z)
Cubic Interpolation
Pixel value
px
px+1
px’
px’
px+2
px+2
px+3
px+3
px+1
px
x
x+1
x’
x+2
Pixel location
px = ax3 + bx2 + cx + d
x+3
Convolution Kernels
Image I(i,j)
Convolution kernel is a matrix f(m,n)

1

Example: f(m,n) = 
1

1

 j)  I  f 
I(i,
1

1
1

1
2
1
1
1
  I(i  m, j  n) f (m,n)
m 1 n1
Convolution Kernel Example
1
5
6
8
7
8
5
17
6
8
9
10
11
11
12
23
1
5
7
8
9
10
4
3
2
I(3,3) = (1*5) + (1*17) +
(1*6) + (2*11) +
(1*11) + (1*1) +
(1*5) + (1*7)
= 74

1

1
f(m,n)= 

1


1
2
1
1

1
1


Smoothing Filters
Weighted average of the neighboring pixels.
Often approximate a Gaussian function.
1 1 1
1 

f (m,n)  1 1 1
9

1 1 1

QuickTime™ and a
TIFF (LZW)
(LZW) decompressor
TIFF
are needed to see this picture.
are
Edge Detectors
Laplacian
2
2

f

f
2 

x 2 y 2
0

 2  1

0

1
4
1
0

1
0

QuickTime™
QuickTime™ and
and aa
TIFF
TIFF (LZW)
(LZW)decompressor
decompressor
are
areneeded
needed to
to see
seethis
this picture.
picture.

Sharpening
1.0 1.0 1.0


1.0
9.0
1.0



1.0 1.0 1.0

QuickTime™ and a
TIFF
TIFF (LZW)
(LZW) decompressor
are
are needed to see
see this picture.
picture.
Java2D Image Operations
•
•
•
•
•
•
AffineTransformOp
ConvolveOp
LookupOp
RescaleOp
ColorConvertOp
BandCombineOp (RasterOp)
Code Example
Interpolation/Edge Detection
Image I/O
“The Java Image I/O API provides a pluggable architecture for
working with images stored in files and accessed across the
network. It offers substantially more flexibility and power than
the previously available APIs for loading and saving images.”
Image I/O Key Features
• Read & write multiple image formats
– JPG, PNG, GIF, etc.
• Expose image metadata uniformly
– W3C DOM
•
•
•
•
Provide Access to thumbnail images
Support multi-image files
Provide pluggable architecture for additional formats
Handle large image files
Image I/O Packages
• javax.imageio
– ImageIO, ImageReader, ImageWriter
• javax.imageio.event
• javax.imageio.metadata
– IIOMetadata, IIOMetadataFormatImpl,
IIOMetadataNode
• javax.imageio.spi
• Javax.imageio.stream
Custom Plug-ins
• Implement:
– Reader - extends ImageReader
– Writer - extends ImageWriter
– Provider interfaces:
• ImageReaderSpi
• ImageWriterSpi
• Optionally provide
– Image and stream metadata classes extending
IIOMetadata
Image I/O Code Examples
• Reading & writing images
• Querying for supported image types
• Implementing a plug-in
Java Advanced Image
JAI Concepts
•
•
•
•
•
•
Multi-tiled images
Deferred execution
Network images
Image property management
Image operators with multiple sources
Three-dimensional image data
Image Processing DAGs
constant
fileload
add
rotate
Display Widgets
JAI Operator Categories
•
•
•
•
•
•
•
•
•
Point Operators
Area Operators
Geometric Operators
Color Quantization Operators
File Operators
Frequency Operators
Statistical Operators
Edge Extraction Operators
Miscellaneous Operators
Point Operators
Absolute
Add
AddCollection
AddConst
AddConstToCollection
And
AndConst
BandCombine
BandSelect
Clamp
ColorConvert
Composite
Constant
Divide
DivideByConst
DivideComplex
DivideIntoConst
Exp
Invert
Log
Lookup
MatchCDF
Max
Min
Multiply
MultiplyConst
MultiplyComplex
MultiplyConst
Not
Or
OrConst
Overlay
Pattern
Piecewise
Rescale
Subtract
SubtractConst
SubtractConst
SubtractFromConst
Threshold
Xor
XorConst
Area Operators
•
•
•
•
•
Border
BoxFilter
Convolve
Crop
MedianFilter
Geometric Operators
•
•
•
•
•
•
•
Affine
Rotate
Scale
Shear
Translate
Transpose
Warp
Color Quantization
• ErrorDiffusion
• OrderedDither
File Operators
AWTImage
Encode
Filestore
FPX
IIP
JPEG
NM
TIFF
BMP
FileLoad
Format
GIF
IIPResolution
PNG
Stream
URL
Frequency Operators
Conjugate
DCT
DFT
IDCT
IDFT
ImageFunction
Magnitude
MagnitudeSquared
PeriodicShift
Phase
PolarToComplex
API
PlanarImage
RenderOP
Java.awt.image.RenderedImage介面
Java.awt.image.BufferedImage
ParameterBlock
• 用來儲存 RenderableImageOp 或處理影像專
用class 所需要的 sources 與 參數資訊
• 雖然你可以放置任意的objcet 到他的 source
vector, 但是一般都只是放置 RenderedImages
與 RenderableImage object.
• 所有放在 ParameterBlock 中的參數都以
object 方式表示
short s;
add(s);
short s;
add( new Short(s));
意思是一樣的
ParameterBlock
Side Effect
• ParameterBlock 中的 get 與 set methods 都是
使用 reference
例如:
ParameterBlock addSource(ParameterBlock pb, RenderableImage im) {
ParameterBlock pb1 = new ParameterBlock(pb.getSources());
pb1.addSource(im);
pb1 與 pb share 相同的 source Vector
return pb1;
(a change in either is visible to both)
}
ParameterBlock pb1 = new ParameterBlock( pb.getSources().clone() );
ParameterBlock clone method
shadowClone method
• clone method 提供 ParameterBlock 中的
source 與 參數 Vectors 的內容複製 即只有 references 的複製
• 所以你可以藉此更改 Sources 的順序, 當改變內容時,
Objects
會有 Side Effect
Source Vector
Parameter Vector
Source Vector
Parameter Vector
• shadowClone 複製 vectors 本身的 reference
Objects
Source Vector
Parameter Vector
Source Vector
Parameter Vector
ParameterBlock
• add method 傳回的是 this
• 所以我們可以方便增加 objects
連續增加新的 objects
ParameterBlock pb = new ParameterBlock();
op = new RenderableImageOp("operation", pb.add(arg1).add(arg2) );
More api
這個程式可以 decode 所有 JAI 支援的圖形檔案格式
(GIF,JPEG,TIFF,BMP,PNM,PNG) 到 RenderedImage.
並且將影像利用 bilinear interpolation 的方式放大 2倍
import java.awt.Frame;
import java.awt.image.renderable.ParameterBlock;
import java.io.IOException;
import javax.media.jai.Interpolation;
// bilinear operator usage
import javax.media.jai.JAI;
import javax.media.jai.RenderedOp;
// operator
import com.sun.media.jai.codec.FileSeekableStream; // 影像解碼使用
import javax.media.jai.widget.ScrollingImagePanel;
EX1
public class JAISampleProgram {
public static void main(String[] args) {
// Step 1:
FileSeekableStream stream = null;
try {
stream = new FileSeekableStream("as01.jpg");
} catch (IOException e) {
利用 FileSeekableStream 建立影像檔案
e.printStackTrace();
串流
System.exit(0);
}
// Step 2: RenderedOp image1 = JAI.create("stream", stream);
scale operator
// Step 3:
Interpolation interp = Interpolation.getInstance(
Interpolation.INTERP_BILINEAR);
// Step 4:
ParameterBlock params = new ParameterBlock();
params.addSource(image1);
params.add(2.0F);
// x scale factor
params.add(2.0F);
// y scale factor
params.add(0.0F);
// x translate
params.add(0.0F);
// y translate
params.add(interp);
// interpolation method
Interpolation 物件使用
bilinear 演算法作 scale
指定 scale 處理目標物
// Step 5: 建立一個 operator 來 scale 影像
RenderedOp image2 = JAI.create("scale", params);
執行 scale operator
API
// Scale
int width = image2.getWidth();
int height = image2.getHeight();
// attach scrolling panel
利用 ScallingImagePanel 秀結果
ScrollingImagePanel panel = new ScrollingImagePanel( image2, width, height);
/* Create a frame to contain the panel. */
Frame window = new Frame("JAI Sample Program");
window.add(panel);
window.pack();
window.show();
}
}
JAISampleProgram.jpx
Rendered Graphics
• the simplest form of rendering in JAI
• Rendered graphs are useful when it is necessary
to work directly with the pixels
• Immediate
– the image source:
• have been evaluated at the moment it is instantiated and
added to the graph
– a new operation is added to the chain:
• compute its results immediately.
Rendered Graphics
• A Rendered graph
– nodes are usually instances of the RenderedOp
(any subclass of PlanarImage )
• Image sources
– objects that implement the RenderedImage interface
import javax.jai.*;
import javax.jai.widget.*;
import java.awt.Frame;
Example
public class AddExample extends Frame {
ScrollingImagePanel imagePanel1;
public AddExample(ParameterBlock param1, ParameterBlock param2) {
RenderedOp im0 = JAI.create("constant", param1);
Create a constant image
RenderedOp im1 = JAI.create("constant", param2);
Create another constant image
RenderedOp im2 = JAI.create("add", im0, im1);
Add the two images together.
.
// Display the original in a scrolling window
imagePanel1 = new ScrollingImagePanel(im2, 100, 100);
// Add the display widget to our frame.
add(imagePanel1);
}
}
Renderable Graphs
• A Renderable graph
– is made up of nodes implementing the
RenderableImage interface Instance of RenderableOP
Example
• reads a TIFF file, inverts its pixel values, then
adds a constant value to the pixels
RenderedOp sourceImg = JAI.create("TIFF",”Jing01.tiff”);
ParameterBlock pb = new ParameterBlock();
將 Rendered Image 轉換成RenderableImage
pb.addSource(sourceImg);
pb.add(null).add(null).add(null).add(null).add(null);
RenderableImage ren = JAI.createRenderable("renderable", pb);
ParameterBlock pb1 = new ParameterBlock();
建立反相 operator
pb1.addSource(ren);
RenderableOp Op1 = JAI.createRenderable("invert", pb1);
ParameterBlock pb2 = new ParameterBlock();
pb2.addSource(Op1);
// Op1 as the source
建立 addconst operator
pb2.add(2.0f);
// 2.0f as the constant
RenderableOp Op2 = JAI.createRenderable("addconst", pb2);
AffineTransform screenResolution = ...;
RenderContext rc = new RenderContext(screenResolution);
RenderedImage rndImg1 = Op2.createRendering(rc); Get a rendering
// Display the rendering onscreen using screenResolution.
imagePanel1 = new ScrollingImagePanel(rndImg1, 100, 100);
Geometric Transformation
Rotation
// 取出Scale後的影像寬高
int width = image2.getWidth();
int height = image2.getHeight();
// Create the rotation angle (45 degrees) and convert to
// radians.
int value = 45;
float angle = (float) (value * (Math.PI / 180.0F));
// Create a ParameterBlock and specify the source and
// parameters
ParameterBlock pb = new ParameterBlock();
pb.addSource(image2); // The source image
pb.add(0.0F); // The x origin
pb.add(0.0F); // The y origin
指定 rotate 處理目標物
pb.add(angle); // The rotation angle
pb.add(interp); // The interpolation
// Create the rotate operation
RenderedOp RotatedImage = JAI.create("Rotate", pb, null);
Rotation