Transcript Document

Image Filtering Advanced
Image filtering with GDI and DX
HW accelerations
An introduction : domain terms
♦ Bitmap (let’s assume Bitmap == GDI+ Bitmap)
System memory portion containing Pixel as color’s byte
♦ DX Surface
System or Video memory portion representing a bitmap
♦ Texture
Mainly Video memory portion containing a bitmap that
will be mapped onto a polygon
Image Filtering Advanced
Enhanced Image Filter Library
♦ What does a filter do?
♦ What is a filter?
♦ What is a filter chain?
A filter is a function from image to image,
A filter chain is a functional composition so,
it’s a function again.
Image Filtering Advanced
The speed part 1 : unsafe
protected override void filter(Bitmap b, Bitmap[] temp) {
BitmapData bmData = b.LockBits(new Rectangle(0, 0, b.Width, b.Height), ImageLockMode.ReadWrite,
PixelFormat.Format24bppRgb);
int stride = bmData.Stride;
System.IntPtr Scan0 = bmData.Scan0;
unsafe {
byte * p = (byte *)(void *)Scan0;
int nOffset = stride - b.Width*3;
int nWidth = b.Width * 3;
for(int y=0;y<b.Height;++y) {
for(int x=0; x < nWidth; ++x ) {
p[0] = (byte)(255-p[0]);
++p;
}
p += nOffset;
}
This way is faster than
accessing pixels through
the GetPixel GDI function
}
b.UnlockBits(bmData);
}
}
Image Filtering Advanced
Library structure
Smooth
MeanRemoval
EdgeDetectVertical
ColorCorrector
Invert
GrayScale
EdgeDetectQuick
Gamma
Conv3x3
Filter
Contrast
Brightness
EdgeDetectConvolution
EdgeDetectDifference
EdgeEnhance
EmbossLaplacian
EdgeDetectHomogenity
EdgeDetectHorizontal
Image Filtering Advanced
Sharpen
GaussianBlur
The key for success : the filter base
class design
Create all the temporary public void Apply(Bitmap b) {
[Serializable]
public abstract class
Filter { before your filter
int n = this.NumCopies;
Images
private Filter next;
Filter p = next;
chain goes with its work!
protected Filter(Filter f) {
while (p != null) {
next = f;
n = Math.Max(n, p.NumCopies);
}
p = p.next;
protected Filter() {
}
next = null;
Bitmap[] tmp = new Bitmap[n];
}
for (int i = 0; i < n; i++)
protected abstract void filter(Bitmap b,
tmp[i] = new Bitmap(b.Width,
Bitmap[] temp);
b.Height);
p = this;
protected abstract int NumCopies { get; }
do {
p.filter(b, tmp);
public BaseConf GetConfig(){
p = p.next;
return new BaseConf();
} while (p != null);
}
foreach (Bitmap bm in tmp) {
bm.Dispose();
}
}
}
Image Filtering Advanced
The speed part 2 : ngen
♦ Let’s get native!
Run the ngen tool on the filter library
assembly to get it compiled once for all.
Image Filtering Advanced
The speed part 3 : Get HW!
♦ A little trick to achieve the real magic
♦ Let’s pretend :
Image => plane => polygon
♦ Remember you gain speed but you get
also limitations!
Image Filtering Advanced
DX Structure
♦ We can use the HW
3D acceleration to
boost 2D image
filtering using the
Programmable Pixel
Shader
Image Filtering Advanced
HSL with DX 9
♦ Cross Hardware language!
♦ Can contains multiple techniques and
passes!
♦ Exposes a large number of common
function as lerp and so on
Image Filtering Advanced
HSL Stupid Example
float4 Light(float3 LightDir : TEXCOORD1, uniform
float4 LightColor, float2 texcrd : TEXCOORD0,
uniform sampler samp) : COLOR
{
float3 Normal = tex2D(samp,texcrd);
return dot((Normal*2 - 1), LightDir)*LightColor;
}
Image Filtering Advanced
The Idea
1. Create a DX device on a control
2. Create a 3D plane
3. Put the image you want to process on
the plane as a texture
4. Use the Pixel Shader Program to make
the GPU works for you
5. Use the Control Graphics class to save
your processed image.
Image Filtering Advanced
A simple but real FX example
Image Filtering Advanced
And the cons?
Here we are . . .
♦ You can use only power of 2 sized bitmap
♦ The display size is the maximum size for
the output image
. . . But you get a common PC running as it’s
got a real expensive (and dedicated) DSP!
Image Filtering Advanced
The Direct Show way
♦ DShow use a graph from sources to
renderes (audio and video)
♦ Along the graph you can have filters
♦ 2 approaches
– In place transformations
– Not in place transformations
♦ Only C++ (the unmanaged one!)
Image Filtering Advanced
CTransformFilter::Transform
♦ This filter uses the CTransformInputPin
class for its input pin, and the
CTransformOutputPin class for its output
pin.
♦ Use this base class if you want to try filter
and then pass it through the graph
♦ Beware of memory leak!
Image Filtering Advanced
Example of transform
HRESULT CRleFilter::Transform(IMediaSample *pSource, IMediaSample *pDest) {
// Get pointers to the underlying buffers. BYTE *pBufferIn, *pBufferOut;
hr = pSource->GetPointer(&pBufferIn);
if (FAILED(hr)) {
return hr;
}
hr = pDest->GetPointer(&pBufferOut);
if (FAILED(hr)) {
return hr;
}
// Process the data.
DWORD cbDest = EncodeFrame(pBufferIn, pBufferOut); KASSERT((long)cbDest <=
pDest->GetSize());
pDest->SetActualDataLength(cbDest);
pDest->SetSyncPoint(TRUE);
return S_OK;
}
Image Filtering Advanced
CTransInPlaceFilter::Transform
♦ Transform the input sample in place
♦ Use it for real real-time processing
♦ What do you want more?
Image Filtering Advanced
The Frame Dispatcer Class
If you want to apply filter to a real time
camera or to an avi file in C# there’s an
utility from the Medialab (university of
Pisa) that can help you to get your frames
inside a managed application!
Image Filtering Advanced
References
♦ Enhanced Filter Library and Frame
Dispatcer
– http://dotnet.di.unipi.it
Image Filtering Advanced