1. Image Filtering Advanced
Image filtering with GDI and DX HW accelerations

2. An introduction : domain terms
Bitmap (let’s assume Bitmap == GDI+ Bitmap)
System memory portion containing Pixel as color’s byte
DX Surface
System or Video memory portion representing a bitmap
Texture
Mainly Video memory portion containing a bitmap that will be mapped onto a polygon
Image Filtering Advanced

3. Enhanced Image Filter Library
What does a filter do?
What is a filter?
What is a filter chain?
A filter is a function from image to image,
A filter chain is a functional composition so, it’s a function again.
Image Filtering Advanced

4. Image Filtering Advanced
The speed part 1 : unsafe
protected override void filter(Bitmap b, Bitmap[] temp) {
BitmapData bmData = b.LockBits(new Rectangle(0, 0, b.Width, b.Height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
int stride = bmData.Stride;
System.IntPtr Scan0 = bmData.Scan0;
unsafe {
byte * p = (byte *)(void *)Scan0;
int nOffset = stride - b.Width*3;
int nWidth = b.Width * 3;
for(int y=0;y<b.Height;++y) {
for(int x=0; x < nWidth; ++x ) {
p[0] = (byte)(255-p[0]);
++p; }
p += nOffset;
b.UnlockBits(bmData);
This way is faster than accessing pixels through the GetPixel GDI function
Image Filtering Advanced

5. Image Filtering Advanced
Library structure
Image Filtering Advanced

6. The key for success : the filter base class design
Create all the temporary
Images before your filter
chain goes with its work!
[Serializable]
public abstract class Filter {
private Filter next;
protected Filter(Filter f) {
next = f; }
protected Filter() {
next = null;
protected abstract void filter(Bitmap b, Bitmap[] temp);
protected abstract int NumCopies { get; }
public BaseConf GetConfig(){
return new BaseConf();
public void Apply(Bitmap b) {
int n = this.NumCopies;
Filter p = next;
while (p != null) {
n = Math.Max(n, p.NumCopies);
p = p.next; }
Bitmap[] tmp = new Bitmap[n];
for (int i = 0; i < n; i++)
tmp[i] = new Bitmap(b.Width, b.Height);
p = this;
do {
p.filter(b, tmp);
} while (p != null);
foreach (Bitmap bm in tmp) {
bm.Dispose();
Image Filtering Advanced

7. Image Filtering Advanced
The speed part 2 : ngen
Let’s get native!
Run the ngen tool on the filter library assembly to get it compiled once for all.
Image Filtering Advanced

8. Image => plane => polygon
The speed part 3 : Get HW!
A little trick to achieve the real magic
Let’s pretend :
Remember you gain speed but you get also limitations!
Image => plane => polygon
Image Filtering Advanced

9. Image Filtering Advanced
DX Structure
We can use the HW 3D acceleration to boost 2D image filtering using the Programmable Pixel Shader
Image Filtering Advanced

10. Image Filtering Advanced
HSL with DX 9
Cross Hardware language!
Can contains multiple techniques and passes!
Exposes a large number of common function as lerp and so on
Image Filtering Advanced

11. Image Filtering Advanced
HSL Stupid Example
float4 Light(float3 LightDir : TEXCOORD1, uniform float4 LightColor, float2 texcrd : TEXCOORD0, uniform sampler samp) : COLOR {
float3 Normal = tex2D(samp,texcrd);
return dot((Normal*2 - 1), LightDir)*LightColor; }
Image Filtering Advanced

12. Image Filtering Advanced
The Idea
Create a DX device on a control
Create a 3D plane
Put the image you want to process on the plane as a texture
Use the Pixel Shader Program to make the GPU works for you
Use the Control Graphics class to save your processed image.
Image Filtering Advanced

13. A simple but real FX example
Image Filtering Advanced

14. Image Filtering Advanced
And the cons?
Here we are . . .
You can use only power of 2 sized bitmap
The display size is the maximum size for the output image
. . . But you get a common PC running as it’s got a real expensive (and dedicated) DSP!
Image Filtering Advanced

15. Image Filtering Advanced
The Direct Show way
DShow use a graph from sources to renderes (audio and video)
Along the graph you can have filters
2 approaches
In place transformations
Not in place transformations
Only C++ (the unmanaged one!)
Image Filtering Advanced

16. CTransformFilter::Transform
This filter uses the CTransformInputPin class for its input pin, and the CTransformOutputPin class for its output pin.
Use this base class if you want to try filter and then pass it through the graph
Beware of memory leak!
Image Filtering Advanced

17. Image Filtering Advanced
Example of transform
HRESULT CRleFilter::Transform(IMediaSample *pSource, IMediaSample *pDest) {
// Get pointers to the underlying buffers. BYTE *pBufferIn, *pBufferOut;
hr = pSource->GetPointer(&pBufferIn);
if (FAILED(hr)) {
return hr; }
hr = pDest->GetPointer(&pBufferOut);
// Process the data.
DWORD cbDest = EncodeFrame(pBufferIn, pBufferOut); KASSERT((long)cbDest <= pDest->GetSize());
pDest->SetActualDataLength(cbDest);
pDest->SetSyncPoint(TRUE);
return S_OK;
Image Filtering Advanced

18. CTransInPlaceFilter::Transform
Transform the input sample in place
Use it for real real-time processing
What do you want more?
Image Filtering Advanced

19. The Frame Dispatcer Class
If you want to apply filter to a real time camera or to an avi file in C# there’s an utility from the Medialab (university of Pisa) that can help you to get your frames inside a managed application!
Image Filtering Advanced

20. Image Filtering Advanced
References
Enhanced Filter Library and Frame Dispatcer
Image Filtering Advanced