1 ITK Lecture 10 Review & Toolbox Part II Methods in Image Analysis CMU Robotics Institute U. Pitt Bioengineering 2630 Spring Term, 2006 Damion Shelton

2 Some thoughts I’ve collected a bunch of stuff that is either: –Grandiose preaching from the pulpit –Useful filters/tips/tricks I’ve run across A lot of what I’ll be talking about I’ve mentioned in passing before; hopefully you have a different perspective after having written a filter

3 Generic programming revisited Generic programming may be loosely defined as “programming with concepts” (David Musser) What concepts have we looked at so far?

4 Concept of an image An image is rectilinear container in N-space which holds regular samples of some physical space Each of these regular samples is called a pixel

5 Concept of a pixel A pixel is a sample of data in N-space, and may be represented by a variety of data types depending on the modality used to acquire the data

6 Concept of an iterator An iterator is a way to move over an image; it provides method that mimics “sequential” access regardless of the actual access method or dimensionality

7 Concept of a pipeline There are two main types of objects in the world, data objects and process objects Typically, we feed a data object to a process object and get a new data object as a result A sequentially chain of process objects is called a pipeline

8 Writing generic code Successful generic programming means that you “ignore” concerns that would be specific to a particular image –pixel type (i.e. the actual data type) –dimensionality The first way you do this is with templating

9 Writing generic code, cont. But... templating alone doesn’t ensure that your code is generic Avoid loops that maneuver through dimensionality, instead, loop over an iterator Use data types (VNL vectors, etc.) to make math easier in N-d

10 Questions to ask yourself Am I making tradeoffs between: –Speed of coding and reusability? –Level of generality and execution speed? –Compactness and clarity? ITK seems to lean towards reusable, generic, and clear code; depending on your needs this may be a criticism or a point in favor of the toolkit

11 When generic programming fails As much as we’d like, not all algorithms extend to N-d or to all pixel types But... don’t assume that things won’t work in higher (or lower) dimensions I was surprised to discover that code I wrote to do medial axis analysis in 3D worked correctly on a 4D hypersphere

12 The “other” way of reading images You may recall that I mentioned there was another way of reading images, besides that presented in class This method relies on ITK’s “factory” architecture

13 Image reading: first technique We know what kind of image we have (from a dialog box, for instance) Create a reader that can read that type of image Read the image

14 Image reading: second technique We know what kinds of images we might have, but don’t assume that the user knows Register instances of all of reader types we might need The readers will let us know if they can read the file and will do so if possible

15 Reading images with the factory m_ImageReader = ImageFileReaderType::New(); itk::MetaImageIOFactory::RegisterOneFactory(); m_ImageReader->SetFileName(m_Filename ); m_ImageReader->Update(); Here I only register one factory, but there’s no reason I couldn’t have more

16 Factory advantages The factory model of image reading is a bit more flexible and requires less advance knowledge If you have a dialog box, you don’t have to parse the filename to decide what type of reader to create Less risk of being “wrong” about file types

17 Factory disadvantages I find it a bit harder to think of conceptually, and you often know what image formats you’re dealing with anyways

18 Gallery of useful ITK classes These are classes I have found that solve particularly common problems that arise in image processing Don’t re-invent the wheel! This list is not comprehensive (obviously) I leave specific documentation of these filters as an EFTR

19 Padding an image Problem: you need to add extra pixels outside of an image (e.g., prior to running a filter) Solution: PadImageFilter & its derived classes

20 Cropping an image Problem: trimming image data from the outside edges of an image (the inverse of padding) Solution: CropImageFilter

21 Rescaling image intensity Problem: you need to translate between two different pixel types, or need to shrink or expand the dynamic range of a particular pixel type Solution: RescaleIntensityImageFilter

22 Computing image derivatives Problem: you need to compute the derivative at each pixel in an image Solution: DerivativeImageFilter, which is a wrapper for the neighborhood tools discussed last week See also LaplacianImageFilter

23 Compute the mirror image Problem: you want to mirror an image about a particular axis or axes Solution: FlipImageFilter - you specify flipping on a per-axis basis

24 Rearrange the axes in an image Problem: the coordinate system of your image isn’t what you want; the x axis should be z, and so on Solution: PermuteAxesImageFilter - you specify which input axis maps to which output axis

25 Resampling an image Problem: you want to apply an arbitrary coordinate transformation to an image, with the output being a new image Solution: ResampleImageFilter - you control the transform and interpolation technique

26 Getting a lower dimension image Problem: you have read time-series volume data as a single 4D image, and want a 3D “slice” of this data (one frame in time), or want a 2D slice of a 3D image, etc. Solution: ExtractImageFilter - you specify the region to extract and the “index” within the parent image of the extraction region

27 An(other) introduction to VTK For the remainder of this class I’ll present a brief summary of how VTK works It’s useful to know what’s going on behind the scenes in myITKgui It’s very likely that some of you will want to use VTK in stand-alone mode without the FLTK wrapper

28 Rendering layout of VTK vtkRenderWindow defines the window that is displayed on your monitor vtkRenderers are attached to windows and are responsible for converting more abstract primitives (vtkActors) into displayed images

29 Actors in VTK The basic “thing” that can be displayed in VTK is an Actor Mappers convert raw data to Actors For example: –boundary points in ITK  VTK pointset  VTK point mask filter  VTK polygon data mapper  VTK actor

30 vtkRenderWindowInteractors Interactors are objects that work with RenderWindows to pass mouse and keyboard events back and forth One particularly useful interactor is vtkFlRenderWindowInteractor, which lets you use VTK windows with the FLTK window manager

31 Widgets Widgets are more complicated objects that combine some of the functionality of Interactors and Actors The ImagePlaneWidget is a mouse- controlled object that provides an arbitrary slice through a 3D volume

32 Program layout 1.Create a RenderWindow 2.Create a Renderer 3.Create a FlRenderWindowInteractor 4.Load an image 5.Create 3 image plane widgets, attach them to the interactor 6.Enter a message loop to run the program

33 Adding additional “stuff” It’s easier than you might think to render additional objects along with the image plane widgets (boundary points for instance) Starting with some sort of object in ITK, you would do the following...

34 Arbitrary object visualization 1.Figure out what type of primitive you have (points/lines/etc.) 2.Create VTK data representing your primitives 3.Convert this to poly data 4.Map this to an actor

35 Data representation in VTK For geometric data, you may be interested in the following classes: –vtkPoints stores a list of 3D points –vtkUnstructuredGrid stores a collection of arbitrary cells, where a cell is a primitive such as a vertex or line (vtkLine)

36 Data representation cont. This may seem a bit convoluted, but in practice it’s pretty simple once you get the hang of it VTK has a pipeline, similar to that of ITK, so changing the data/mapper/etc. will affect downstream filters but not upstream ones

37 Cool side effects My “favorite” 1 added bonus of using VTK is the ability to export scenes to files Since data and rendering are abstracted away from each other, it’s pretty easy to, for example, dump your entire rendering setup to a Pixar Renderman format file 1 Favorite du jour