Data Acquisition, Representation and Reconstruction Jian Huang, CS 594, Spring 2002 This set of slides references slides developed by Profs. Machiraju (Ohio State), Torsten Moeller (Simon Fraser) and Han-Wei Shen (Ohio State).

Acquisition Methods X-Rays Computer Tomography (CT or CAT) MRI (or NMR) PET / SPECT Ultrasound Computational What about microscopic scanners! What about electron microscopes? What about voxelization / discretization? What about synthetic methods? Provide Links to the numerous online tutorials!

X-Rays photons produced by an electron beam similar to visible light, but higher energy!

X-Rays - Physics Associated with inner shell electrons As the electrons decelerate in the target through interaction, they emit electromagnetic radiation in the form of x-rays. patient between an x-ray source and a film -> radiograph cheap and relatively easy to use potentially damaging to biological tissue

X-Rays - Visibility bones contain heavy atoms -> with many electrons, which act as an absorber of x-rays commonly used to image gross bone structure and lungs excellent for detecting foreign metal objects main disadvantage -> lack of anatomical structure all other tissue has very similar absorption coefficient for x-rays

X-Rays - Images Add images!!!

CT or CAT - Principles Computerized (Axial) Tomography introduced in 1972 by Hounsfield and Cormack natural progression from X-rays based on the principle that a three-dimensional object can be reconstructed from its two dimensional projections based on the Radon transform (a map from an n- dimensional space to an (n-1)-dimensional space)

CT or CAT - Methods measures the attenuation of X-rays from many different angles a computer reconstructs the organ under study in a series of cross sections or planes combine X-ray pictures from various angles to reconstruct 3D structures

CT - Reconstruction: FBP Filtered Back Projection common method uses Radon transform and Fourier Slice Theorem y F(u,v) f(x,y) Gf(r) x s u gf(s) f Spatial Domain Frequency Domain

CT - Reconstruction: ART Algebraic Reconstruction Technique iterative technique attributed to Gordon Initial Guess Reconstructed model Back- Projection Projection Actual Data Slices

CT - FBP vs. ART FBP ART Still slow Computationally cheap better quality for fewer projections better quality for non-uniform project. “guided” reconstruct. (initial guess!) Computationally cheap Clinically usually 500 projections per slice problematic for noisy projections

CT - 2D vs. 3D Linear advancement (slice by slice) helical movement typical method tumor might fall between ‘cracks’ takes long time helical movement 5-8 times faster A whole set of trade-offs

CT or CAT - Advantages significantly more data is collected superior to single X-ray scans far easier to separate soft tissues other than bone from one another (e.g. liver, kidney) data exist in digital form -> can be analyzed quantitatively adds enormously to the diagnostic information used in many large hospitals and medical centers throughout the world

CT or CAT - Disadvantages significantly more data is collected soft tissue X-ray absorption still relatively similar still a health risk MRI is used for a detailed imaging of anatomy

MRI Nuclear Magnetic Resonance (NMR) (or Magnetic Resonance Imaging - MRI) most detailed anatomical information high-energy radiation is not used, i.e. “save” based on the principle of nuclear resonance (medicine) uses resonance properties of protons

MRI - polarized all atoms (core) with an odd number of protons have a ‘spin’, which leads to a magnetic behavior Hydrogen (H) - very common in human body + very well magnetizing Stimulate to form a macroscopically measurable magnetic field

MRI - Signal to Noise Ratio proton density pictures - measures H MRI is good for tissues, but not for bone signal recorded in Frequency domain!! Noise - the more protons per volume unit, the more accurate the measurements - better SNR through decreased resolution

PET/SPECT Positron Emission Tomography Single Photon Emission Computerized Tomography recent technique involves the emission of particles of antimatter by compounds injected into the body being scanned follow the movements of the injected compound and its metabolism reconstruction techniques similar to CT - Filter Back Projection & iterative schemes

Ultrasound the use of high-frequency sound (ultrasonic) waves to produce images of structures within the human body above the range of sound audible to humans (typically above 1MHz) piezoelectric crystal creates sound waves aimed at a specific area of the body change in tissue density reflects waves echoes are recorded

Ultrasound (2) Delay of reflected signal and amplitude determines the position of the tissue still images or a moving picture of the inside of the body there are no known examples of tissue damage from conventional ultrasound imaging commonly used to examine fetuses in utero in order to ascertain size, position, or abnormalities also for heart, liver, kidneys, gallbladder, breast, eye, and major blood vessels

Ultrasound (3) by far least expensive very safe very noisy 1D, 2D, 3D scanners irregular sampling - reconstruction problems

Computational Methods (CM) Computational Field Simulations Computational Fluid Dynamics - Flow simulations Computational Chemistry - Electron-electron interactions, Molecular surfaces Computational Mechanics - Fracture Computational Manufacturing - Die-casting

CM - Approach (Continuous) physical model Partial/Ordinary Differential Equation (ODE/PDE) e.g. Navier-Stokes equation for fluid flow e.g. Hosted Equations: e.g. Schrödinger Equation - for waves/quantum Continuous solution doesn’t exist (for most part) Numerical Approximation/Solution 1. Discretize solution space - Grid generation explicit 2. Replace continuous operators with discrete ones 3. Solve for physical quantities

CM - Methods Grid Generation Numerical Methods Time Varying non-elliptical methods: algebraic, conformal, hyperbolic, parabolic, biharmonic elliptical methods (based on elliptical PDE’s) Numerical Methods Newton Runge-Kutta Finite Element Finite Differences Time Varying

CM - Solutions (Structured) PDE usually constrained/given at boundary map from computational to physical space polar maps, elliptical, non-elliptical structured grids physical parametric computational f x t

CM - Solutions (Unstructured) usually scattered data set Delaunay Triangulation Element Size Optimization start with initial tetrahedral grid interactively insert grid points insertion guided by curvature and distance to surface Advancing Front Method start with boundary advance boundary towards inside until “filled”

CM - Grid Types (2) Multiblock structured grids hybrid grids multiple structured grids connected not necessarily structured hybrid grids structured + unstructured chimera grids partially overlapping hierarchical grids generated by quad-tree and octree like subdivision schemes (AKA embedded or semi-structured grids) Include figures from the Nielson book

CM - Grid Examples

CM - Structured vs. Unstructured consider discretization points as samples, points, cells, or voxels Structured Addressing - Cell [i,j,k] provides location of neighbors Boundaries of volume - Easy to determine Unstructured No addressing mechanism - adjacency list required Cannot determine the boundaries easily Cells never of same size Cells are hexahedrons, tetrahedrons, curved patches

Synthetic Methods 3D Discretization Techniques  Voxelization Scan Conversion of Geometric Objects Planes / Triangles Cylinders Sphere Cone NURBS, Bezier patches

Synthetic Methods Solid Textures Hyper Texture - 3D Textures Fur Marble Hair Turbulent flow 3D Regular grid has texture values

Grid Types Structured Grids: rectilinear curvilinear uniform regular Unstructured Grids: regular irregular hybrid curved

Data Representation Regular raster Point list RLE Contour Stacks

Data Representation (2) Polygon mesh, Curvilinear, Unstructured Vertex list Adjacency list of cells (not for curvilinear) No convenient structure Compressed Grids - RLE, JPEG, Wavelets Multi-resolution Grids animate

Data Characteristics Large Data Sets Modest Head 2563 = 16Mbytes !! Visible Human - MRI (2562) + CT (5122) + photo Male - 15GB (1mm slice dist.); Female - 40GB (0.33mm) Noisy - Ultrasound (How about CFD, CAGD ?) Band-limited - Textured Images (Mandrill) wc Frequency FFT Noisy Data set

Data Characteristics Limited Dynamic Range - values between min, max populated with unequal probability Non-Uniform Spatial Occupancy - Quadtree/Octree Rectangular spatial subdivision Find a histogram of an image to include here!

Data Objects Data Object: dataset (representation of information) Structures: how the information is organized - Topology - Geometry Attributes: store the information we want to visualize. e.g. function values

Data Objects: structures Topology: Invariant under geometric transformation (rotation, translation, scaling etc) - Topological structures: Cells Geometry: The instantiation of the topology Geometric structures: cells with positions in 3D space

Cell Types for Unstructured Grid (a) vertex (b) Polyvertex (c) line (d) polyline (e) triangle (e) Quadrilateral (e) Polygon (f) Tetrahedron (f) Hexahedron And more (I am too tired drawing now …)

Data Attributes The information stored at each vertex of the cell Scalars: temperature, pressure, etc Vector: velocity Normal: surface directions Texture coordinates: graphics specific Tensors: matrices

Where are we now ... Remember: Data Object: dataset (representation of information) Structures: how the information is organized - Topology - Geometry Attributes: store the information we want to visualize. e.g. function values

Interpolation (1) Visualization deals with discrete data (a) vertex (b) Polyvertex (c) line (d) polyline (e) triangle (e) Quadrilateral (e) Polygon (f) Tetrahedron (f) Hexahedron Values defined only at cell vertices

Color Mapping R G B v1 v2 v3 ... Values at vertices Color lookup Result table

Interpolation (2) We often need information at positions other than cell vertices p P = ? 10 13 9 12 Interpolation: compute data from known points

Interpolation (3) Three essential information: Cell type Data values at cell vertices Parametric coordinates of the point p D = S Wi * di i=0 n-1 di di: cell point value Wi: weight (S wi = 1) D: interpolated result

Interpolation (4) Parametric Coordinates: Used to specify the location of a point within a cell (a) line r = 0 r = 1 0 <= r <=1 d0 d1 W0 = (1-r) W1 = r D = S Wi * di i=0 n-1 r

Interpolation (5) (b) Triangle s W0 = 1-r-s W1 = r W2 = s p2 r+s = 1 (why?) r=0 Why? r p0 p1 s=0

Interpolation (6) (C) Pixel s s=1 p2 p3 W0 = (1-r)(1-s) W1 = r(1-s) (s,t) r=0 r =1 r p0 s=0 p1 Why? This is also called bi-linear interpolation

Interpolation (7) (D) Polygon p3 p4 Wi = (1/ri) / S(1/ri) r3 p2 p5 Weighted distance function p0 p1

Interpolation (8) (D) Tetrahedron t s p3 W0 = 1-r-s-t p2 W1 = r W2 = s W3 = t p3 p2 p0 r p1

Interpolation (9) (D) Cube (voxel) t W0 = (1-r)(1-s)(1-r) W1 = r(1-s)(1-t) W2 = (1-r)s(1-t) W3 = rs(1-r) W4 = (1-r)(1-s)t W5 = r(1-s)t W6 = (1-r)st W7 = rst p6 p7 p4 p5 s p2 p3 r p0 p1

Interpolation (10) The interpolation function can be used to calculate the geometric position as well. That is, given (r,s,t), calculate the global coordinates Local to global coordinate transformation: P = S Wi * Pi n-1 i=0

Interpolation (11) How to get (r,s,t) ? Line, Pixel, Cube are all trivial Triangle, Tetrahedron can be solved analytically Qudrilateral or Hexahedra need numerical method P = S Wi * Pi n-1 Known: P, Pi Unkown: Wi (i.e. r,s,t) i=0

Contours 3 7 10 4 C = 6

Interpolation 2D 1D Given: Given: Needed: Needed:

The Need for Interpolation Interpolation is needed throughout the visualization process Rendering? Extracting iso-contours? Shading? Texture mapping? … Anytime when resampling is needed

Reconstructed Function General Process Original function Sampled function Acquisition Reconstruction Reconstructed Function Re-sampled function Resampling

Evaluated at discrete points (sum) How? - Convolution Spatial Domain: Frequency Domain: Mathematically:f(x)*h(x) = Multiplication: Convolution: Evaluated at discrete points (sum)

Reconstruction Mathematically: f(x)*h(x) = (Sf[i])*h(x) Sf[i] h(x)

General Process - Frequency Domain Original function Sampled function Acquisition Reconstruction Reconstructed Function Re-sampled function Resampling

Pre-Filtering Original function Band-limited function Pre-Filtering Acquisition Sampled Function Reconstructed function Reconstruction

Ideal Reconstruction with Sinc function Spatial Domain: convolution is exact Frequency Domain: cut off freq. replica

Reconstructing Derivatives Spatial Domain: convolution is exact Frequency Domain: cut off freq. replica

Possible Errors Post-aliasing Smoothing Ringing (overshoot) Anisotropy reconstruction filter passes frequencies beyond the Nyquist frequency (of duplicated frequency spectrum) => frequency components of the original signal appear in the reconstructed signal at different frequencies Smoothing frequencies below the Nyquist frequency are attenuated Ringing (overshoot) occurs when trying to sample/reconstruct discontinuity Anisotropy caused by not spherically symmetric filters

How Good? = Error Spatial Domain: local error asymptotic error numerical error Frequency Domain: global error visual appearance blurring aliasing smoothing Approximation Theory/Analysis Signal Processing

Sources of Aliasing Non-bandlimited signal Low sampling rate (below Nyquist) Non perfect reconstruction

Reconstruction Kernels pass band stop band Ideal filter The spatial extent of reconstruction kernels, or interpolation basis functions, depend on the cut-off frequency as well. Smoothing error Postaliasing error filter

Reconstruction Kernels Nearest Neighbor (Box) Triangular func Sinc Gaussian + many others Spatial d. Frequency d.

Higher Dimensions An-isotropic Filters separable filters (radially symmetric) separable filters

Interpolation (an example) Very important; regardless of algorithm expensive => done very often for one image Requirements for good reconstruction performance stability of the numerical algorithm accuracy Linear Nearest neighbor

Put Things in Perspective In visualization, need to use continuous space functions. But can only work with discrete data So, let’s reconstruct from discrete data to continuous space (convolution) and resample Interpolation is doing the same thing. Computing one data point in the resulting function, say, at x1. So, which reconstruction kernel (basis function) does linear/bilinear/tri-linear interpolations use?