College of Engineering Three-dimensional shape characterization for particle aggregates using multiple projective representations Jonathan Corriveau Thesis Advisor: Dr. Shreekanth Mandayam Committee: Dr. Beena Sukumaran and Dr. Robi Polikar Rowan University College of Engineering 201 Mullica Hill Road Glassboro, NJ 08028 (856) 256-5330 http://engineering.rowan.edu/ Monday, April 17, 2017

Outline Introduction Objectives of Thesis Previous Work Approach Results Conclusions

Characterizing Shapes Shapes are described by names Circle, Triangle, Rectangle, etc. Not possible for complicated shapes Shapes need to be described by numbers Most shapes can be described by a set of numbers Computers need numbers Similar shapes must have similar values Few as possible is desirable

Shapes Rectangle Circle Triangle Arbitrary Shape

Application Computer Vision Face Recognition Fingerprint matching Image 1 Image 2 Image 1 Image 2 Images Match Images Do Not Match

Application Character Recognition Descriptor Database Character Descriptors a b Phi 1: 1.0292 Phi 2: 2.5359 Phi 3: 8.917 Phi 4: 14.1381 Phi 5: 29.2098 Phi 6: 15.4456 Phi 7: 29.1866 Phi 1: 1.058 1.2377 Phi 2: 2.664 3.403 Phi 3: 9.4284 7.8057 Phi 4: 14.2453 13.702 Phi 5: 29.8432 27.6783 Phi 6: 16.0222 16.1285 Phi 7: 29.4245 28.2324

Motivation Soil Behavior Strong relationship between stress-strain behavior of soils and the inherent characteristics of its individual particles Inherent Particle Characteristics Hardness, Specific Gravity Distribution Shape and Angularity Particle Size and Size Distribution SEM Picture of Dry Sand

Aggregate Mixtures #1 Dry Sand Michigan Dune Sand Daytona Beach Sand Glass Beads

Motivation Currently 2-D methods are not enough to characterize a soil mixture for discrete element model Only behavior trends can be captured using 2-D models 3-D information allows a much more accurate model

3-D Shapes 3-D shapes are difficult to characterize as a set of numbers Require sophisticated equipment Large databases of numbers to record the position of each coordinate Aggregates of 3-D objects A collection of 3-D particles must be characterized by a set of numbers

2-D Shapes Computationally inexpensive Many methods already exist for characterizing 2-D shapes Can easily be implemented on a computer with only digital images Question: How can 2-D methods help with finding a 3-D solution?

Objectives of Thesis Design automated algorithms that can estimate 3-D shape descriptors for particle aggregates using a statistical combination of 2-D shape descriptors from multiple 2-D projections. Demonstrate consistency, separability and uniqueness of the 3-D shape-descriptor algorithm by exercising the method on a set of sand particle mixes. Preliminary efforts towards the demonstration of the algorithm’s ability to accurately and repeatably construct composite 3-D shapes from multiple 2-D shape-descriptors.

Desirable Descriptor Qualities Fundamental Qualities Uniqueness Parsimony Independent Invariance Rotation Scale Translation Original Rotation Scale Translation

Additional Qualities Reconstruction Interpretation Allow for a shape to be constructed from the descriptors Interpretation Relate to some physical property Automatic Collection Collection and evaluation automation Removes human error

Previous Work Proponents Method Explanation Sebestyn and Benson “unrolling” a closed outline The concept of creating a 1-D function from a 2-D boundary. Introduced by Benson into the field of geology. Hu 2-D Invariant Moments 2-D moments that invariant to translation, rotation, scale and reflection. Ehrlich and Weinberg Radius Expansion Introduced Fourier analysis for radius expansion into sedimentology. Medalia Equivalent Ellipses Fits an ellipse to have similar properties to the actual shape. Does not need outline. Davis and Dexter Chord to Perimeter Measures chord lengths between various points along an outline.

Previous Work Proponents Method Explanation Zahn and Roskies Angular Bend Introduced by Sebestyn, but made widely known by Zahn and Roskies. Discretize an outline into a series of straight lines and angles Granlund Fourier Descriptors Uses x+jy from the coordinates of an outline to be analyzed by Fourier analysis. Sadjadi and Hall 3-D Invariant Moments 3-D moments that are invariant to translation, rotation, and scale. Garboczi, Martys, Saleh, and Livingston Spherical Harmonics A process similar to 3-D Fourier analysis, and requires 3-D information. Sukumaran and Ashmawy Shape and Angularity Factor Compares shapes to circles and measures their deviation. Uses a mean and standard deviation of many particles to compare a mixes.

Radius Expansion R3 R2 R1 R4

Radius Expansion y R2() R1()  x

Angular Bend L2 L1 1 L3 2

Complex Coordinates y (x1, y1) x

Chord to Perimeter The covered perimeter length divided by total perimeter determines the amount of irregularity Small ratio measures small irregularities Approaching one measures large irregularities Chord Length Perimeter Length

Equivalent Ellipses Two factors are calculated from ellipses Anisometry – ratio of long to short axis of ellipse Bulkiness – ratio of areas of figure and ellipse

Approach: Premise 2-D images of 3-D particles in an aggregate mix can be used to denote 2-D projections of a composite 3-D particle that represent the entire mixture

Overview of Approach 2-D facets of 3-D particles in mix Composite 3-D “Reconstruction” 2-D facets of Composite Particle 2-D Descriptors from Mix 3-D Descriptors 2-D Descriptors from Particle

Particles Orientation Regularity Every particle observed offers a different angle of a composite particle Many different facets should be represented by the images Regularity Similar particles should have similar shapes

Aggregate Mixtures #1 Dry Sand Michigan Dune Sand Daytona Beach Sand Glass Beads

Statistics Similar shapes should have similar descriptors Find a distribution for each descriptor from all particle images Calculate both the mean and variance that characterize the distribution Allows a set of 2-D projections to represent a composite 3-D object using a small set of numbers [S1, S2, S3, S4,…… SN] [S1, S2, S3, S4,…….SN] s3 f(s3) m3

From 2-D to 3-D 3-D aggregate mixes can be characterized by a set of numbers Multiple 2-D images can be used to construct a single composite 3-D object Very little equipment required Microscope and Camera (data collection) Computer (analysis)

Shape Characterization Methods Complex Coordinate Fourier Analysis Allows random generation of projections from 3-D descriptors Invariant Moments Requires less computation, less preprocessing, and is more parsimonious, but does not allow projection generation

Fourier Analysis Object must be described as a function Function should be periodic Fourier Transform can be applied to analyze the frequencies Low Frequencies hold general shape information, while high frequencies carry more detail Effective for compression since reconstruction is possible with fewer values than the original

Fourier Descriptors

Fourier Descriptors Descriptors Near Zero Values

Moments Statistical moments Normalized combinations of mean, variance, and higher order moments Moments of similar objects should share similar moment calculations 2-D moments evaluate the images without having to extract the boundary Parsimonious (only 7 moments)

2-D Central Moments Equation of 2-D moment is given as:

Moments For a digital image the discrete equation becomes: Normalized Central Moments are defined as: where,

Invariant Moments

Overview of Approach 2-D facets of 3-D particles in mix Composite 3-D “Reconstruction” 2-D facets of Composite Particle 2-D Descriptors from Mix 3-D Descriptors 2-D Descriptors from Particle

Creation of Composite Particle

“Reconstruction” of 3-D Composite Particle Three techniques were tested for constructing a 3-D composite particle using 2-D projections Extrusion Rotation into 3-D Tomographic

Extrusion Method

Rotation into 3-D Method

Tomographic Method

Implementation and Results Experimental Setup Normalization and Results of Complex Coordinate Fourier Analysis Invariant Moment Results Preliminary “reconstruction” results of the different methods introduced

Optical Microscope, Digital Camera, and Computer Experimental Setup Equipment Data Samples Optical Microscope, Digital Camera, and Computer #1 Dry Sand Daytona Beach Sand Glass Bead

Preprocessing of Images Original Image Black and White Inverted Final Image Cleaned

Obtaining Fourier Descriptors Edge detection of the image Plot of coordinates extracted from image FFT of 1-D Signal Plotted as a 1-D Function

Reconstruction of 2-D Projections Reconstruction using all descriptors Reconstruction using 20 descriptors

Frequency Normalization Process Original Image Half-Sized Image

Original Functions and FFTs Original Image Half-Sized Image

After Normalization Original Image Half-Sized Image

Overview of Approach 2-D facets of 3-D particles in mix Composite 3-D “Reconstruction” 2-D facets of Composite Particle 2-D Descriptors from Mix 3-D Descriptors 2-D Descriptors from Particle

Statistics of Fourier Descriptors #1 Dry Sand Standard Melt Sand Daytona Beach Sand Michigan Dune Sand

Ellipsoid Model for 3-D Shape Characterization Center of Ellipsoid – 3-D Coordinate of Descriptor Means Radius in X – Variance of First Descriptor Radius in Y - Variance of Second Descriptor x z y Radius in Z – Variance of Third Descriptor

Separability of Soil Mixes using Fourier Descriptors #1 Dry Melt Daytona Beach Glass Bead Michigan Dune

Classification Effectiveness using Fourier Descriptors #1 Dry Melt Daytona Beach Glass Bead Michigan Dune

Invariant Moments of Similar Images Original Rotated and Resized

Invariant Moments of Similar Images Difference 1 7.1164 7.1176 0.02% 2 15.2953 15.3027 0.05% 3 12.4116 12.1704 1.94% 4 25.1942 25.2073 5 50.3498 49.0710 2.54% 6 32.9973 33.0157 0.06% 7 50.7640 50.7842 0.04%

Invariant Moments of Dissimilar Images

Invariant Moments of Dissimilar Images Difference 1 7.1164 7.2694 2.15% 2 15.2953 16.7749 9.67% 3 12.4116 17.4857 40.88% 4 25.1942 26.9251 6.87% 5 50.3498 52.5509 4.37% 6 32.9973 35.5863 7.85% 7 50.7640 52.5528 3.52%

Overview of Approach 2-D facets of 3-D particles in mix Composite 3-D “Reconstruction” 2-D facets of Composite Particle 2-D Descriptors from Mix 3-D Descriptors 2-D Descriptors from Particle

Statistics of Invariant Moment Descriptors #1 Dry Sand Standard Melt Sand Daytona Beach Sand Michigan Dune Sand

Separability of Soil Mixes using Invariant Moment Descriptors #1 Dry Melt Daytona Beach Michigan Dune

Classification Effectiveness using Invariant Moment Descriptors #1 Dry Melt Daytona Beach Michigan Dune

Overview of Approach 2-D facets of 3-D particles in mix Composite 3-D “Reconstruction” 2-D facets of Composite Particle 2-D Descriptors from Mix 3-D Descriptors 2-D Descriptors from Particle

Reconstruction of Projections from 3-D Descriptors Original Image Reconstructed Image

Generation of Random Projections from 3-D Descriptors

Separability of Soil Mixes using Randomly Generated Projections

Comparison between Original and Generated Projections

Overview of Approach 2-D facets of 3-D particles in mix Composite 3-D “Reconstruction” 2-D facets of Composite Particle 2-D Descriptors from Mix 3-D Descriptors 2-D Descriptors from Particle

Extrusion Method All Projections in 3-D Space 1st Projection of Dry Sand 2nd Projection of Dry Sand 3rd Projection of Dry Sand

Implementation of Extrusion Method on Dry Sand Projections after Extrusion Final “Reconstruction”

Effectiveness of Extrusion “Reconstructed” Composite Particle #1 Dry Melt Daytona Beach Michigan Dune

Rotate Into 3–D Method for Dry Sand

Effectiveness of Rotation into 3-D “Reconstructed” Composite Particle #1 Dry Melt Daytona Beach Michigan Dune

Tomographic Method

Effectiveness of Tomographic “Reconstructed” Composite Particle #1 Dry Melt Daytona Beach Michigan Dune

Results of Dry Sand “Reconstruction” Distance Reconstruction Method Inter-ellipsoid Distance Percentages Dry Melt Daytona Beach Michigan Dune Extrusion 31% 40% 100% 46% 3-D Rotation 60% 73% 17% Tomographic 45% 33% 85%

Conclusion 2-D facets of 3-D particles in mix Composite 3-D “Reconstruction” 2-D facets of Composite Particle 2-D Descriptors from Mix 3-D Descriptors 2-D Descriptors from Particle

Summary of Accomplishments Development of automated algorithms that can estimate 3-D shape descriptors for particle aggregates Statistical combination of 2-D shape descriptors from multiple 2-D projections Database containing a library of 2-D digital images for 5 aggregate mixtures PCA and ellipsoid model to show consistency, separability and uniqueness of the algorithm Composite 3-D shapes from multiple 2-D projections. Extrusion, Rotation and Tomographic reconstruction

Conclusions Dissimilar soil mixes can be separated using the descriptor algorithms Generation of random projections from the Fourier descriptors proves to be effective Construction of a 3-D composite particle using a collection of 2-D projections appears feasible

Recommendations for Future Work The optimal number and value of descriptors can be found, which allows the greatest separability More work on Reconstruction Methods Extrusion – use more projections on more axes Tomographic – Rotate more images about multiple axes and combine objects Apply composite particles created to a discrete element model Algorithms can be applied to other application areas (i.e. ink toner, industrial)

Acknowledgements National Science Foundation, Division of Civil and Mechanical Systems, Geomechanics and Geotechnic Systems Program, Award #0324437 Dr. Shreekanth Mandayam, Dr. Beena Sukumaran, and Dr. Robi Polikar Michael Kim and Scott Papson