1. Materials Process Design and Control Laboratory Sethuraman Sankaran and Nicholas Zabaras Materials Process Design and Control Laboratory Sibley School of Mechanical and Aerospace Engineering 188 Frank H. T. Rhodes Hall Cornell University Ithaca, NY 14853-3801 Email: ss524@cornell.edu, zabaras@cornell.edu URL: http://mpdc.mae.cornell.edu/ An Information-theoretic Tool for Property Prediction Of Random Microstructures

2. Materials Process Design and Control Laboratory RESEARCH SPONSORS U.S. AIR FORCE PARTNERS Materials Process Design Branch, AFRL Computational Mathematics Program, AFOSR CORNELL THEORY CENTER ARMY RESEARCH OFFICE Mechanical Behavior of Materials Program NATIONAL SCIENCE FOUNDATION (NSF) Design and Integration Engineering Program

3. Materials Process Design and Control Laboratory An overview Mathematical representation of random microstructures Extraction of higher order features from limited microstructural information : the MAXENT approach MAXENT optimization schemes Evaluation of homogenized elastic properties from microstructures Effect of varying information content on property statistics Numerical examples Summary and future work

4. Materials Process Design and Control Laboratory Idea Behind Information Theoretic Approach Statistical Mechanics Information Theory Rigorously quantifying and modeling uncertainty, linking scales using criterion derived from information theory, and use information theoretic tools to predict parameters in the face of incomplete Information etc Linkage? Information Theory Basic Questions: 1. Microstructures are realizations of a random field. Is there a principle by which the underlying pdf itself can be obtained. 2. If so, how can the known information about microstructure be incorporated in the solution. 3. How do we obtain actual statistics of properties of the microstructure characterized at macro scale.

5. Materials Process Design and Control Laboratory Representation of random microstructures  Indicator functions used to represent microstructure at different regions in the physical domain  Indicator functions take values over a binary alphabet  Statistical features of microstructure are mathematically tractable in terms of expected values over indicator functions if Two-phase material if n-phase material I i I i Define I i as the set comprising (I i (x 1 ), I i (x 2 ), … I i (x n )). I i represents a random field of indicator functions over the domain. Microstructures are hierarchically characterized over a set of random variables of this field

6. Materials Process Design and Control Laboratory Defining correlation vectors using indicator functions Two-point probability functions Lineal Path Functions n-point probability functions

7. Materials Process Design and Control Laboratory Microstructure Reconstruction Schemes

8. Materials Process Design and Control Laboratory Reconstruction of microstructures Correlation features of desired microstructures is provided. Aim to reconstruct microstructures that satisfy these ensemble statistical properties. Ill-posed problem with many distributions that satisfy given ensemble properties. Pb-Sn microstructures High strength steel microstructures obtained by thermal processing Media with short range interactions

9. Materials Process Design and Control Laboratory Current schemes for microstructure reconstruction D. Cule and S. Torquato ’99 Reconstruction of porous media using Stochastic Optimization C. Manwart, S. Torquato and R. Hilfer ’00, Reconstruction of sandstone structures using stochastic optimization N. Zabaras et.al. ’05 Reconstruction of microstructures using SVM’s T.C.Baroni et al. ’02, Reconstruction of microstructures using contrast imaging techniques A.P. Roberts ’97, Reconstruction of porous media using image mapping techniques from 2d planar images. Input: Given statistical correlation or lineal path functions Obtain: microstructures that satisfy the given properties Start from a random configuration over the specified problem domain such that the volume fraction information is satisfied. Randomly choose two locations (pixels) and define a move by interchanging the intensities of the two pixels. If the error norm defined as the deviation of the correlation features from target features reduces, accept the move, otherwise reject it. Stochastic Optimization Procedure

10. Materials Process Design and Control Laboratory A MAXENT viewpoint

11. Materials Process Design and Control Laboratory Information Theoretic Scheme: the MAXENT principle Input: Given statistical correlation or lineal path functions Obtain: microstructures that satisfy the given properties Constraints are viewed as expectations of features over a random field. Problem is viewed as finding that distribution whose ensemble properties match those that are given. Since, problem is ill-posed, we choose the distribution that has the maximum entropy. Additional statistical information is available using this scheme.

12. Materials Process Design and Control Laboratory The MAXENT Principle The principle of maximum entropy (MAXENT) states that amongst the probability distributions that satisfy our incomplete information about the system, the probability distribution that maximizes entropy is the least-biased estimate that can be made. It agrees with everything that is known but carefully avoids anything that is unknown. E.T. Jaynes 1957 A MAXENT viewpoint Trivial case: no information is available about microstructure. From MAXENT, the equiprobable case is the case with maximum entropy for an unconstrained problem. This agrees with intuition as to the most unbiased case Information about volume fraction given. The MAXENT distribution is one wherein we sample from the volume fraction distribution itself at all material points Correlation between material points to be taken into account. Result is not trivial and needs to be numerically computed Higher order information provided

13. Materials Process Design and Control Laboratory MAXENT as a feature matching tool D. Pietra et al. ‘96, MAXENT principle for language processing. Features of language extracted and MAXENT principle is used to develop a language translator Zhu et al. ‘98, MAXENT principle for texture processing Texture features from images in the form of histograms is extracted and MAXENT principle used to reconstruct texture images Sobczyk ’03 MAXENT used for obtaining distributions of grain sizes from macro constraints in the form of expected grain size. Koutsourelakis ‘05, MAXENT for generation of random media. Correlation features of random media used as constraints to generate samples of random media.

14. Materials Process Design and Control Laboratory MAXENT for microstructure reconstruction MAXENT is essentially a way of generating a PDF on a hypothesis space which, given a measure of entropy, is guaranteed to incorporate only known constraints. MAXENT is essentially a way of generating a PDF on a hypothesis space which, given a measure of entropy, is guaranteed to incorporate only known constraints. MAXENT cannot be derived from Bayes theorem. It is fundamentally different, as Bayes theorem concerns itself with inferring a-posteriori probability once the likelihood and a-priori probability are known, while MAXENT is a guiding principle to construct the a-priori PDF. MAXENT cannot be derived from Bayes theorem. It is fundamentally different, as Bayes theorem concerns itself with inferring a-posteriori probability once the likelihood and a-priori probability are known, while MAXENT is a guiding principle to construct the a-priori PDF. We associate the PDF with a microstructure image and generate samples of the image. We associate the PDF with a microstructure image and generate samples of the image. MAXENT produces images with features (information) that are consistent with the known constraints. Another way of stating this is that MAXENT produces the most uniform distribution consistent with the data. MAXENT produces images with features (information) that are consistent with the known constraints. Another way of stating this is that MAXENT produces the most uniform distribution consistent with the data.

15. Materials Process Design and Control Laboratory MAXENT optimization schemes

16. Materials Process Design and Control Laboratory Find Subject to Lagrange Multiplier optimization feature constraints features of image I MAXENT as an optimization problem Partition Function

17. Materials Process Design and Control Laboratory Equivalent log-linear model Find that maximizes Equivalent log-likelihood problem Kuhn-Tucker theorem: The that maximizes the dual function L also maximizes the system entropy and satisfies the constraints posed by the problem Direct models Log-linear models ConcaveConcave Constrained (simplex) Unconstrained “Count and normalize” (closed form solution) Iterative methods AComparison

18. Materials Process Design and Control Laboratory Optimization Schemes Generalized Iterative Scaling Generalized Iterative Scaling Improved Iterative Scaling Improved Iterative Scaling Gradient Ascent Gradient Ascent Newton/Quasi-Newton Methods Newton/Quasi-Newton Methods – Conjugate Gradient – BFGS – … Start from a equal to 0. This is equivalent to uniform distribution over sample space. Start from a equal to 0. This is equivalent to uniform distribution over sample space. Evaluate gradient at this point. Evaluate gradient at this point. Perform a line search on a direction based on the gradient information. Perform a line search on a direction based on the gradient information. Evaluate the gradient information at the next point and continue the procedure till it is within tolerance limit. Evaluate the gradient information at the next point and continue the procedure till it is within tolerance limit.

19. Materials Process Design and Control Laboratory Gradient Evaluation Objective function and its gradients: Objective function and its gradients: Infeasible to compute at all points in one conjugate gradient iteration Infeasible to compute at all points in one conjugate gradient iteration Use sampling techniques to sample from the distribution evaluated at the previous point. (Gibbs Sampler) Use sampling techniques to sample from the distribution evaluated at the previous point. (Gibbs Sampler) stochastic function

20. Materials Process Design and Control Laboratory Sampling techniques Sample from an exponential distribution using the Gibbs algorithm Sample from an exponential distribution using the Gibbs algorithm Choose a random point. Choose a random point. Evaluate the effective “energy” for various phases at that point using the updation algorithm to estimate “energy”. Evaluate the effective “energy” for various phases at that point using the updation algorithm to estimate “energy”. Draw a sample from the given distribution and replace the pixel value at the material point. Draw a sample from the given distribution and replace the pixel value at the material point. Continue the procedure till a sufficiently large number of samples are drawn. Continue the procedure till a sufficiently large number of samples are drawn.

21. Materials Process Design and Control Laboratory Updation Scheme r zone of influence Rozman,Utz ‘01 Rozman,Utz ‘01 Two point Correlation Function Lineal Path Function A scheme to update correlation function of an image when the phase of a single pixel is changed r zone of influence (region where correlation function is affected) Material point whose intensity is changed

22. Materials Process Design and Control Laboratory Line search and conjugate directions  Brent’s parabolic interpolation used for line search.  Stabilization in conjugate gradient machinery (Schraudolph ’02) Add a correction term so that as line search becomes increasingly inaccurate, its effect on the conjugate direction is also subdued. Stabilization term

23. Materials Process Design and Control Laboratory Optimization Schemes Convergence analysis with stabilization Convergence analysis w/o stabilization Noise in function evaluation increases as step size for the next minima increases. This ensures that the impact on the next evaluation is reduced.

24. Materials Process Design and Control Laboratory Entropy variation during MAXENT algorithmic scheme

25. Materials Process Design and Control Laboratory Evaluation of effective elastic properties

26. Materials Process Design and Control Laboratory Effective elastic property of microstructures Variational Principle: Subject to applied loads and other boundary conditions, minimize the energy stored in the microstructure. Pixel based mesh with a single phase inside each pixel (E. Garboczi, NIST ’98). Each pixel attributed the property of that particular phase. Homogenization: The effective homogenized property of the microstructure is obtained by equating energy of microstructure with that of a specimen with uniform properties

27. Materials Process Design and Control Laboratory Consolidated Algorithm Experimental images Analytical Correlation functions Extract features and rephrase as mathematical constraints Pose as a MAXENT problem and use gradient-based schemes for obtaining solution Use Gibbs sampling algorithm for sampling from underlying distribution Generate samples and interrogate using FEM Obtain property statistics and use them for further analysis

28. Materials Process Design and Control Laboratory Numerical Examples

29. Materials Process Design and Control Laboratory Example 1

30. Materials Process Design and Control Laboratory Reconstruction of 1d hard disks Reconstruct one-dimensional hard disk microstructures based on two different kinds of information: (a) two-point correlation functions (b) two point correlation and Lineal path function. Obtain elastic property statistics and compare for the two schemes. Input: Analytical two-point and lineal path functions (Torquato et.al. ’99)

31. Materials Process Design and Control Laboratory Microstructures based on two-point correlation function MAXENT distribution

32. Materials Process Design and Control Laboratory Microstructures based on two-point and lineal path function MAXENT distribution

33. Materials Process Design and Control Laboratory Comparison of property statistics between two schemes

34. Materials Process Design and Control Laboratory Example 2

35. Materials Process Design and Control Laboratory Porous Media with short range order To generate microstructures of porous media which exhibit short range orders of given specific structure. (S 2 is the two point correlation function, k and r o depend depend on characteristic length scales chosen) Problem Parameters correlation length r o = 32 oscillation parameter a o = 8 Input: Analytical two-point correlation functions (Torquato et.al. ’99)

36. Materials Process Design and Control Laboratory Property statistics for media with short range order MAXENT distribution

37. Materials Process Design and Control Laboratory Example 3

38. Materials Process Design and Control Laboratory Reconstruction using heterogeneous graded materials Heterogeneous Graded Materials Given a description of the gradation of phase-distribution in a graded material, reconstruct microstructures compatible with the given information, estimate statistics of microstructure properties from this set. Applications Tools with desirable properties at tips. Artificial joints for implants in humans Input: Analytical volume fraction information throughout sample (Koutsourelakis ’04)

39. Materials Process Design and Control Laboratory Samples of bilinearly graded heterogeneous materials at smooth resolution levels

40. Materials Process Design and Control Laboratory Effective elastic properties for a tungsten-silver bilinear graded material at 25 o C Elastic properties of bilinear graded materials

41. Materials Process Design and Control Laboratory Conclusions and future work

42. Materials Process Design and Control Laboratory Conclusions Microstructures were characterized stochastically and scheme for obtaining samples based on a MAXENT and time efficient update scheme implemented. Gradient based schemes and property of system entropy were analyzed in detail. Elastic properties were obtained using FEM and property statistics developed Schemes were discussed for numerical microstructures and effect of incorporation of higher information on property statistics studied.

43. Materials Process Design and Control Laboratory Future Work Extend the method for polycrystal materials incorporating information in the form of odf’s. Couple the scheme with pixel based methods for obtaining plastic properties. Extend the method to physical deformation processes taking into account the evolution of microstructure.

44. Materials Process Design and Control Laboratory References 1. E.T. Jaynes, Information Theory and Statistical Mechanics I, Physical Review 106(4)(1957) 620—630. 2. D. Cule and S. Torquato, Generating random media from limited microstructural information via stochastic optimization, Journal of Applied Physics 86(6)(1999) 3428— 3437 3.P.S. Koutsourelakis, A general framework for simulating random multi-phase media, NSF Workshop-Probability and Materials: From Nano to Macro scale (2005) 4. K. Sobczyk, Reconstruction of random material microstructures: patterns of Maximum Entropy, Probabilistic Engineering Mechanics 18(2003) 279—287 5. S.C.Zhu et al, Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling, IJCV 27(1998) 107-126 6. A.Berger et.al., A maximum entropy approach to natural language modeling, (1996), Computational Linguistics 22 (1996),39-71