1. Design and Analysis of Nonlinear Models for the Mars 2020 Rover
March 21, 2018

2. Plan Background on Mars Rover program
Fitting data to nonlinear models: drill bit degradation
Creating a design with a known nonlinear functional form: binary response for core quality
Augmenting with a new factor known to have a nonlinear effect: spindle torque and space-filling designs

3. Mars 2020 Rover
The Mars 2020 rover will robotically explore the red planet’s surface for at least 1 Martian year (687 Earth days)
Builds on success of Mars Science Laboratory’s Curiosity Rover to minimize program risk

4. Sample Caching Subsystem
Few words, mostly pictures about SCS
Basic concept of sample caching
Photo of all the cores?

5. Subsystem Design We don’t get to control Mars! Key requirements:
Subsystem design highly informed by testing
Key requirements:
Collect ~40 cores of varying sample types, based on notional distribution of Earth analog rocks
Core quality - Best core has a few number of large pieces
Samples must be “hermetically” sealed
Measurable responses
Core quality
Mass and number of pieces to pass through sieves of 2, 5, 10 and >10 mm
Sample volume
Seal leak rate
Drilling Performance
Avg cycle sideload
Avg cycle percussion current
Avg drilling torque
Avg percussion power
Avg rate of penetration
PEC068
Can’t control: rock properties, environment
PEC113

6. Nonlinear Modeling Intro Modelling Coring Bit Wear
Problem: How can we model the impact of bit wear on drill performance metrics? Is there a surrogate measure for bit wear?
Methodology: Measure bit wear 4 times over a series of 30 tests (not an easy task), fit candidate nonlinear models, create new control variable as bit wear over time, run regression models against easily observable factors
Results:
Bit wear approximated well by Logistic 3Parameter Sigmoid Curve
Makes sense from physics of failure/degradation models
Highly correlated with Time in USB rocks
Useful control variable for many of the responses

7. Nonlinear Modeling Intro Finding the Right Model
3 Parameter Logistic Model

8. Nonlinear Modeling Intro Finding a Bit Wear Surrogate
Overall core quality decreases with bit wear

9. Nonlinear Modeling Intro Using Nonlinear Predictor

10. Design for Nonlinear
Problem: Ok got it that you can model nonlinear, but how do you actually set yourself up to have the right tests knowing you have a nonlinear relationship. Specifically, we have a binary response on core quality so we know logistic regression is our nonlinear relationship.
Methodology: Limited options to take advantage of what you know to some level of the functional form. Bayesian D-Optimal Designs can help.
Results:
Designs differ from Custom RSM
Better estimates where matters
General approach

11. Nonlinear Designs
2-level factorial designs are good for linear models and we can use centers to detect curvature and then augment with axials to determine quadratic effects
Alternatively, use a response surface design or definitive screening design from the beginning if know higher order effects likely to be present
Not all responses modeled adequately by 2nd order Taylor Series approximation with normal error
Engineering principals may point functional form to a different direction with complex curvature
Possible to model nonlinearities, but how do you create a design that is set up to maximize the precision given the nonlinear behavior?
To optimize the design points, you need to know the functional form and a good estimate of the parameters (so why are we testing if we know all this already???)

12. Nonlinear Designs JMP Implementation
Consider using a Bayesian approach where the parameter values are restricted to some reasonable range via the prior distribution
Bayesian D-optimal criteria is expectation of Log |Information| with respect to parameter vectors from the prior
Where D is matrix of partials of the expectation function evaluated wrt each
model parameter evaluated at each design point
Gotwalt, Jones, and Steinberg (Technometrics, 2009) solve this and implement in JMP
Goal is to place points at high prediction variance points in the design space—not the vertices like linear models though!
Zoomed in view of sample tube, explain the 3 pieces that comprise the seal
Why samples must be “hermetically” sealed

13. Nonlinear Designs Binomial Example
Core Quality is a function of many variables and ultimately a very noisy and nonlinear process
For simplicity, we model whether or not core was good or not as a function of Maximum Sideload and Time in HardRocks in a logistic regression model for USB rock

14. Nonlinear Designs Binomial Example
We want to design a set of tests accounting for these two factors given a binary response
We know the functional form of logistic regression
1 / (1 + Exp( -(b0 + b1 * :AvgCycleMaxSideload + b2 * :TimeInHardRocks) ))
We also have historical data to suggest parameter estimates to inform our prior distribution
Create new data table with columns for two input factors Sideload and Time as well as a continuous response
Create column that has logistic regression model formula specifying parameters for b0, b1, and b2 {b0 = -5, b1 = 0.25, b2 = 0.015}
From this table run Nonlinear DOE to input the range of factors and the prior information
Specify number of runs and create design
Plot factors to see geometry

15. Nonlinear Designs Binomial Example
Default Diffuse Prior
Informative Prior

16. Nonlinear Designs Augmenting Binomial Example
Augment existing diffuse prior with 6 more runs (diffuse, uniform) and another 6 (informative, uniform)

17. Design for Nonlinear
Problem: We’ve run an original design in 7 factors and want to augment to account for a new nonlinear variable. We want to run this new variable at 5 settings. Oh, and we didn’t record its level during the original runs.
Methodology: Augmenting designs to get nonlinear relationships is common, augmenting to account for a variable not in the original design is not common particularly if you require it to be at 5 levels.
Results:
Used JPL (Jones Practical Level) Covariates: Consider all of the factors as covariates when augment; replicate possible candidate points; use custom design to balance across the 5 levels of new factors.
New design will be used to capture non-linear effect of

18. Design for Nonlinear
Original Design: Two 65 run space filling designs in 6 factors; R2 = .96; slight curvature in one factor
Realized in other experiments sensitivity (nonlinear) to another factor not originally included: chuck tip angle
Question: how can we augment the current design to include a new discrete numeric factor at 5 levels while building on what we know?

19. JPL Covariate Approach
Augment original space filling design with the number of additional runs you desire—20 for us.
Add a column for new factor
Record its level if known for original design
Replicate the new augmented runs across the discrete numeric levels resulting in more runs than want.
Build a Custom Design
Use all factors as covariates
Force original runs in new design
Custom will balance remaining 20 runs across levels of the discrete numeric value using
Execute new runs and analyze with nonlinear if needed

20. Summary
High priority Mars 2020 program is experimenting to accurately characterize and optimize coring performance
Many relationships are nonlinear and have follow expected physics and engineering relationships
Improved results are possible by considering a Bayesian D-Optimal Design, augmenting with a new nonlinear variable using the JPL covariate approach, and fitting nonlinear surfaces to known functions
“I would like to die on Mars—just not on impact” Elon Musk

21. Questions