1. Myopic Policies for Budgeted Optimization with Constrained Experiments Javad Azimi, Xiaoli Fern, Alan Fern Oregon State University AAAI, July 2010 1

2. Motivation: Electricity Production in a Microbial Fuel Cell (MFC) Anode Cathode bacteria Oxidation products (CO 2 ) Fuel (organic matter) e-e- e-e- O2O2 H2OH2O H+H+ This is how an MFC works SEM image of bacteria sp. on Ni nanoparticle enhanced carbon fibers. Nano-structure of anode significantly impact the electricity production. We should optimize anode nano-structure to maximize power. 2

3. Experiment Selection Experiments are costly and there is a fixed budget. How to select the best sequence of experiments. 3 Current Experiments Scientist selects Experiment Run Experiment

4. Bayesian Optimization (BO) Since Running experiment is very expensive we use BO. Select one experiment to run at a time based on results of previous experiments. Current Experiments Gaussian Process Surface Select Single Experiment Run Experiment 4

5. Bayesian Optimization (BO) BO assumes that we can ask for specific experiment. This is unreasonable assumption in many applications. – In Fuel Cell it takes many trials to create a nano-structure with specific requested properties. – Costly to fulfill. 5 Space of Experiments Average Circularity Averaged Area

6. Constrained Experiments It is less costly to fulfill a request that specifies ranges for the nanostructure properties E.g. run an experiment with Averaged Area in range r1 and Average Circularity in range r2 We will call such requests “constrained experiments” Space of Experiments Average Circularity Averaged Area Constrained Experiment 1 large ranges low cost high uncertainty about which experiment will be run Constrained Experiment 2 small ranges high cost low uncertainty about which experiment will be run 6

7. BO for Constrained Experiment Given a fixed budget, select the best constrained experiments. 7 Run Experiment Current Experiments Gaussian Process Surface Select Single Experiment Select Constrained Experiment

8. Constrained Experiment We generalized BO heuristics to constrained experiments. Two challenges: – How to compute heuristics for constrained experiment? – How to take experimental cost into account?(which has been ignored by most of the approaches in BO). 8

9. Standard BO Heuristics Standard heuristics are statistics of the posterior p(y|x,D) where D is our current observation. Maximum Upper bound Interval (MUI) – Select point with highest 95% upper confidence bound – Purely explorative approach. Maximum Probability of Improvement (MPI) – It computes the probability that the output is more than (1+m) times of the best current observation, m>0. – Explorative and Exploitative. Maximum Expected of Improvement (MEI) – Similar to MPI but parameter free – It simply computes the expected amount of improvement after sampling at any point. 9

10. Generalizing BO for Constrained Experiment Having the posterior distribution of p(y|x,D) and p x (.|D) we can calculate the posterior of the output of each constrained experiment which has a closed form solution. Therefore we can compute standard BO heuristics for constrained experiments. – There are closed form solution for these heuristics. 10 Input space Discretization Level

11. Budgeted Constrained Experiments We are limited with Budget B. Unfortunately heuristics will typically select the smallest and most costly constrained experiments which is not a good use of budget. How can we consider the cost of each constrained experiment in making the decision? – Cost Normalized Policy (CN) – Constraint Minimum Cost Policy(CMC) 11 -Low uncertainty -High uncertainty -Better heuristic value -Lower heuristic value -Expensive-Cheap

12. Cost Normalized Policy It selects the constrained experiment achieving the highest expected improvement per unit cost. We report this approach for MEI policy only. 12

13. Constraint Minimum Cost Policy (CMC) Motivation: 1.Approximately maximizes the heuristic value. 2.Has expected improvement at least as great as spending the same amount of budget on random experiments. Example: Very expensive: 10 random experiments likely to be better Selected Constrained experiment Poor heuristic value: not select due to 1 st condition 13 Cost=4 random Cost=10 randomCost=5 random

14. Experimental Results (Setup) Gaussian process is used as our model with squared exponential kernel. Cost function is defined as: – There is a constant cost for running any constrained experiment plus an additional cost depending on the size of the experiment. – The value of slope dictates how fast the cost increases as the size of a constrained experiment decreases. 14 Space of Experiments r1 r2

15. Experimental Results (Real Applications) Two Real data sets: – Fuel Cell: Fuel Cell electricity generation – Hydrogene: Biosolar hydrogen production 15

16. Experimental Results (Benchmarks Functions) 3 popular benchmarks used in BO literature. 16 Rosenbrock DiscontinuousCosines

17. Overall Performance The normalized regret of each framework is shown which is calculated as y * - y max over Random performance for budget 15 where y* is the highest possible output. 17 Average regret of each approach over all frameworks.

18. Different Budget(1) 18 Random Cosines Fuel Cell Real Rosenbrock CMC-MUI

19. Different Budget(2) 19 CN-MEI Cosines Fuel Cell Real Rosenbrock

20. Different Budget(3) 20 CMC-MPI(0.2) Cosines Fuel Cell Real Rosenbrock

21. Different Budget(4) 21 CMC-MEI Cosines Fuel Cell Real Rosenbrock

22. Conclusion We introduced a new constrained experiment framework which asks for hyper-rectangle rather than exact point. We extended model-free BO heuristics to our frame work. We introduced two approaches to optimize our budgeted framework. CMC-MEI is working better than other approaches. 22

23. Thanks for attendance Question? 23