1. Håkan L. S. YounesDavid J. Musliner Carnegie Mellon UniversityHoneywell Laboratories Probabilistic Plan Verification through Acceptance Sampling

2. Introduction Probabilistic extension to CIRCA Efficient plan verification algorithm Monte Carlo simulation Acceptance sampling Guaranteed error bounds

3. Planning via Model Checking Planner Model checker candidate plan verification result objectives, environment safety constraints

4. World Model States… normal path no threat normal path radar threat evasive path radar threat evasive path no threat FAILURE

5. World Model States + events = environment normal path no threat radar threat Exp(150) hit Exp(50) + 120 safe U(50,100) normal path radar threat evasive path radar threat evasive path no threat FAILURE

6. World Model A plan maps states to actions normal path no threat radar threat Exp(150) hit Exp(50) + 120 safe U(50,100) end evasive U(25,50) begin evasive U(25,50) normal path radar threat evasive path radar threat evasive path no threat FAILURE

7. Sample Execution Paths normal path no threat normal path radar threat evasive path radar threat evasive path no threat normal path no threat radar threatbegin evasivesafeend evasive 41.945.893.543.4… normal path no threat normal path radar threat evasive path radar threat FAILURE begin evasivehitradar threat 44.148.792.2

8. Plan Safety Two parameters Failure probability threshold:  Maximum execution time: t max A plan is safe if the probability of reaching a failure state within t max time units is at most 

9. Safety Over Sample Execution Paths Given t max = 200: normal path no threat normal path radar threat evasive path radar threat evasive path no threat normal path no threat radar threatbegin evasivesafeend evasive 41.945.893.543.4… +++ > 200 Safe!

10. Safety Over Sample Execution Paths Given t max = 200: normal path no threat normal path radar threat evasive path radar threat FAILURE begin evasivehitradar threat 44.148.792.2++ ≤ 200 Not safe! (safe if t max < 185)

11. Verifying Plan Safety Symbolic Methods Pro: Exact solution Con: Works only for restricted class of models Sampling Pro: Works for any model that can be simulated Con: Uncertainty in correctness of solution

12. Our Approach Use simulation to generate sample execution paths Use sequential acceptance sampling to verify plan safety

13. Error Bounds Probability of false negative: ≤  We say that a plan is not safe when it is Probability of false positive: ≤  We say that a plan is safe when it is not

14. Acceptance Sampling Test hypothesis Pr ≤  (X) In our case  is the failure probability threshold X is the proposition that a failure state is reached within the time limit

15. Sequential Acceptance Sampling Test hypothesis Pr ≤  (X) True, false, or another sample?

16. Performance of Test Actual failure probability of plan Probability of accepting Pr ≤  (X) as true  1 –  

17. Ideal Performance False positives Actual failure probability of plan Probability of accepting Pr ≤  (X) as true  1 –   False negatives

18. Actual Performance  –  +  Actual failure probability of plan Probability of accepting Pr ≤  (X) as true  1 –   False positives False negatives Indifference region

19. Graphical Representation of Sequential Test Number of samples Number of negative samples

20. Graphical Representation of Sequential Test We can find an acceptance line and a rejection line given , , , and  Accept Reject Continue sampling Number of samples Number of negative samples

21. Graphical Representation of Sequential Test Accept hypothesis Accept Reject Continue sampling Number of samples Number of negative samples

22. Graphical Representation of Sequential Test Reject hypothesis Accept Reject Continue sampling Number of samples Number of negative samples

23. Example Verify plan with  =0.05,  =0.01,  =  =0.05, t max =200 normal path no threat radar threat Exp(150) hit Exp(50) + 120 safe U(50,100) end evasive U(25,50) begin evasive U(25,50) normal path radar threat evasive path radar threat evasive path no threat FAILURE

24. Example Verify plan with  =0.05,  =0.01,  =  =0.05, t max =200 Simulator 150100 200 2 6 Number of samples 8 Negative samples 50 4 10 12 14 16 18

25. Performance Failure probability Average number of samples  = 0.01,  =  = 0.05  = 0.01,  =  = 0.10  = 0.02,  =  = 0.05  = 0.02,  =  = 0.10 

26. Summary Probabilistic extension to CIRCA Allows for plans with non-zero failure probability Efficient plan verification algorithm based on acceptance sampling Guaranteed error bounds Easy to trade efficiency for accuracy

27. Future Work Sensitivity analysis Using verification result to guide plan generation “Generalized semi-Markov Decision Processes”