1. 1 Automatic deployment of robotic teams from rich specifications Calin Belta Hybrid and Networked Systems (HyNeSs) Lab Departments of Mechanical / Systems Engineering Boston University

2. 2 Rich, natural-language specification, e.g., “USV always avoid P6. UGVs always avoid P4. Any UGV visit P1 or P2 and then go to P3. After P3 is occupied, USV go to P7 and UGV2 start surveilling P1 and P2” P1 P3 P2 P4 P5 UGV1 UGV2 UGV4 UGV3 UAV USV UUV P6 P7 Automatic synthesis of provably-correct control and communication strategies. Problem 1: Deployment of small heterogeneous teams (<10)

3. 3 “Always obey traffic rules. Visit Road R1 and then Road R2 without crossing intersection I1. If Road R8 is ever visited, then Road R3 must never be reached. Park simultaneously in adjacent parking spaces and remain there for all future times.” Rich, natural language specification Robotic Urban-Like Environment (RULE) Automatic synthesis of provably-correct control and communication strategies. Problem 1: Deployment of small heterogeneous teams (<10)

4. 4 “Eventually cover an elliptic region with center c1, semiaxes s1 and s2, and orientation θ, then a circular region with center c2 and area a, and maintain this configuration for all future times. Always maintain a pair-wise distance Dmin < D < Dmax. Always avoid obstacles” Problem 2: Deployment of large homogeneous swarms Rich, natural-language specification over a small set of global features, e.g., Automatic synthesis of control and communication strategies.

5. 5 Motivation “USV always avoid P6. UGVs always avoid P4. Any UGV visit P1 or P2 and then go to P3. After P3 is occupied, USV go to P7 and UGV2 start visiting P1 and P2, in this order, infinitely often.” “Always obey traffic rules. Visit Road R1 and then Road R2 without crossing intersection I1. If Road R8 is ever visited, then Road R3 must never be reached. Park simultaneously in adjacent parking spaces and remain there for all future times.” Place the human operator as “high” as possible in the decision hierarchy Formalize the human-robot interaction process

6. 6 Approach Draw inspiration from formal analysis (verification) “Is deadlock ever possible?” “If a request is received, make sure it is eventually granted.” Specification Process “Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.” ? Analysis / control Compositionality Model Model checking (SPIN, NuSMV)

7. 7 Outline Finite quotients and control of continuous systems Deployment for small teams Deployment for swarms Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.”

8. 8 Outline Finite quotients and control of continuous systems Deployment for small teams Deployment for swarms Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.”

9. 9 Feedback automaton Language equivalence! control state Feedback controller region Feedback hybrid automaton Finite quotients and control of continuous systems “Avoid the grey region for all times. Visit the blue region, then the green region, and then keep surveying the striped blue and green regions, in this order.” “(pi2 = TRUE and pi4 = FALSE and pi3 = FALSE) should never happen. Then pi4 = TRUE and then pi1 = TRUE should happen. After that, (pi3 = TRUE and pi4 = TRUE) and then (pi1 = TRUE and pi3 = FALSE) should occur infinitely often.”

10. 10

11. 11 Outline Finite quotients and control of continuous systems Deployment for small teams Deployment for swarms Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.”

12. 12 Deployment of small teams Arbitrary temporal and logic statements about the reachability of regions in a partitioned environment, e.g., “Avoid the blue regions until the green and red regions are simultaneously visited. Then visit any one of the blue regions” Provably-correct control strategies and communication protocols Assumptions: the specification is robot-abstract the robots can only communicate when in adjacent regions (varying communication constraint) communication relation is symmetric all robots in a component of the communication graph can instantaneously communicate

13. 13 “Avoid the blue regions until the green and red regions are simultaneously visited. Then visit any one of the blue regions” Deployment of small teams

14. 14 “Avoid the blue regions until the green and red regions are simultaneously visited. Then visit any one of the blue regions” Deployment of small teams ???? ???

15. 15 “Do not visit any yellow region until all are simultaneously entered, and always avoid the gray regions” Assume we have three agents Deployment of small teams Example Simulation - movie

16. 16 Outline Finite quotients and control of continuous systems Deployment for small teams Deployment for swarms Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.”

17. 17 Examples: “Eventually reach a configuration with mean ml < m < mh and covariance s1l < s1 < s1h, s2l < s2 < s2h, θl < θ < θh and maintain it for all future times. Always maintain a pair-wise distance Dmin < D < Dmax (inter-robot collision avoidance, maintaining the connectivity of the communication graph, coverage). Always avoid obstacles” Automatic synthesis of provably- correct control and communication strategies. Communication graph Sensing range Deployment for swarms

18. 18 Examples: “Eventually reach a configuration with mean ml < m < mh and covariance s1l < s1 < s1h, s2l < s2 < s2h, θl < θ < θh and maintain it for all future times. Always maintain a pair-wise distance Dmin < D < Dmax (inter-robot collision avoidance, maintaining the connectivity of the communication graph, coverage). Always avoid obstacles” Automatic synthesis of provably- correct control and communication strategies. m θ s1 s2 Deployment for swarms

19. 19 Approach

20. 20 Approach

21. 21 Approach

22. 22 Collision avoidance: Quantifier elimination always Approach

23. 23 always eventually and thenand Approach

24. 24 Approach always eventually and thenand

25. 25 Distributed communication architecture Lower dimensional continuous description Set of essential features of the swarm Output: provably correct control laws Continuous abstraction Hierarchical abstraction architecture Finite dimensional discrete description Input: Temporal logic specification over essential features Discrete abstraction Provably correct control law Hierarchical abstractions

26. 26 Distributed communication architecture Lower dimensional continuous description Set of essential features of the swarm Output: provably correct control laws Continuous abstraction Hierarchical abstraction architecture Finite dimensional discrete description Input: Temporal logic specification over essential features Discrete abstraction Provably correct control law Continuous abstraction ?

27. 27 Continuous abstraction Consistency Equivalent states remain equivalent for all times under the flow Actuation At any point any velocity can be achieved. Detectability if and only if Correct aggregation = Consistency + Actuation + Detectability

28. 28 Continuous abstraction (non – scalability) is invariant to robot permutations Control architecture independent on the choice of a world frame is left invariant The group G and shape S can be controlled independently

29. 29 Continuous abstraction: examples group: SE(2) (mean, rotation diagonalizing the cov. matrix) shape: spectrum of cov. matrix group: SE(2) (centroid, rotation diagonalizing the in. tensor) shape: spectrum of in. tensor group: R 2 (centroid) shape: scaling factor (sum of eig) group: SE(3) (mean, rotation diagonalizing the in. tensor) shape: spectrum of in. tensor

30. 30 Distributed communication architecture Lower dimensional continuous description Set of essential features of the swarm Output: provably correct control laws Continuous abstraction Hierarchical abstraction architecture Finite dimensional discrete description Input: Temporal logic specification over essential features Discrete abstraction Provably correct control law Hierarchical abstraction

31. 31 Fully automated framework for swarm deployment from specifications allowing for: Containment, obstacle avoidance, inter- robot collision avoidance, cohesion Arbitrary tasks given in terms of LTL formulas over linear predicates over mean and variance of the swarm Hierarchical abstraction based on mean and variance

32. 32 30 robots with control bounds [-2,2] x [-2,2] in a rectangular environment with two obstacles Hierarchical abstraction based on mean and variance Example: Specification: Always stay inside environment and avoid obstacles and Visit region R1 before reaching region R2 with area greater than 4 and Before visiting R1, make sure that the pairwise distances are all greater than 0.03

33. 33 Acknowledgements Funding: National Science Foundation (CNS, IIS, CCF) Air Force Office of Sponsored Research (Computational Mathematics) Army Research Office (Mathematical Sciences Division) Collaborators: M. Kloetzer (Boston University) V. Kumar, G. J. Pappas (U Penn) L.C.G.J.M. Habets (CWI, Eindhoven TU)