1. V-Lab  … An Intelligent Virtual Laboratory for Autonomous Agents – Towards Simulating System of Systems Mo Jamshidi, Ph.D., DEgr., Dr. H.C. F-IEEE, F-ASME, F-AAAS, F-NYAS, F-HAE, F-TWAS Regents Professor, Electrical and Computer Engr. Department and Director, Autonomous Control Engineering (ACE) Center University of New Mexico, Albuquerque Advisor, JPL (1992-93), NASA Headquarters (1996-2003) Sr. Research Advisor, US AF Research Lab. (1984-90,2001-present) Consultant, US DOE Office of Renewable Energy (2001-2003) http://ace.unm.edu www.vlab.unm.edu pursue.unm.edu, moj@wacong.orgwww.vlab.unm.edu

2. OUTLINE Introduction to V-Lab ® Discrete-EVent simulation Specification (DEVS) Intelligent DEVS … I-DEVS Applications to Multi-agent Systems System of Systems Engineering Simulation and Experiment Movies

3. V-Lab ® is … … a NASA Research Announcement (NRA), Cross-Enterprise Grant with NASA Ames Research Center and a joint effort between UNM ACE Center, JPL, in cooperation with the Arizona Integrate Modeling and Simulation Center (U Arizona and Arizona State U) Dr. Edward Tunstel, Jr. of JPL is one of the 3 Co-PIs on the grant.

4. V-Lab ® ’s potential role for NASA Robotic Colonies – System of Systems Paradigm MER

5. Simulation Design Qualities Layered Architecture Breaks up the entire simulation into layers, each with a distinct purpose and function Object Modularity Functional components of the individual layers should be broken up into distinct objects to allow for: Maintainability – Changing a single object without modifying entire simulation. Reuse – Distinct objects can be reused in different simulations Object Hierarchy – Provide a structure for the composition and communication of objects Our Solution – V-Lab®

6. V-Lab® V-Lab® Environment Layers Hardware Networking Networking foundation for inter- machine communication Middleware Software environment for inter- process communication (CORBA, HLA, Sockets, etc.) I-DEVS Library of tools for computational intelligence in DEVS formalism V-Lab® Organizational structure for DEVS objects. Management objects to control time and message flow in multi- agent systems PHYSICAL NETWORK MIDDLEWARE I-DEVS V-Lab®

7. DEVS Provides C++ and Java classes for object design Hierarchical structure of objects Two Types of Objects Coupled (A-BC, B-C) Contains other objects Atomic (A, B, C) Basic functionality to be used by coupled objects A-BC AB-C BC Hierarchal tree A-BC ABC B-C Coupling Relation

8. DEVS execution Create Models Determine imminent models Imminent models send messages to output ports Imminent models call Internal Transition function Other models receive external messages Other models call External Transition function Repeat until no models can be imminent

9. I DEVS … I ntelligent DEVS DEVS Coupled Models Atomic Models Atomic Models Discrete Events Discrete Events Computational Intelligence

10. Fuzzy-DEVS Object: Implement various fuzzy functions and operations within a DEVS Java environment. Example – fuzzy inferencing XY μ XY μ OR XY μ x y z A B C AMF CMF A DEVS model for a typical fuzzy rule: IF x is A OR y is B THEN z is C

11. Fuzzy-DEVS, Cont’d. Example of using fuzzy-DEVS to control an inverted pendulum a) Membership functions for fuzzy-DEVS b) The fuzzy-DEVS controller POS NEG ZERO c) The close-loop system d) The output of the system (  )

12. GA’s Basic Cycle EvaluationRecombination New Population Selection Old Population 011000110010 A binary chromosome Crossover Mutation

13. GA-DEVS, Cont’d. GA-DEVS implementation inside V-Lab®

14. GA-DEVS, Cont’d. 2 A simulation window of GA-DEVS

15. Stochastic Learning Automata What is SLA? SLA is a sequential machine. Given a finite number of actions that can be performed in a random environment, when a specific action is taken place the environment provides a random response which is either favorable or unfavorable. The objective in the design of the automaton is to determine how the choice of the action at any stage should be guided by past actions and responses.

16. Stochastic Learning Automata – Reinforcement Learning Update Action Probabilities G(.) F(w(n),s(n)) s(n) Stochastic Learning Automaton y(n)

17. SLA-DEVS Schematic of SLA inside the DEVS Environment Stochastic Learning Automaton – A statistical Learning approach to learning …

18. Simulation, DEVS-SLA … 2 Agents (a) (d)(c) (b)

19. Simulation, Cont’d … 2 Agents (e) (f) (g) (h)

20. Neural Networks Multi-Layer Perceptron Input Layer Hidden Layers Output Layer Input Output 1 Output 2 The interest in neural networks comes from the networks’ ability to mimic human brain as well as its ability to learn and respond.

21. NN-DEVS A perceptron structure inside DEVS Neural Networks – A neural-based approach to learning … BPNN

22. NN-DEVS Number of training data errors versus epochs (X, training epochs; Y, number of training data errors) Neural Networks – Logical connectives “AND”, “OR”, and “XOR” are implemented in IDEVS (b) OR(a) AND (c) XOR

23. V-Lab  Uses the I-DEVS structure for creating objects Organizes objects into 6 different categories SimEnv/SimMan Terrain Models Agent Models Physics Models Control Models Dynamic Models Provides objects for the time and message flow of simulation SimEnv Terrain Agents Physics Control SimManDynamic Hierarchical Tree

24. SimMan and SimEnv SimEnv Highest leveled coupled model Instatiates all other models Houses all other models Couples all other models together SimMan Message liaison providing indirection between all other models Controls flow of time for the simulation Dynamic SimEnv SimMan AgentControlPhysics Terrain out in out in/out Coupling Relation

25. High-Level Models Physics Model physical phenomena Possibly a differential equation solver and set of differential equations Terrain Information about the layout of the environment Dynamic Analyze agent’s actuator state to make environmental changes, e.g. agent’s position and velocity.

26. High-Level Models Control Algorithms that control the actuators of an agent model Agent Represent the agents in the simulation Contain Sensor models Sensor models contain all information about external world agent is aware of Contain Actuator models Actuators contain state information that dynamic models use

27. V-Lab® Cycle Simulation execution is iteration through a 5 phase cycle Phase 1 Check for termination conditions Phase 2 Update Agent Sensors Phase 3 Control Algorithms update Agent Actuators Phase 4 Wait for all actuators to change state Phase 5 Dynamic models update agents and environment

28. A V-Lab ® THEME EXAMPLE SIMULATION SimEnv Terrain Model Terrain Model SimMan Plot Model Plot Model Rover Dynamics Sensors Controller Block Diagram of Theme Example

29. A THEME EXAMPLE SIMULATION – Rover Dynamics dx/dt=vcos dy/dt=vcsin d/dt=w

30. A THEME EXAMPLE SIMULATION – Fuzzy Controller c) Forward velocityd) Angular velocity. d) Angular velocity. a ) Distance to objectiveb) Angle error b) Angle error Rule1: IF distance is ‘Near’, THEN forward velocity is ‘Slow’. Rule2: IF distance is ‘Far’, THEN forward velocity is ‘Speedy’. Rule3: IF angle is ‘Right’, THEN angular velocity is‘TurnRight’; forward velocity is ‘Slow’. etc.

31. Pushing task: Pushing region rArA rBrB f r =0 f r <0 f r >0 f l <0 f l =0 f l >0 A

32. Cooperative pushing task: rBrB Pushing regions

33. Cooperative Pushing Task “in”?command “out”!goal_reached “in”?object_reached “in”?object_lost “out”?object_touched “in”?everybody_ready “in”?outside_pushing_region “out”?object_reached_the_goal “in” “out”

34. Simulation Results

35. GA-DEVS Controller (1) x Goal (x g, y g ) l (x, y,θ) y ir 1 ir 3 ir 2 θgθg θ vlvl vrvr v w

36. GA-DEVS Controller (4) Goal 123 456 First simulation sample task using the GA control system, T =142 steps

37. GA-DEVS Controller (5) Second simulation sample task using the GA control system, T=236 steps Goal 1234 5678

38. GA-DEVS Controller (6) Trajectory of the robot in a task X YY X Trajectory of the robot for first taskTrajectory of the robot for second task

39. THEME EXAMPLE SIMULATION - Sensors Infra-red sensors. … Each sensor has a unique ID,X_s, Y_s and theta_s, where X_s is the X-position, Y_s is the Y-position and theta_s is the angle of each sensor relative to the rover in X-Y plane. GPS sensors … An atomic model simply simulates a position sensor, giving out the current position of the rover Compass Sensors … Respective atomic models simulate the sensing by converting the angle  of the rover from radians to degrees.

40. A THEME EXAMPLE SIMULATION – Distributed 2 Parts of a distributed simulation with GenDEVS and Sockets

41. Commercial ROVERS Two Pioneer II Mobile rovers are used to implement IDEVS-JAVA (V-Lab®). Movies will show demos at the end.

42. Experiment Results (DEVS-Fuzzy)

43. Experiment Results (GA-DEVS) Application: Pioneer 2 mobile robot Two experiments of path planning Goal searching task w/o obstacles. Robot (10, 9, -π/2)  (4, 4, -π/2)  returns to the home point (10, 9, -π/2) after traveling to the goal point. Robot task of goal seeking (Home -> Goal -> Home)

44. System of Systems - Introduction Changing Aerospace and Defense Industry Emphasis on “large-scale systems integration” Customers seeking solutions to problems, not asking for specific vehicles Mix of multiple systems capable of independent operation but interact with each other Emerging System of Systems Context

45. EMERGING CONTEXT: SYSTEM OF SYSTEMS Meeting a need or set of needs with a mix of independently operating systems New and existing aircraft, spacecraft, ground equipment, other independent systems System of Systems Examples Coast Guard Deepwater Program FAA Air Traffic Management Army Future Combat Systems Robotic Colonies

46. System of Systems SoS: A metasystem consisting of multiple autonomous embedded complex systems that can be diverse in: Technology Technology Context Context Operation Operation Geography Geography Conceptual frame Conceptual frame An airplane is not SoS, an airport is a SoS. A robot is not a SoS, but a robotic colony is a SoS Significant challenges: Determining the appropriate mix of independent systems The operation of a SoS occurs in an uncertain environment Interoperability Keating, et al., 2003

47. System of Systems Definitions SoS: No universally accepted definition 1. Operational & Management independence+Geographical Dist. + Emerging Behavior+Evolionary Dev. (ML, Space) 2. Integration+Inter- Operability.+Optmiz. to enhance battlefield scenarios (ML) 3. Large scale + distributed Systems Leading to more complex systems (Private Enterprize) 4. Within the context of warfighting systems – Inter Op.+Com ’ d. Synergy+Cont.+ Comp.+ Comm. +Info. (C4I) & Intel. (ML) Keating, et al., 2003

48. DISTINCTION BETWEEN SYSTEM ENGINEERING AND SoSE SoSE represents a necessary extension and evolution of traditional system engineering. Greatly expanded SoS requirements for tiered levels of discipline and rigor. Centralized control structure vs. de- centralized control structure A typical individual system (well defined end state, fixed budget, well defined schedule, technical baselines,homogeneous) A typical System of Systems (not well defined end state, periodic budget variations, heterogeneous)

49. Nature of SoSE Engineering Existing Complex Systems Exclusive, Autonomous, Local Transformation Keating, et al., 2003

50. System of Systems Integrated, Aligned, and Transforming System of Systems Interconnected, Integrated Mission, Global, Emergent Structure Keating, et al., 2003

51. System of Systems Engineering The design, deployment, operation, and transformation of metasystems that must function as an integrated complex system to produce desirable results. Keating, et. al 2003 Jamshidi, 2005

52. System of Systems PROBLEM THEMES 1. Fragmented Perspectives 2. Lack of Rigorous Development 3. Lack of Theoretical Grounding 4. IT Dominance 5. Limitations of trad. SE single system focus 6. Whole Systems Analysis Keating, et al., 2003

53. Modeling Issues Modeling of Systems of Systems? LSS ss … Traditional LSS Modeling LSS TOP BOT. TOP SoSE Modeling Difficulty SS-1 SS-2 SoSE LSS

54. EXAMPLES Air Traffic Control Personal Air Vehicles Future Combat Systems Internet Intelligent Transport Systems US Coast Guard Integrated Deepwater System, etc., etc.

55. US COAST GUARD INTEGRATED DEEPWATER SYSTEM The United States Coast Guard Protect the public, the environment, and U.S. economic and security interests in any maritime region International waters and America's coasts, ports, and inland waterways. Missions Maritime Security Maritime Safety Maritime Mobility National Defense Protection of Natural Resources

56. US COAST GUARD INTEGRATED DEEPWATER SYSTEM An integrated approach to upgrading existing assets while transitioning to newer, more capable platforms with improved systems for command, control, communications, computers, intelligence, surveillance, and reconnaissance and innovative logistics support. Ensure compatibility and interoperability of deepwater asstes, while providing high levels of operational effectiveness.

57. Conclusions V-Lab® is still at the beginning of its development Three sub-groups are working: Software: Responsible for the design and implementation of V-LAB® (from Network to V-LAB®) Hardware: Implementation of its tasks, features on industrial or commercial rovers Theory: Develop new theories to bridge the gap between Soft Computing (FL, NN, GA, GP, SLA, etc.) and DEVS environment. Future: Leverage V-Lab ® success and 35 years of research on LSS for SoSE applications is among future directions.

58. Cover, Modeling and Simulation Magazine, 2003

59. Publications – to date 1. J. Burge, A. El-Osery, M. Jamshidi and M. Fathi “V-LAB : A Virtual Laboratory for Distributed Robotic Modeling and Simulation,” Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Tucson, AZ, October, 2001. 2. A. El-Osery, J. Burge, M. Jamshidi and M. Fathi “Stochastic Learning Automaton for Learning Control of Robotic Agents, “Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Tucson, AZ, October, 2001. 3. A. El-Osery, J. Burge, A. Saha, M. Jamshidi, M. Fathi and M. Akbarzadeh-T., “V-Lab – A Distributed Simulation and Modeling Environment for Robotic Agents – Control Through Stochastic Learning Automata,” to appear in IEEE Transactions on Systems, Man and Cybernetics, Vol. 32, No. 6, pp. 795-804, December, 2002.

60. Publications – to date 2 4. A. El-Osery and M. Jamshidi, “Implementation of an SLA-Based Controller in a Virtual Laboratory,” Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Hammamet, Tunisia, October, 2002 5. M. Jamshidi, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Xia, Y. Wang, J. Liu, E. Tunstel, Jr, M. Akbarzadeh, A. El-Osery, M. Fathi, X. Hu, and B. P. Zeigler, “A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,” Chapter 11, Applied Simulation, Kluwer Publishers, Amsterdam, the Netherlands, 2003. 6. M. JAMSHIDI, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Beatty, S. Xia, Y. Wang, T. Song, U. Dole, J. Liu, E. Tunstel, Jr., M. Akbarzadeh, P. Lino, A. El-Osery, M. fathi, X. Hu, and B. P. Zeigler, “V-Lab© - A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,” Intelligent Automation and Soft Computing - Autosoft Journal, Vol. 9, No. 3, pp. 181-214 2003. 6. M. Jamshidi, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Xia, Y. Wang, J. Liu, E. Tunstel, Jr, M. Akbarzadeh, A. El-Osery, T. Song, M. Fathi,“A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,” Chapter 11, Modeling and Simulation Magazine, (Cover article), M&S International, Snn Diego, CA, 2003. 7. P. Sridhar and M. Jamshidi, “A Framework for Multi-agent Discrete Event Simulation: V-Lab®,” Proc. IEEE SMC, the Hague, the Netherlands, Oct 10-12, 2004.

61. Publications – to date 3 Discrete Event Fuzzy Logic Control with Application to Sensor-Based Intelligent Mobile Robot,” 8. S. Sheikh-Bahaei, M. Jamshidi,” Discrete Event Fuzzy Logic Control with Application to Sensor-Based Intelligent Mobile Robot,” Proc. WAC 2004, Seville, Spain, June 28-July 1, 2004. 9. P. Sridhar and M. Jamshidi, “A Multi-agent Discrete Event Simulator: V-Lab®,” Proc. WAC 2004. Seville, Spain, July 28-July 1, 2004. 10. S. Sheikh-Bahaei, M. Jamshidi, and P. Lino, “An Intelligent Discrete Event Approach to Modeling, Simulation and Control of Autonomous Agents,” Intelligent Automation and Soft Computing - Autosoft Journal, Vol. 10, No. 4, pp. 337-348, 2004.

62. THANK YOU!