2. Stochastic Model of a Micro Agents population Dejan Milutinovic dejan@isr.ist.utl.pt

3. Outline Motivating problem Introduction to Math. Analysis Mathematical Analysis Applications BiologyRobotics

4. T-Cell Receptor (TCR) triggering APC MHC peptide T-Cell CD3 T-Cell, CD3 receptor, Antigen Presenting Cell (APC), peptide-MHC complex Motivating problem

5. T-Cell population - T-Cell - APC Introduction to Math. Analysis

6. T-Cell population - T-Cell - APC Introduction to Math. Analysis Complex System !!!

7. How the Micro Dynamics of the Individuals propagates to the Dynamics of Macro observations ? Introduction to Math. Analysis

8. 1 –never connected, 2 - connected, 3- disconnected, a-connection, b-disconnection q=3 u(t)=a q=1 q=2 u(t)=a u(t)=b The Micro Agent model of the T-Cell Mathematical Analysis

9. Micro Agent (  A) Initial condition (x 0,q 0 ) Input event sequence AA u(t) Continuous output Y(t) Deterministic system u(t) Y(t) abc

10. Mathematical Analysis Stochastic Micro Agent (S  A) AA AA AA Stoch. process Determinist. system Stoch. process SASA Stochastic system

11. Mathematical Analysis Micro and Macro Dynamics relation PDF function describes the state probability of one  A Looking to the large population of  A, PDF is a normalized distribution of the state occupancy by all  A Micro dynamics of  A and macro dynamics of  A population are related through the state PDFs Dual Meaning of the State Probability Density Function Statistical Physics reasoning (Boltzman distribution)

12. A stochastic process (x(t),q(t))  X  Q is called a Micro Agent Stochastic Execution iff a Micro Agent stochastic input event sequence e(  n ),n  N,  0 = 0   1   2  … generates transitions such that in each interval [  n,  n+1 ), n  N, q(t)  q(  n ). Micro Agent Stochastic Execution Remark 1. The x(t) of a Stochastic Execution is a continuous time function since the transition changes only the discrete state of a Micro Agent. Mathematical Analysis f(x,N) 1 i N V V V x1x1 xnxn x n-1... f(x,1) e(  n ) f(x,i) q e(  n+1 ) e(  n+2 ) X x Q

13. A Stochastic Micro Agent is a pair S  A=(  A,e(t)) where  A is a Micro Agent and e(t) is a Micro Agent stochastic input event sequence such that the stochastic process (x(t),q(t))  X  Q is a Micro Agent Stochastic Execution. Stochastic Micro Agent (S  A) Mathematical Analysis AA AA AA Stoch. process Determinist. system Stoch. process SASA Stochastic system

14. A Stochastic Micro Agent is called a Continuous Time Markov Process Micro Agent iff (x(t),q(t))  X  Q is a Micro Agent Continuous Time Markov Process Execution. Mathematical Analysis Continuous Time Markov Process Micro Agent (CTMP  A) SASA q=3 u(t)=a q=1 q=2 u(t)=a u(t)=b

15. Mathematical Analysis The Continuous Time Markov Chain Micro Agent with N discrete state and state probability given by where state i and is the probability of discrete is transition rate matrixand is rate of transition from discrete state i to discrete state j. The vector of probability density functions whereis probability density functionof state (x,i) at time t, satisfies is the vector of vector fields value at state (x,i).where

16. Biological application The Micro Agent model of the T-Cell u(t)=a q=1 q=2 q=3 u(t)=a u(t)=b 12 23 32 b-disconnection, ij – event rate which leads to transition from state i to state j 0 –never connected, 1 - connected, 2- disconnected, a-connection,

17. Biological application Case I solution 12 =0.9, 23 = 0, 32 =0.5, k 2 =0.5, k 3 =0.25

18. Biological application Case II solution 12 =0.9, 23 = 0.8, 32 =0.9, k 2 =0.5, k 3 =0.05

19. Biological application Case III solution 12 =0.9, 23 = 0.8, 32 =0.9, k 2 =0.5, k 3 =0.25

20. Biological application

23. Robotics application Source 1 Source 3 Source 2 x 2 x 1 x 2 x 1  =  /4  =-  /4  =0 Population a) b) q=3 23 21 32 12 q=1 q=2

24. Robotics application

25. Milutinovic, D., Athans, M., Lima, P., Carneiro, J. “Application of Nonlinear Estimation Theory in T-Cell Receptor Triggering Model Identification”, Technical Report RT-401-02, RT-701-02, 2002, ISR/IST Lisbon, Portugal Milutinovic D., Lima, P., Athans, M. “Biologically Inspired Stochastic Hybrid Control of Multi-Robot Systems”, submitted to the 11th International Conference on Advanced Robotics ICAR 2003,June 30 - July 3, 2003 University of Coimbra, Portugal Milutinovic D., Carneiro J., Athans, M., Lima, P. “A Hybrid Automata Modell of TCR Triggering Dynamics”, submitted to the 11th Mediterranian Conference on Control and Automation MED 2003,June 18 - 20, 2003, Rhodes, Greece Milutinovic, D., “Stochastic Model of a Micro Agents Population”, Technical Report ISR/IST Lisbon, Portugal (working version) Publications

26. Stochastic Model of a Micro Agents population Dejan Milutinovic dejan@isr.ist.utl.pt