1. SOCIAL CONTINGENCY Dr. Tom Froese IIMAS-UNAM

2. An integrative methodology Froese and Fuchs (2012)

3. Defining social interaction “We propose the following definition of social interaction: Social interaction is the regulated coupling between at least two autonomous agents, where the regulation is aimed at aspects of the coupling itself so that it constitutes an emergent autonomous organization in the domain of relational dynamics, without destroying in the process the autonomy of the agents involved (though the latter’s scope can be augmented or reduced).” De Jaegher and Di Paolo (2007, p. 493)

4. Intra-bodily resonance “Let us assume that A is a person whose emotion, e.g. anger, manifests itself in typical bodily (facial, gestural, interoceptive, adrenergic, circulatory, etc.) changes. His pre-reflectively experienced ‘lived’ body thus functions as a felt ‘resonance board’ for the emotion: A feels the anger as the tension in his face, as the sharpness of his voice, the arousal in his body etc. These proprio- and interoceptive bodily feelings may be termed intra-bodily resonance.” Froese and Fuchs (2012, p. 212)

5. Intra-bodily resonance “However, this resonance is an expression of the emotion at the same time, that means, the anger becomes visible and is perceived as such by A’s partner B. But what is more, the expression will also produce an impression, namely by triggering corresponding or complementary bodily feelings in B. Thus, A’s sinister gaze, the sharpness of his voice or expansive bodily movements might induce in B an unpleasant tension or even a jerk, a tendency to withdraw, etc. Thus, B not only sees the anger immediately in A’s face and gesture, but also senses it with his own body, through his own intra-bodily resonance.” Froese and Fuchs (2012, p. 212)

6. Inter-bodily resonance “However, it does not stay like this, for the impression and bodily reaction caused in B in turn becomes an expression for A; it will immediately affect his bodily reaction, change his expression, however slightly, and so forth. This creates a circular interplay of expressions and reactions running in split seconds and constantly modifying each partner’s bodily state, in a process that becomes highly autonomous and is not directly controlled by the partners. They have become parts of a dynamic sensorimotor and inter- affective system that connects both bodies by reciprocal movements and reactions, that means, in inter-bodily resonance.” Froese and Fuchs (2012, p. 213)

7. Intra- and inter-bodily resonance Froese and Fuchs (2012)

8. Inter-bodily memory “This account of inter-bodily resonance may be complemented by the historical dimension. In embodied agents, the history of interactions continuously changes their dispositions. From early childhood on, patterns of interaction are sedimented in the infant’s implicit or bodily memory, resulting in what may be called inter-bodily or ‘intercorporeal memory’ (Fuchs 2008; Fuchs and De Jaegher 2009; Fuchs 2012). This means a pre-reflective, practical knowledge of how to interact with others — e.g. how to share pleasure, elicit attention, avoid rejection, re-establish contact, etc. It may also be termed “implicit relational knowledge” (Lyons-Ruth et al. 1998). As such, intercorporeal memory enables the basic formation of dyadic and more generally intersubjective patterns of interaction.” Froese and Fuchs (2012, p. 213)

9. Experiments in social contingency “One promising target for such an endeavor is Murray and Trevarthen’s (1985) double TV monitor experiment. In this psychological study 2 month old infants were animated by their mothers to engage in coordination via a live double video link. However, when the live video of the mother was replaced with a video playback of her actions recorded previously, the infants became distressed or removed. These results, and those of a more rigorous follow-up study by Nadel and colleagues (1999), indicate that 2 month old infants are sensitive to social contingency, i.e. the mutual responsiveness during an interaction, and that this sensitivity plays a role in the unfolding of coordination.” Froese (2009, p. 128)

10. Double TV monitor paradigm Murray and Trevarthen (1985)

11. Double TV monitor paradigm revisited Nadel et al. (1999)

13. Double TV monitor paradigm Nadel et al. (1999)

14. How to explain the findings? “Traditional explanations of this sensitivity have focused on inborn factors. For example, Gergely and Watson (1999) have postulated the presence of an innate cognitive module which enables the detection of social contingency, and Russell (1996) hypothesizes that infants have a native capacity to understand intentionality and to process agency. Are these postulations of innate capacities on the part of the infant necessary in order to explain the empirical results?” Froese (2009, p. 129)

15. Evo. robotics as a scientific method Froese (2009)

16. Double TV monitor paradigm Froese (2009)

17. Minimalist agent-based model v1.0 Iizuka and Di Paolo (2007)

18. Fitness evaluation function “The fitness is calculated on the basis of two factors. One is how many times the agent can cross its central position with that of a live interacting agent (live interaction). The other is how much the agent can stay away from a dummy agent which only replays the motions of their partner as recorded in the first stage (one-way interaction). It should be clear that both conditions start from the same initial configurations such as positions, speeds, and neural states and then, at a certain point, the top agent is replaced with the recorded motions. It means that the bottom agent has a exactly same states as those being taken in the live condition when the top being replaced.” Iizuka and Di Paolo (2007, p.4)

19. ‘Live’ versus ‘replay’ interaction Iizuka and Di Paolo (2007)

20. Evolutionary robotics Di Paolo, Rohde and Iizuka (2008)

21. Detection of social contingency Iizuka and Di Paolo (2007)

22. Differences in CTRNN dynamics Iizuka and Di Paolo (2007)

23. Generalization of findings Iizuka and Di Paolo (2007) specifically evolved one of the two agents to be sensitive to the responsiveness of the other agent (detection of social contingency). Maximize number of crossings during live condition, minimize number of crossings during playback condition. In other words, even though the agent’s sensitivity depends on properties of the interaction process, it may still depend on specific innate capacities. Could we find the same behavioral sensitivity even if we do not explicitly evolve agents to be sensitive?

24. Minimalist agent-based model v2.0 Froese and Fuchs (2012)

25. Continuous-time recurrent neural network The artificial ‘brains’ of the agents was implemented based on the standard continuous-time recurrent neural network equations: But each agent’s neural network only used 3 rather than 8 CTRNN nodes. And the fitness evaluation only rewarded successful interactive coordination of movements by both agents (goal: maximize joint distance from starting point). No sensitivity to social contingency was rewarded.

26. Live interaction condition Froese and Fuchs (2012)

27. ‘Playback’ interaction condition Froese and Fuchs (2012)

28. Robustness to ‘motor’ noise Froese and Fuchs (2012)

29. Robustness to ‘sensor’ noise Froese and Di Paolo (2008)

30. Coordination in joint finger tapping “To study the mechanisms of coordination that are fundamental to successful interactions we carried out a joint finger tapping experiment in which pairs of participants were asked to maintain a given beat while synchronizing to an auditory signal coming from the other person or the computer. When both were hearing each other, the pair became a coupled, mutually and continuously adaptive unit of two “hyper-followers”, with their intertap intervals (ITIs) oscillating in opposite directions on a tap-to-tap basis. There was thus no evidence for the emergence of a leader–follower strategy. We also found that dyads were equally good at synchronizing with the irregular, but responsive other as with the predictable, unresponsive computer. However, they performed worse when the “other” was both irregular and unresponsive. We thus propose that interpersonal coordination is facilitated by the mutual abilities to (a) predict the other’s subsequent action and (b) adapt accordingly on a millisecond timescale.” Konvalinka et al. (2010)

31. Evolved CTRNN structure Froese (2009)

32. Autonomous dynamics of CTRNN Froese and Fuchs (2012)

33. Coupled ‘live’ CTRNN dynamics Top: Froese and Fuchs (2012)

34. Coupled ‘playback’ CTRNN dynamics Bottom: Froese and Fuchs (2012)

35. Interactively enabled transients “What Fig. 8 reveals is that the robustness of the agents’ behavior, as demonstrated by their resilience to external perturbations (Fig. 5), depends on their internal state following a transient pattern within a far-from-equilibrium region of state space. In this transient region the agent’s internal flow structure effectively operates as a stable quasi-periodic equilibrium, rather than as a fixed-point attractor, which in this case has the desirable effect of expanding the agent’s behavioral repertoire. Instead of being effectively limited to a single direction of movement, the agent can now move both left and right in a flexible manner due to the interactively stabilized transient region.” Froese and Fuchs (2012, p. 231)

36. Proof of concept of an extended body “Importantly, while this transient region enables the kind of flexible behavior that is required for sustaining a responsive interaction, it is also the case that what enables an agent’s internal state to first enter into (and then to remain within) this region is precisely the responsive behavior between the agents. The agents must guide each other’s internal state into this transient region by switching each other’s internal flow structure in an appropriate manner. This finding confirms that we are indeed dealing with a model of an extended body: each of the agent’s intra-bodily dynamics is extended by the other agent’s intra-bodily dynamics by means of the inter-bodily dynamics of their interaction process.” Froese and Fuchs (2012, p. 231)

37. Further experimental variations “we introduced some additional constraints into the artificial evolution of the controllers. The idea was to induce local (structural) instability into the CTRNNs by explicitly requiring that all nodes have excitatory self-connections, while connections between nodes must be inhibitory. The weights of all self-connections (wii) have an identical magnitude. Other synaptic strengths (wij) are constrained as follows: w12 = w23 = w31 = β1 < 0 and w21 = w32 = w13 = β2 < 0, where β1, β2, and wii are parameters under evolution. Such local instability might facilitate the emergence of behavioral robustness.” Fernandez-Leon and Froese (2010, p. 4646)

38. Enhanced robustness Fernandez-Leon and Froese (2010)

39. Enhanced robustness Fernandez-Leon and Froese (2010)

40. Enhanced robustness Fernandez-Leon and Froese (2010)

41. Conclusions In general, sensitivity to social contingency does not have to be reducible to internal ‘contingency detection’ neural systems, because it can also depend on the stability of the interaction process itself (Di Paolo et al. 2008). Loss of stability due to loss of responsiveness during interactive coordination comes for free, unless explicitly counteracted by mechanisms of robustness, so it should be the preferred explanation unless it is demonstrated that the behavior is reducible to internal neural modules. Avoidance of humoncular explanations does not deny the importance of the subjective perspective, but it situates it where it belongs: at the personal, phenomenological level. In fact, an explanation based on a truly interactive, extended body fits better with the phenomenology.

42. Conclusions “The communication or comprehension of gestures comes about through the reciprocity of my intentions and the gestures of others, of my gestures and the intentions discernible in the conduct of other people. It is as if the other person’s intentions inhabited my body and mine his.” (Merleau-Ponty [1945] 1962: 185; emphasis added). “In sum, our perceptions [of other agents] arouse in us a reorganization of motor conduct, without our already having learned the gestures in question.” (Merleau-Ponty [1960] 1964: 145; emphasis added). Embodied interaction creates a “mixture of myself and the other” (Merleau-Ponty [1960] 1964: 155). Quoted from Froese and Fuchs (2012, p. 214)

43. Conclusions “Husserl said that the perception of others is like a ‘phenomenon of coupling.’ The term is anything but a metaphor. In perceiving the other, my body and the other’s body are coupled, resulting in a sort of action which pairs them. This conduct which I am able to only see, I live somehow from a distance. I make it mine; I take it up or understand it. […] there is initially a state of pre-communication (Max Scheler), wherein the other’s intentions somehow play across my body while my intentions play across his.” Merleau-Ponty ([1960] 1964, p. 148)

44. Homework No class this Thursday, March 26. Finish reading previous homework. Read this for next class (after holy week): Froese, T., Gershenson, C., & Rosenblueth, D. A. (2013). The dynamically extended mind: A minimal modeling case study. 2013 IEEE Congress on Evolutionary Computation (pp. 1419-1426): IEEE Press

45. References Di Paolo, E. A., Rohde, M., & Iizuka, H. (2008). Sensitivity to social contingency or stability of interaction? Modelling the dynamics of perceptual crossing. New Ideas in Psychology, 26(2), 278-294 Fernandez-Leon, J. A., & Froese, T. (2010). What is the relationship between behavioral robustness and distributed mechanisms of cognitive behavior? 2010 IEEE Congress on Evolutionary Computation (pp. 4645-4652). IEEE Press Froese, T. (2009). Sociality and the Life-Mind Continuity Thesis: A Study in Evolutionary Robotics. D.Phil. Dissertation, University of Sussex, Brighton Froese, T., & Di Paolo, E. A. (2008). Stability of coordination requires mutuality of interaction in a model of embodied agents. In M. Asada, J. C. T. Hallam, J.-A. Meyer & J. Tani (Eds.), From Animals to Animats 10: 10th International Conference on Simulation of Adaptive Behavior, SAB 2008 (pp. 52-61). Berlin, Germany: Springer Verlag Froese, T., & Fuchs, T. (2012). The extended body: A case study in the neurophenomenology of social interaction. Phenomenology and the Cognitive Sciences, 11(2), 205-235 Iizuka, H., & Di Paolo, E. A. (2007). Minimal agency detection of embodied agents. In F. Almeida e Costa, L. M. Rocha, E. Costa, I. Harvey & A. Coutinho (Eds.), Advances in Artificial Life: 9th European Conference, ECAL 2007 (pp. 485-494). Berlin, Germany: Springer-Verlag Konvalinka, I., Vuust, P., Roepstorff, A., & Frith, C. D. (2010). Follow you, follow me: Continuous mutual prediction and adaptation in joint tapping. The Quarterly Journal of Experimental Psychology, 63(11), 2220- 2230 Merleau-Ponty, M. ([1960] 1964). The child's relations with others. In J. M. Edie (Ed.), The Primacy of Perception (pp. 96-158). Evanston, IL: Northwestern University Press Murray, L., & Trevarthen, C. (1985). Emotional regulations of interactions between two-month-olds and their mothers. In T. M. Field & N. A. Fox (Eds.), Social Perception in Infants (pp. 177-197). Norwood, NJ: Ablex Publishing Nadel, J., Carchon, I., Kervella, C., Marcelli, D., & Réserbat-Plantey, D. (1999). Expectancies for social contingency in 2-month-olds. Developmental Science, 2(2), 164-173