1. Bio-Cybernetic Control for WBI: a bird’s eye view Prof. A
Bio-Cybernetic Control for WBI: a bird’s eye view Prof. A. Taleb-Bendiab School of Computing Liverpool John Moores University
Bio-cybernetic Control of Whole Body Interaction
In the context of the Whole Body Interaction (WBI), this talk will focus on the technical challenges, requirements and conceptual frameworks for bio-cybernetics control.
The talk will start with a brief overview of emerging requirements for WBI, which will be followed by a review of the state-of-the-art relevant to bio-cybernetics control for WBI including: ambient intelligence, body area sensor and actuator networks, cognitive systems and middleware. This will be followed by a presentation of the underpinning technologies, and a detailed description of a conceptual framework for bio-cybernetics control. Real-life examples will be used to illustrate the contents of this presentation. The talk will conclude with a discussion on remaining research challenges relevant to WBI.
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction Workshop’07, Date: 01/04/2017, Slide: 1

2. Outline In the context of the Whole Body Interaction (WBI)
Definitions
Biocybernetics, control, Augmented Reality, WBI
Drivers for WBI
Overview of emerging requirements for WBI,
Technical challenges, requirements
State-of-the-art relevant to bio-cybernetics control for WBI including:
Bio and socio-inspired computational models
ambient intelligence, Autonomic Computing
Middleware
body area sensor and actuator networks,
cognitive systems
Underpinning technologies,
Conceptual frameworks for bio-cybernetics control
Conclusions
Open questions research challenges relevant to WBI
Q&A
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 2

3. Some Definitions #1 Biological Cybernetics (Bio-Cybernetics)
… is the application of cybernetics theory and systemics principles to further our understanding of systems biology
integrate different levels of information and information processing to understand biological systems
function and development of cellular, multicellular systems, organisms, complex living systems and ecosystems. [
[Source: Wikipedia]
Cybernetics
the study of control and communication in the animal and the machine including:
organization, information flows, control
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 3

4. Some Definitions #2 Augmented Reality:
…. combination of real world and computer generated data
uses of live video imagery which is digitally processed and "augmented" by
the addition of computer generated graphics.
Advanced research includes the use of motion tracking data, etc..,
and the construction of controlled environments containing any number of sensors and actuators. [
Mixed reality:
refers to the merging of real and virtual worlds to produce new environments and visualisations where physical and digital objects co-exist and interact in real time. [
Virtual Worlds:
“… simulated environment intended for its users to inhabit and interact via avatars …” [
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 4

5. Socially Augmented Intelligent PICT
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 5

6. Convergence of Real and Virtual Worlds: Second Life
hand-held and wearable systems that act as gateways between the real and virtual worlds
Eyewear, display badges and speakers worn about the neck will allow us to live more fully through our avatars
a wearable box that creates a 3D sound field that allows the wearer to hear voices from the virtual world without completely shutting out the real people around him
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 6

7. Vodafone in Second Life
More recently Vodafone now allows you to interact with your real phone via a Second Life avatar of the phone
The real phone can be answered from within SL
Text messages can be sent and received between real-world and SL
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 7

8. Sensors that Control Second Life Objects
Sending external data into virtual worlds such as Second Life
Using Potentiometer the cube can be controlled
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 8

9. Importance of WBI WBI Technical Requirements Interface
Transparency and virtualisation
Systems and Processes
Interaction
Behaviour
Structure/organisation
Affective
Sensing and Actuation
Co-evolve
Technical Requirements
Protocols for WBI
Standards and legacy systems
Programming, Interaction and Control Models
Usability
Assurance
QoS
Safety and Security, Etc.
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 9

10. Example: Bio-Cybernetics Control
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 10

11. Virtual Lab
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 11

12. IBM Sensors Link Real and Virtual
Sensor data is visualised in real-time
Blue balls with white designs represent active Bluetooth devices
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 12

13. The Story so Far #1 State-of-the-art in Situated AmI including;
autonomic software models and architecture, standards
tools and techniques to support
the design, modelling, analysis
and evolution of autonomic software
Define models for their
programming, control
interaction models with human and/or other non-AC systems (legacy).
Delegation of authority and its adjustment
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 13

14. The Story so Far #2 Has been informed by a set of design paradigms
Model-based vs self-organising systems design models
Top-down vs bottom-up
Applying and/or revisiting:
cybernetic principles
control systems theory, regulation, reward and sanctions
Decision theory, DAI and CI
dynamic planning, deliberative models, ML
Middleware support
self-awareness, reflection and deliberation
etc.
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 14

15. Research Challenges #1 Social and other requirements
Conceptual Design and Management models
“… how do we design, build, and evolve such systems so that they can meet given—and evolving—requirements ...”
Incremental deployment of SAmI capabilities in legacy systems.
Hot-swapping, Dynamic AOP-based evolution, Interoperation
Programming, control and Interaction Models
Usability and Interface
Balancing and adjusting governance and autonomy
Social and other requirements
Not covered here
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 15

16. Cybernetics: The Viable System Model
Beer’s VSM implements a control & communication structure via hierarchies of homeostats (feedback loops) (1950)
Defines 6 major systems ensure ‘viability’ of the system
Implementation S1
Monitoring S2
Audit S3*
Control S3
Intelligence S4
Policy S5
Offers an extensible, recursive, model-based architecture, devolving autonomy to sub-systems
Autonomic
Systems
Cognitive
Systems
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 16

17. Situated AmI Capabilities
A software system is autonomic, if it possesses the following capabilities:
Self-configuring — choosing a suitable behaviour, based on user preferences, context, …
Self-tuning — choosing behaviours that optimize certain qualities (performance, year-end profits, …)
Self-repairing — shifting execution to another behaviour, given that the current one is failing
Self-protecting — choosing a behaviour that minimizes risks (attacks, viruses, …)
Self-knowledge - Detailed knowledge of constituent components, current status, etc.
Self-configuration/re-configuration - Adjustments to a changing environment
Self-optimizing - Monitor constituent parts and optimize accordingly
Self-healing - The ability to recover from malfunction
Self-protecting - Detect, identify and protect itself from attack
Environmentally aware – know its environment, the context surrounding its activity and act accordingly
Co-operative – Interact with other systems in a heterogeneous world – open standards
Anticipatory – anticipate and transparently implement the resources required to meet user goals
Self configuring: Automated configuration of components and systems that follow high-level policies. The rest of the system adjusts automatically and seamlessly.
Self-healing: The System automatically detects, diagnoses, and repairs localized software and hardware problems. Sometimes also reintegrating the repaired resource back into itself.
Self-optimization: Components and systems continually seek to improve their own performance and efficiency.
Self-protection: The system automatically defends against malicious attacks and cascading failures. It anticipates and prevents system-wide failures.
Self monitoring: The system continually measures its current state to be compared with the desired state.
Environment aware: The system periodically monitors and measures its related environment to have a suitable action against threats or different circumstances.
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 17

18. Our Approaches #1 Related Work: Our model – J Reference
IBM blueprint (www-03.ibm.com/autonomic/pdfs/ACBP2_ pdf)
An autonomic manager contains a continuous control loop that monitors activities and takes actions to adjust the system to meet business objectives
Autonomic managers learn from past experience to build action plans
Elements need to be instrumented consistently, based on open standards
Our model – J Reference
Gauges for dynamic assembly and adaptation
Requirements / Event-based architectural models
(Before)
Run-time performance adaptation (After)
Model -- Implementation mapping / Transformation
(During)
Ontology-based gauges
Run-time Configuration
Open Points
Connectivity
Constraint Gauges
Reconfiguration Cost
Early Warning
Dependability (for Security)
Runtime Event Monitoring
Architecture Transformation Technology
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 18
Evolution and Integration Command Center
Performance-based Architectural Adaptation
Architecture-driven Dynamism

19. Projects in the Networked Appliances Lab.
Composition of an Internet of Things
Infinitely Adaptable Gaming
Bridging between Physical and Virtual (Demo)
Managing 3D Multimedia Streams
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 19

20. What It Looks Like
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 20

21. Prototype Implementation
Paul Fergus, David Llewellyn-Jones, Amjad Shaheed, Madjid Merabti, Abdennour El Rhalibi, Networked Appliances: Manipulation of 3D Multimedia Streams, Accepted for publication in the 5th IEEE International Consumer Communications and Networking Conference: Workshop on Networking Issues in Multimedia Entertainment, 2008, Las Vegas, USA
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 21

22. Augmenting Frames
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 22

23. Modelling Autonomy Our model Evolving and Adjustable Autonomy
Gauges for dynamic assembly and adaptation
Our model
Using SSC used to formalizes the behaviour of dynamically changing systems FOL (McCarthy, 1963)..
Support concurrent actions and timing constraints.
Each situation can be viewed as a history of previous actions.
Action, guards and time can be modelled at deliberation points in an autonomic setting.
M. Randles, A. Taleb-Bendiab, Philip Miseldine, Andy Laws, "Adjustable Deliberation of Self-Managing Systems", ECBS 2005: [ppt]
Evolving and Adjustable Autonomy
Software governance (Control) via
Formal modelling of norms, policies
Enactment support – from spec. to code using Neptune language
Requirements / Event-based architectural models
(Before)
Run-time performance adaptation (After)
Model -- Implementation mapping / Transformation
(During)
Ontology-based gauges
Run-time Configuration
Open Points
Connectivity
Constraint Gauges
Reconfiguration Cost
Early Warning
Dependability (for Security)
Runtime Event Monitoring
Architecture Transformation Technology
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 23
Evolution and Integration Command Center
Performance-based Architectural Adaptation
Architecture-driven Dynamism

24. Programming Autonomic Systems
Neptune Meta-Language
Integrated Development Environment:
Miseldine, P., Taleb-Bendiab A. A Programmatic Approach to Applying Sympathetic and Parasympathetic Autonomic Systems to Software Design. to appear in the 2005 International Conference on Self-Organization and Adaptation of Multi-agent and Grid Systems (SOAS’2005).
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 24

25. Conclusions and Future Work
Understanding WBI
Interface
Transparency and virtualisation
Systems and Processes
Interaction
Behaviour
Structure/organisation
Affective
Sensing and Actuation
Co-evolve
Address the challenges including:
Protocols for WBI
Standards and legacy systems
Programming, Interaction and Control Models
Usability
Assurance
QoS
Safety and Security, Etc.
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 25

26. Acknowledgements Acknowledgements My thanks to All CMS Research Team 
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 26

27. That’s the end – so I’m off !
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 27

28. Extra Slides …
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 01/04/2017, Slide: 28