Second GSFC/IEEE Workshop on Radical Agent Concepts Genetically Modified Software: Realizing Viable Autonomic Agency A.G. Laws, A. Taleb-Bendiab & S.J. Wade Liverpool John Moores University, UK

Introduction & Overview Our search for a unifying conceptual framework and design model for the development of autonomic computing systems Management Models for Self-Managing Systems –Managerial Cybernetics and the Viable System Model A Cybernetic Reference Model for Autonomic Systems –and some drawbacks Some classical cybernetics and their implications More speculatively –Some small scale experiments in adaptation with genetic algorithms –Lessons, limitations and possibilities –Dynamic Model Development for Open Environments using LCS Conclusions

The Viable System Model Underpinned by “classical” cybernetics though structured using the human CNS as a model Beer’s VSM implements a control & communication structure via hierarchies of (homeostatic) feedback loops 6 major systems ensure ‘viability’ of the system – ImplementationS1 – MonitoringS2 – AuditS3* – ControlS3 – Intelligence S4 – Policy S5 Offers an extensible, recursive, model-based architecture, devolving autonomy to sub-systems Autonomic Systems Anticipatory Self-awareness Deliberative

Where this begins to get interesting The recursive nature of the underlying model suggests that each System One should develop as a Viable System Model in its own right. If the recursion is pursued below the level of the human software team then we arrive at the level of the software system. This strongly implies that the software system should assume the same overall management structure as the human software process Consequently, we appear to have a theoretically supported management architecture both for the human process and for “self-managing autonomic software” This also fits closely with IBM’s “dynamic management by business rules/policies” aim for autonomic systems

Bratman et al’s IRMA Architecture Bratman et al’s Intelligent Resource-Bounded Machine Architecture (IRMA), a classic BDI agent model It is noticeable that many of the elements contained in the model correlate directly to elements of the VSM So – Planning & Means-Ends Reasoning S3 –Opportunity Analyzer & Filtering S4 –Beliefs about the world S5 –Desires and Deliberative Process S5 –Intentions - S5 Policy passed to S3 for enactment It is also apparent that some elements are missing for Viability

A Viable Intelligent Agent Architecture? Using Bratman et al’s IRMA approach and using the VSM as a reference model, we designed a Viable Intelligent Agent Architecture realizable using a multi-agent approach. The design specifies, in some detail, the higher, deliberative processes of adaptation. The embedding of this architecture in a hierarchy of VSM’s keeps the “Man in the Loop” However, the VSM uses human agents to realize its systems. These provide implicit (unmapped) elements of creativity/learning to the organization.

Ashbean Cybernetics The Homeostat - ultra-stable system capable of returning to stability after it has been disturbed in a way not envisaged by the designer. Self-vetoing homeostasis The Law of Requisite Variety effective control can only be attained if the repertoire of responses that the controller has at its disposal at least matches the disturbances the situation to be controlled can display. More recently, this has been supplemented by the Law of Requisite Knowledge, which states that the system must know i.e. learn, the most appropriate response to use in a given circumstance – i.e. optimized response Taken together these two laws underpin the Conant-Ashby theorem, namely “Every good regulator of a system must be (contain) a model of that system”.

A (simple) Scenario to address the Law of Requisite Variety Ashby’s Law states that a system must “derive” a repertoire of actions that can be deployed according to environmental circumstances Developed a “toy” system where the environment simply delivers arrays of integers for sorting The system is provide with a range of responses from which it tries to select the “optimal” solution using timings/statistical analysis Advantages – Fully controllable environment Finite (n = 8 = permutations) Smooth/turbulent as you like Measure the degree “sortedness” of test cases easily However, more interesting is how the system acquires new responses to address new environmental circumstances.

Hillis’s Sorting Networks Hillis had used a “Connection Machine” coupled with an approach using Holland’s Genetic Algorithms to “evolve” sorting networks for n = 16 arrays Although no Connection Machine was available, we decided to try with n = 8 arrays Here, for n = 8, a sorting network is represented by 5 pairs of 12 bit chromosomes Each 3 bits represents an array index (0 – 7) Each 12 bit chromosome holds 4 array indices and hence 2 comparisons E.g. Compare Index 1 with Index 2 and swap if necessary Chromosomes are paired so that duplicate entries do not appear in the phenotype. This gives a minimum of 10 comparisons and a maximum of 20 for the sorting network. Hence more/less “optimal” solutions?

Breeding GA’s Used a population of 200 networks organized on a two- dimensional array and randomly filled with bits. Testing each against our engineered environmental test arrays and measuring reduction in “unsortedness” allowed the identification of “fitter” individuals and hence the new breeding population. Select two fit parents, using random crossover, each parent contributes 5 gametes to make a new 10 chromosome member in the new population. The new population is then subjected to random mutation (bit-flipping) using the standard probability rate of 0.001

Emergence of Full Fitness Individuals Somewhat surprisingly, fully-fit individuals appeared in the population very quickly. Here, a sorting network capable of correctly sorting the 100 test cases emerged after 11 generations (i.e. in minutes) Admittedly, this was using a “gentle” environment –Inv. 0 “ ” –Inv. 9 “ ” Interestingly, more than one “type” of fully-fit individuals emerged – efficiency ? Unfortunately, less success in highly disordered environments –Inv. 19 “ ” –Inv. 28 “ ” = 97% Of course, we’re not suggesting more generalized use, but perhaps “genetic programming?”

Learning Classifier Systems & The Law of Requisite Knowledge So, assuming our system can develop a repertoire of responses tailored to specific environmental circumstances, we still need a means to achieve Requisite Knowledge One attractive candidate is Holland’s Learning Classifier Systems approach Here the system constructs “creates” candidate production rules in the form: if then i.e. situation/response Rules compete (bid) to participate in a response to an environmental condition. The “solution” provided by the rules is tested in the environment and a “payoff” determined. This is distributed amongst contributing rules and either strengthens or weakens rules depending on success/failure A GA is used to generate new rules using provably fit rules. What emerges is an adaptive, learning “model” of the environment.

Summary So, we have looked at the VSM and what it has to contribute –It clearly defines the types of systems that must exist for viability –Lead to a recursive architecture that appears to be able to incorporate both human systems and autonomic software systems –Lends itself to the future development of a design pattern/grammar for autonomic system development We have noted some of the difficulties encountered by extending the model from human to artificial systems Model completeness in open environments Although we didn’t refer to it today we need to define communications protocols Etc. We briefly examined some of the cybernetic underpinnings for clues to a way forward Presented some very small scale examples of the use of genetic algorithms to generate a repertoire of responses and speculated on the use of other perhaps more generalizable genetic approaches e.g. artifical immune systems Speculated (even more briefly) on the potential use of Learning Classifier Systems as an approach to address the environmental modelling problems apparent in open systems Thanks for your attention & Any Comments or Questions would be welcome?