1. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-1 Artificial Science – a simulation test-bed for studying the social processes of science Bruce Edmonds Centre for Policy Modelling Manchester Metropolitan University

2. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-2 Part 1: Introduction Science as an object of study

3. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-3 Why study science as a process? Its important impacts on society Improve the “management” of science Understand its strengths and weaknesses It is interesting Science seems to achieve things that have been found difficult in DAI/MAS, e.g.: –building complex value-added chains –spontaneous autopoesis and self-organisation of fields and sub-fields –spontaneous specialisation and distribution of skills across the problem space –reliability of established results in comparison to the uncertain reliability of individuals' work

4. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-4 Philosophy/Sociology of Science Normative philosophy of science (basically until 1960s and Kuhn) Then a move towards a more descriptive approach Tends to focus on what an individual scientists do Social processes tend to be introduced either as: –Part of a critique of the received picture etc. –To support special status of science (e.g. its norms) Now some more study/discussion of observed social processes in science (Giere, Longino, Knorr-Cetina, etc.)

5. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-5 Computational models of science Aspects of scientific thinking: –Problem-solving by sub-casing (Newell&Simon) –Induction (Holland &al) –Inference of causality (Pearl) More integrated models: –BACON (Langley &al) –Meta-Dendral (Buchan&Mitchel) –PI (Thagard) Social aspects of science: –Gilbert (1997) A simulation of the structure of academic science

6. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-6 Part 2: A Test-Bed How can we simulate the innovative discovery of the unknown?

7. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-7 Aims of test-bed To explore the relationship between the (social) behaviour of individuals and the knowledge that is collectively produced In particular: –What behaviours seem to reproduce observed patterns of discovered knowledge? –What are the effects of certain behaviours on the structure of discovered knowledge? –What behaviours might be better than others, wrt target properties of discovered knowledge?

8. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-8 Keys design issues The test-bed must… be a sufficiently challenging problem requiring much independent (i.e. parallel) work by individual agents but also requiring the combination of knowledge produced by others in chains involving the discovery of the unknown where some knowledge may be used to transform other knowledge.

9. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-9 Structure of the test-bed A formal (i.e. logical or mathematical) system stands in for what is true Existing knowledge is represented by the theorems that are in the agents’ individual memories or as published in journals etc. Unknown to the agents certain theories are designated as intrinsically important corresponding to important discoveries It is very difficult for agents (and probably the simulator) to find out what is true (or how to get there) without painstakingly “working forward” Theories may be used as knowledge or as techniques for producing new knowledge

10. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-10 Part 3: An Example Simulation Exploring the aspect of distributed inference and coordination via journal publication

11. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-11 Aims of this Simulation To illustrate the workings and challenge of this test-bed To show how a simulation might be done within it To explore a particular subset of behaviours, particularly w.r.t. inferring new knowledge (as in maths) and how this is disseminated via a journal Also (not discussed here) to explore the different kinds of behaviour the agents might use and the effect on the effectiveness of collective knowledge development

12. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-12 The test-bed problem The theories are those of classical propositional logic with connectives: , , , , , T, F formulated as a “Hilbert System” with: –14 axioms –1 rule, Modus Ponens (MP) (explained shortly) 110 designated “target theories” taken from textbooks New theories developed by taking applying MP to to existing theories Makes for a fairly tough problem - space of theorems is more than exponential in size

13. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-13 Agent-1 -2 Structure of the Simulation The Journal The Axioms MP

14. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-14 Action of the MP inference rule yyxx  ))(( )() aaaa  BA  (Major Premise) A(Minor Premise) )(aa  B(Inference)

15. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-15 Agents Agents have two (limited) stores, for knowledge (minor) and techniques (major) Each iteration each agent: 1.Decides what new items of knowledge to add to its private stores from the published set, also which to drop (both major and minor). 2.Decides which major premise and what set of minor premises it will try with the MP rule and add any results to its (minor) store. 3.Decides which of its private knowledge (that is not already public) it will submit to the journal Agents may “panic” if they have not discovered anything within a certain number of iterations and replace their knowledge (minor or major)

16. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-16 The Journal ( the Journal of Artificial Sentences and Successful Syllogisms ) The journal is the public repository of knowledge (accessible to all) Each iteration the journal: 1.Makes a short-list of submissions that meet basic criteria (e.g. novelty, number of vars.) 2.Ranks the short-list using a weighted score (in this case, shortening, shortness, past success of submitter, number of variables) 3.Chooses from the ranked short-list (e.g. top N, randomly, probabilistically etc.)

17. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-17 Part 4: Some results from the example simulation Exploring the effects of differing rates of publishing by a journal

18. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-18 The Experiment 20 agents over 50 iterations Each agent stores 4 major and 27 minor premises as its current knowledge and submit all unpublished formulas they find 1 journal, selecting for (in descending order) shortening; shortness; prestige; num vars. Vary the number of formula the journal publishes each iteration from 1…10 Results are averages over 25 independent runs for each setting

19. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-19 Other settings addRateMajor = 1dropRateMajor = 1 addRateMinor = 2dropRateMinor = 2 interalScoreDecay = 0.9 fertilityDecay = 0.9 numToTry = 9 panicSoonTime = 4panicLaterTime = 12 minNumberOfVariablesForPublication = 1 shorteningWeight = 0.1shortnessWeight = 0.01 sourceWeight = 0.001numVariablesWeight = 0.0001 General behaviour: fertilityIsInverseLength matchTargetGenerically publishBest submitAll Major Behaviour: addFertileProb dropWorst feedbackOnUsedFailure panicLater replaceBestIfBad tryBest Minor Behaviour: addRandom dropWorstProb feedbackOnFertility panicSoon replaceProbFertileIfBad tryUntried

20. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-20 Example output from a run … Iteration 3 agent 3 found '->' ('->' A ('->' B C)) ('->' B ('->' A C)) agent 3 found '->' ('->' A ('->' ('->' B C) D)) ('->' ('->' B C) ('->' A D)) agent 6 found '->' ('->' A ('->' B B)) ('->' A ('->' A ('->' B B))) agent 6 found '->' ('->' A ('->' A B)) ('->' A B) agent 6 found '->' ('->' A B) ('->' ('->' B A) ('->' A B)) agent 17 found '->' ('->' A ('->' A B)) ('->' A B) agent 19 found '->' ('->' A B) ('->' ('->' C A) ('->' C B)) agent 19 found '->' ('->' A B) ('->' ('->' C ('->' ('->' A B) D)) ('->' C D)) Iteration 4 agent 7 found '->' ('->' A B) ('->' ('->' ('->' A B) C) C) agent 7 found '->' A ('->' ('->' A B) B) agent 13 found '->' ('->' A ('¬' A)) ('¬' A) agent 15 found '->' ('->' A ('¬' A)) ('¬' A) Iteration 5 Iteration 6 …

21. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-21 Number of formulas in public domain

22. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-22 Number of targets found

23. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-23 Total number found by agents (also num. submitted for publication)

24. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-24 (Average) spread (the SD) of numbers of formulas found by agents

25. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-25 Number found by agents (a single run, njp=2)

26. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-26 Number found by agents (a single run, njp=10)

27. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-27 Part 5: Concluding Discussion

28. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-28 Limitations of current simulation It is not validated, though it could be via citation structure etc. (at least partially) Includes many ‘kludges, (e.g. journal’s selection mechanism by weighted rank) Focuses on certain inference (more like maths than experimental science) Does not manage to sustain the discovery of important theories – this is a challenge! Does not scale well (with a single journal)

29. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-29 What has been achieved? I’ve had a lot of fun It shows that (at least some aspects) of the social processes in science can be simulated It shows that there are some interesting unanswered questions to be addressed It suggests that increasing the publication rate does not necessarily result in more meaningful knowledge being produced The dynamics in even such simplified simulations of science can be quite complex

30. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-30 Extensions (a few of the many possible…) Many journals co-evolving in terms of scope, publication and selection policies Instead of inference, agents could hypothesise theories and test them experimentally (a line of the truth table) Journals could use a peer-review system with agents rating shortlisted submissions Informal social networks etc. can be another channel for knowledge New student agents inherit knowledge from older agents and sometimes they retire

31. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-31 The connection with distributed theorem-proving Single agents in this simulation operate in a similar way to some established forward- chaining theorem provers (e.g. OTTER ) Not designed to be an efficient (“tuned”) theorem proved But lessons learned might be applicable to the design of distributed theorem provers (exploiting parallelism) Collections of agents may be able to be “trained” on a logic and then deployed

32. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-32 Some issues that could be investigated in this test-bed include: Is any norm on methods for discovering new knowledge is counterproductive? ( Feyerabend ) What is the effect of the framework (within which knowledge is expressed) on the structure of new knowledge? ( Kuhn ) When and how do social processes act to increase the reliability of knowledge collectively produced (or otherwise)? ( Merton, Popper ) Is it helpful to have an inviolate core of knowledge/techniques that is not open to revision? ( Lakatos )

33. Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-33 The End Bruce Edmonds bruce.edmonds.name Centre for Policy Modelling cfpm.org Ad: Socially Inspired Computing A 2-day workshop 12 th &13 th April 2005 Part of AISB 2005 convention: Social Intelligence and Interaction in Animals, Robots and Agents At Univ. of Hertfordshire, UK (just north of London) Workshop website: http://cfpm.org/sic Co-located with 15 other symposia, including: Emerging Artificial Societies (14 th &15 th April)