The Scientific Community Game for STEM Innovation and Education (STEM: Science, Technology, Engineering and Mathematics) Karl Lieberherr Ahmed Abdelmeged 3/16/20111Open House 2011

3/16/2011Open House SCG = Scientific Community Game

Why Scientific Community Game? Why are you here? … motives in academic publishing: – desire for recognition and respect from the people one regards as peers, – desire to have impact (on conclusions being reached, on the development of the discipline, etc.), and – desire to participate in significant knowledge- building discourse. e.g., Scardamalia, M., & Bereiter, C. (1994) Open House /16/2011

Why do we model Scientific Communities? Scientific Communities create and disseminate new knowledge to help society. A computational model of scientific communities supports the same efforts for computational problems: – focused collaboration and competition – checking of the rules of a scientific community – knowledge maintenance and evaluation 3/16/20114Open House 2011

How it Works Scholars propose and oppose (refute or strengthen) or agree on claims. Strengthen and agree are reduced to refute. Claims predict the outcome of a refutation protocol. Parameterized by two structures: Domain and Protocol. Claim Example: Alice claims that she can solve problem instances in instance set I with quality at least q using resources at most r. 3/16/20115Open House 2011

Mathematics Example: Strengthen, Agree and Refute Alice claims Claim C(t): ForAll [x in [0,1]] Exists[y in [0,1]] (x*y + (1-x)*(1-y^2)) >= t Consider claims: C(0.55): Bob strengthens this claim (0.6). C(0.60): Bob agrees with this claim. C(0.65): Bob attempts to refute this claim. 3/16/2011Open House Roles: Alice claims Bob opposes or agrees

Refute Alice claims ForAll [x in [0,1]] Exists [y in [0,1]] (x*y + (1-x)*(1-y^2)) >= 0.65 Bob refutes Protocol: – Bob provides x – Alice provides y – Bob refutes iff (x*y + (1-x)*(1-y^2)) < /16/2011Open House 20117

Karl Popper 3/16/20118Open House 2011

SCG is a web-based implementation of Karl Popper’s science ideas One of the greatest philosophers of science of the 20th century.philosophers of science Falsifiability or refutability is the logical possibility that an assertion could be shown false by a particular observation or physical experiment. Error elimination (refutation), performs a similar function for science that natural selection performs for biological evolution.natural selectionbiological evolution Open House from Wikipedia 3/16/2011

Comparison Karl Popper: Conjectures and Refutations Scientific Community Game: Claims and Refutations Our claims are about computational problems. Open House /16/2011

Automating the refutations There can be “bugs” in refutations. With a computational model of scientific communities we can check for many “bugs”. Can detect misunderstandings students might have. Fair evaluation of scholars. 3/16/201111Open House 2011

Designers SCG Domain – Instance, Solution, InstanceSet, valid, quality – basic domain functionality, like standard solvers and solvers for niches. providing instances with “interesting” solutions Protocol: using protocol language – standard protocols: ForAllExists, PositiveSecret, etc. Playground: Configure – Research/Development Managers (Innovation) – Professors (Teaching) Avatar – researchers, developers – students 3/16/201112Open House 2011

Domain Instance (language) Solution (language) – boolean valid(Instance) – [0,1] quality(Instance) InstanceSet (language, subset of Instance) – boolean valid() – boolean belongsTo(Instance) Response = Instance union Solution 3/16/201113Open House 2011

SCG(Domain) Protocol (fixed language) Claim(Domain) – boolean strengthen(Claim other) // other strengthens this – Domain.InstanceSet getInstanceSet() – Protocol getProtocol() – [0,1] getQuality() – [-1..1] getResult(List(Domain.Response)) 3/16/201114Open House 2011

Refutations of Claim and !Claim are efficient Claim: F unsatisfiable if refuted: Bob finds satisfying J; proof of !Claim. If defended: baby step towards proof of Claim. Proof: long !Claim: F satisfiable if refuted: Alice does not find satisfying J; baby step towards proof of Claim. If defended: proof of !Claim. Proof: short 3/16/201115Open House 2011 Roles: Alice claims Bob attempts to refute

Both refutations are efficient Claim: Exists F in IS All J: fsat(F,J)<=t if refuted: Bob finds J; proof of !Claim assuming Alice is perfect. If defended: baby step towards proof of Claim. Proof short. !Claim: F has J: fsat(F,J)>=t All F in IS Exists J: fsat(F,J)>=t if refuted: Alice does not find J; baby step towards proof of Claim. If defended: proof of !Claim if Bob is perfect. Proof short. 3/16/201116Open House 2011

Claim involving Experiment Claim ExperimentalTechnique(X,Y,q,r) I claim, given raw materials x in X, I can produce product y in Y of quality q and using resources at most r. 17Bionetics 2010

Our vision Researchers and Professors come to the SCG website and configure a new playground X in which tournaments will take place. Participating teams get baby avatars generated for X that participate in daily competitions. Competition generates a wealth of information: educated employees/students, good (undefeated) software, good algorithms, good potential employees. Reward is given to the winner. 3/16/2011Open House

Conclusions Computational Modeling of Scientific Communities is a good idea: – foster Innovation – improve education STEM domains: use the web effectively Current use: – Algorithms class – Software development class 3/16/2011Open House

3/16/2011Open House Thank you!

Thank you! 3/16/2011Open House

Strengthening When claim C is strengthened by Bob to C', Alice must try to refute C' and the strengthening holds only if Bob defends C'. strengthenP(C,C') must hold. When scholar Bob successfully strengthens a claim of Alice, Bob wins reputation: – Bob + ClaimConfidence + |quality(C)-quality(C')| When scholar Alice successfully defends her own claim against Bob, Alice wins reputation. – Alice + ClaimConfidence 3/16/2011Open House

Agreement When Bob agrees on claim C with Alice, – (1) Bob must defend C against Alice (if not, Bob loses) – (2) Bob must refute C' = C minimally strengthened along quality dimension (using the configuration file constant minStrengthen) with Alice as defender (if not, Bob loses). Then Alice must do the same: 3/16/2011Open House

Agreement – (1) Alice must defend C against Bob (if not, Alice loses) – (2) Alice must refute C' with Bob as defender (if not, Alice loses) If all those protocols produce the result as described, the claim goes into the social welfare set (the knowledge base of claims believed to hold and having maximum strength). 3/16/2011Open House

Refutations of Claim and !Claim are efficient Claim: F unsatisfiable if refuted: Bob finds satisfying J; proof of !Claim. If defended: baby step towards proof of Claim. Proof: long !Claim: F satisfiable if refuted: Alice does not find satisfying J; baby step towards proof of Claim. If defended: proof of !Claim. Proof: short Alice should never have made the claim! 3/16/201125Open House 2011

Both refutations are efficient Claim: Exists F in IS All J: fsat(F,J)<=t if refuted: Bob finds J; proof of !Claim assuming Alice is perfect. If defended: baby step towards proof of Claim. Proof short. !Claim: F has J: fsat(F,J)>=t All F in IS Exists J: fsat(F,J)>=t if refuted: Alice does not find J; baby step towards proof of Claim. If defended: proof of !Claim if Bob is perfect. Proof short. Alice should never have made the claim!? 3/16/201126Open House 2011

Designers SCG Domain – includes designing basic components for avatar like standard solvers. Example: HSR: linear search solver Protocol Playground: Goal: make playground designers configurators. Avatar designers 3/16/201127Open House 2011

Example Playground Design Highest Safe Rung Configuration: – domain HSR – claim 1: instanceSetClass protocolClass – claim 2: instanceSetClass !protocolClass – initialReputation = 100 – … 3/16/201128Open House 2011

Designers: what they produce SCG /scg – scg.cd, scg.beh – /protocol Java classes: Singleton Pattern Domain /domain – /hsr: hsr.cd, hsr.beh /avatar (components for avatar) Playground – config file: location of configuration file is given as argument to admin 3/16/201129Open House 2011

Config Config = // to configure admin SCGConfig Wrap(DomainConfigI). Example entries: – domain CSP – claim 1: instanceSetClass protocolClass – claim 2: instanceSetClass !protocolClass – initialReputation = 100 – … 3/16/201130Open House 2011

Where can we find configuration settings If there is a configuration file location given to the admin – in the configuration file If not: the default value given in the code. 3/16/201131Open House 2011

3/16/201132Open House 2011

Designers SCG Domain – Instance, Solution, InstanceSet, valid, quality – basic domain functionality, like standard solvers and solvers for niches. providing instances with “interesting” solutions Protocol: using protocol language – standard protocols: ForAllExists, PositiveSecret, etc. Playground: Configurate – Research/Development Managers (Innovation) – Professors (Teaching) Avatar – researchers, developers – students 3/16/201133Open House 2011

Example Playground Design Highest Safe Rung Configuration: – domain HSR – claim 1: instanceSetClass protocolClass – claim 2: instanceSetClass !protocolClass – initialReputation = 100 – … 3/16/201134Open House 2011