Automating programming via concept mining, probabilistic reasoning over semantic knowledge base of SE domain by Max Talanov

 A lot of trivial tasks that could be automated  Add/remove field Patronymic on Customer page  Add dropdown list on the form...  A lot of not so trivial solutions that should be reused but are not reused  How-tos  Libraries...  Approximately 60% of developer's time in outsourcing is spent to solve this kind of problems </ Problem

 Template based code generators:  IDEs  Visual studio.  IDEA. ...  Template based generators  Maven  Archetypes  CASE Tools:  Rational Rose  ArgoUML </ Current Solutions

 Once generated solution is hardly maintainable and require to allocate significant amount of money for farther support.  Developer generated the solution based on DB structure, then added some functionality in it, then customer wants to change the DB structure.  Developer has to regenerate and merge his further changes  This is done for only one reason – generator does not understand what it's doing. </ They cannot solve problem

2005 MIT media lab published the article “FEASIBILITY STUDIES FOR PROGRAMMING IN NATURAL LANGUAGE”. Metafor is the program that creates the sceleton of the Python classes based on shallow English descripton. </ MIT Metafor

Input: shallow English description Output: scaffolding Python classes. Metafor utilised natural language processor Montylingua, common sence KB ConceptNet, programmatic interpreter. </ MIT Metafor

System should:  Operate with changes to be applied not the static structure of the target application (How-to).  Use domain knowledge model to map the inbound requirements into acceptance criteria to be used to create the solution.  Use both trained data and generation to create the solution.  Use several abstraction layers of the target application. </ Key Ideas

 System has to operate with knowledge.  System has to understand what it is doing  Architecture of the target application.  Methods to change the architecture.  Domain specific information.  Requirements for changes.  System has to understand the human operator  Communicate in natural language. </ Requirements

 Knowledge base.  Linguistic component.  Perceiving component.  Solution generator.  Communication component. </ Key Components

Linguistic Req Perceiving Solution generator Communicator Updated app Request KB </ Collaboration Diagram

Is the main storage of the data to be used by the system. KB is RDF storage with OWL data. KB is used to store the semantic information:  Target application architecture  Domain specific knowledge (How-tos)  Common sense information  Predicates generated based on requirements text  Acceptance criteria for generated solution ... </ Knowledge Base

Human expert specifies the requirements, linguistuc component generates set of predicates for further processing. Inbound: CR, bug report or FRS according to SE standard SPICE. Outbound: set of predicates. Stanford Parser creates the set of predicates that are treated as inbound knowledge. Linguistic Requirements </ Linguistic Component

Perceiving module maps inbound predicates to the model (domain model) in knowledge base, using trained data and stochastic search generation. In case of failure invokes Communicator to generate clarification request. Inbound: set of predicates and domain knowledge model. Outbound: predicates mapped to domain knowledge model. LinguisticPerceiving Solution generator Predicates Updated model </ Perceiving Component

Selects or generates the solution for the specified acceptance criteria in updated model. Provides the updated application with the confirmation request to human expert. Inbound: acceptance criteria in KB Outbound: solution in actual code. Updated app Perceiving Solution generator Updated model </ Solution Generator Components

 Reasoner.  Reasoner interface.  Genetic generator.  Solution checker.  Trainer.  Associator.  Generalizer.  Analogy detector.  Target language translator. </ Solution Generator Components

</ Solution Generator Activities Analogy Detector: retrieves solution for similar acceptance criteria Solution checker: returns solution assessment Genetic generator: generates new solution Communicator: sends confirmation request to human expert. Communicator: analyses reply of human expert Trainer: run Solutuon found? Solution ok? yes no

For the purpose of making a logical inference for axepted alternatives within environment of possible contradictions and several probable variants, we decided to use probabilistic reasoner NARS. NARS main features:  Deduction  Induction  Analogy ... </ Reasoner

Inbound: Acceptance criteria. Outbound: Solution in form of How-tos sequences. Generator is capable of creation of the sequences of How-tos according to the acceptance criteria. Inference is produced by NARS. This could be interpreted as human imagination mechanism. </ Genetic Generator

Is devoted to logically infer the percentage of how good is generated solution according to acceptance criteria. Solution checker mainly relies on NARS probabilistic mechanisms, but collects all proper information from KB to be processed by reasoner. </ Solution Checker

Machine learning component, is used to detect associations and infer generic associations of inbound acceptance criteria and approved solutions. Analogy detector is used to retrieve previously learned associations that could be used for specified acceptance criteria. </ Trainer

Translates knowledge representation of architecture in actual files in target language, based on previously described syntax in KB. </ Target Language Translator

Component is dedicated to generate requests to human expert and analyse replyes of the expert. Perceiving Solution generator Communicator Request </ Communicator

</ Feedback Loops Linguistic Req Perceiving Solution generator Communicator Updated app Request KB

</ Current Implementation Acceptance criteria Solution generator Communicator Updated app Request KB

 Add evolution mechanism.  Add inbound information analysis.  Add architectural analysis.  Add self optimization and self improvement.  Extend perceiving algorithm to use encyclopaedic resources to extend domain knowledge. </ Future Plans

 Metafor : Prog.pdf  Maven : lifecycle.html  Stanford Parser:  Open NARS:  Menta: </ References