Assurance techniques for code generators Ewen Denney USRA/RIACS, NASA Ames Bernd Fischer ECS, U Southampton

Assurance problem Safety/mission-critical software requires assurance that it meets a certain level of “quality” What are the issues in assuring automatically generated code? –Different forms of assurance –Different assurance techniques –Diverse generator paradigms

Forms of assurance What exactly might we need to assure? Compliance with requirements Compliance with spec/model Certification standards Coding standards Absence of run-time errors Traceability Appropriate documentation Correctness Reliability Legibility

Participants Harold Ossher Markus Pueschel Julia Lawall Ann Le Meur Yannis Smaragdakis Oleg Kiselyov Tom Ellman Gabor Karsai Kevin Hammond Laurence Tratt Baris Aktemur Walid Taha Bernd Fischer Ewen Denney

Target domains numerical code –statistical data analysis –GN&C –physics-based animation –linear transforms embedded systems –real-time systems –device drivers optimizing simulators programming language tools

Generator paradigms mathematical, schema-based –templates and symbolic reasoning source-level transformations –DSLs –AOP –template metaprogramming –staged programming model-driven –graph-transformations

The Holy War?!? Thou shalt qualify thy generator vs. Certify the generated programs, Luke Certification ≠ Verification! Safety ≠ Correctness! Should prove parts of the generator correct –find problems earlier: in generator rather than at compilation time –domain knowledge (much) easier to understand at higher- level than in generated code Generate proofs that can be checked Compositional verification Safety is ultimately a system question

Some Current Approaches Distinction between generator framework and domain knowledge reflected in distinction between verification and validation Testing generator rules in Spiral: –domain source might be wrong –formalization might be wrong –plug in parameters and check an instance of the transformation Simulate algorithm instances in AutoFilter Compose aspects while ensuring they don’t corrupt each other –ultimately: want behavioral equivalence “Type systems can encode interesting things” "Our formal abilities are laughable“

Traceability and Documentation Doing it manually very tedious and error-prone Adding "rationale system" to explain the transformation steps –Programming traceability info was harder than the rest of the system, but very important –Good for debugging, but users don't care Relating performance model to higher-level description? Optimization blurs boundaries Establishing bisimulation gives trace Tracing is much easier in “horizontal” systems rather than vertical systems

Bake-off A bake-off for assuring generators? Need challenge problems, consisting of –classes of specs, –algorithms for generating programs –proofs that the algorithms are correct –…

Conclusions??? In Europe, everything is proven, but nothing works. In the US, nothing is proven, but it works. And in code generation, nothing works and nothing is proven…