1 Al Aho Computational Thinking in Programming Language Design Alfred V. Aho CMU-Qatar Dean’s Lecture March 21, 2016

2 Al Aho

3 Al Aho Software in our world today How much software does the world use today? Guesstimate: over one trillion lines of source code What is the sunk cost of the legacy base? $10 to $100 per line of finished, tested source code How many bugs are there in the legacy base? 10 to 10,000 defects per million lines of source code Adapted from A. V. Aho Software and the Future of Programming Languages Science, February 27, 2004, pp

4 Al Aho The most popular programming languages 1970 Fortran Lisp Cobol Algol 60 APL Snobol 4 Simula 67 Basic PL/1 Pascal 2016 Java C C++ C# Python PHP Visual Basic.NET JavaScript Perl Ruby [ March 2016]

5 Al Aho Why are there so many languages? One language cannot serve all application areas well – e.g., programming web pages (JavaScript) – e.g., electronic design automation (VHDL) – e.g., parser generation (YACC) Programmers often have strongly held opinions about – what makes a good language – how programming should be done There is no universally accepted metric for the goodness of a programming language!

6 Al Aho Evolutionary forces on languages Increasing diversity of applications Increasing programmer productivity and shortening time to market Need to improve software security, reliability and maintainability Emphasis on mobility and distribution Support for parallelism and concurrency New mechanisms for modularity Trend toward multi-paradigm programming

7 Al Aho The importance of computational thinking Computational thinking is a fundamental skill for everyone, not just for computer scientists. To reading, writing, and arithmetic, we should add computational thinking to every child’s analytical ability. Just as the printing press facilitated the spread of the three Rs, what is appropriately incestuous about this vision is that computing and computers facilitate the spread of computational thinking. Jeannette M. Wing Computational Thinking CACM, vol. 49, no. 3, pp , 2006

8 Al Aho What is computational thinking? The thought processes involved in formulating problems so their solutions can be represented as computation steps and algorithms. Alfred V. Aho Computation and Computational Thinking The Computer Journal, vol. 55, no. 7, pp , 2012

9 Al Aho What is an algorithm? A finite sequence of instructions, each of which has a clear meaning and can be performed with a finite amount of effort in a finite length of time. Alfred V. Aho, John E. Hopcroft, and Jeffrey D. Ullman Data Structures and Algorithms Addison Wesley, 1983

10 Al Aho A good way to learn computational thinking Design and implement your own programming language!

11 Al Aho Programming languages Programming languages are notations for describing computations to people and to machines. Underlying every programming language is a model of computation: Procedural: C, C++, C#, Java Declarative: SQL Logic: Prolog Functional: Haskell Scripting: AWK, Perl, Python, Ruby

12 Al Aho Computational thinking for programming language design Problem Domain Mathematical Abstraction Mechanizable Model of Computation Programming Language

13 Al Aho AWK is a scripting language for routine data- processing tasks designed by Al Aho, Brian Kernighan, Peter Weinberger at Bell Labs Each co-designer had a slightly different motivation: –Aho wanted a generalized grep –Kernighan wanted a programmable editor –Weinberger wanted a database query tool Each co-designer wanted a simple, easy-to-use language The birth of AWK

14 Al Aho for each file for each line of the current file for each pattern in the AWK program if the pattern matches the input line then execute the associated action AWK’s model of computation: pattern-action programming

15 Al Aho An AWK program is a sequence of pattern-action statements pattern { action }... Each pattern is a boolean combination of regular, numeric, and string expressions An action is a C-like program If there is no { action }, the default is to print the line Invocation awk ’program’ [file1 file2... ] awk –f progfile [file1 file2... ] Structure of an AWK program

16 Al Aho 1.Print the total number of input lines END { print NR } 2.Print the last field of every input line { print $NF } 3.Print each input line preceded by its line number { print NR, $0 } 4.Print all unique input lines !x[$0]++ Some useful AWK “one-liners”

17 Al Aho Comparison: regular expression pattern matching in Perl, Python, Ruby vs. AWK Time to check whether a? n a n matches a n regular expression and text size n Russ Cox, Regular expression matching can be simple and fast (but is slow in Java, Perl, PHP, Python, Ruby,...) [ 2007]

18 Al Aho Towards a model of computation for quantum programming languages The four postulates of quantum mechanics M. A. Nielsen and I. L. Chuang Quantum Computation and Quantum Information Cambridge University Press, 2000

19 Al Aho State-space postulate The state of an isolated quantum system can be described by a unit vector in a complex Hilbert space. Postulate 1

20 Al Aho Qubit: quantum bit The state of a quantum bit can be described by a unit vector in a 2- dimensional complex Hilbert space (in Dirac notation) where α and β are complex coefficients called the amplitudes of the basis states and, and In linear algebra

21 Al Aho Time-evolution postulate Postulate 2 The evolution of a closed quantum system can be described by a unitary operator U. (An operator U is unitary if U † U = I.) U state of the system at time t 1 state of the system at time t 2

22 Al Aho Useful quantum operators: Hadamard The Hadamard operator has the matrix representation H maps the computational basis states as follows Note that HH = I.

23 Al Aho Composition-of-systems postulate The state space of a combined physical system is the tensor product space of the state spaces of the component subsystems. If one system is in the state and another in the state, then the combined system is in the state. is often written as or as. Postulate 3

24 Al Aho Useful quantum operators: CNOT The two-qubit CNOT (controlled-NOT) operator has the matrix representation: CNOT flips the target bit t iff the control bit c has the value 1:. c t c The CNOT gate maps

25 Al Aho Measurement postulate Quantum measurements can be described by a collection of operators acting on the state space of the system being measured. If the state of the system is before the measurement, then the probability that the result m occurs is and the state of the system after measurement is Postulate 4

26 Al Aho Properties of measurement operators The measurement operators satisfy the completeness equation: The completeness equation says the probabilities sum to one:

27 Al Aho Computational model: quantum circuits Quantum circuit to create Bell (Einstein-Podulsky-Rosen) states: Circuit maps Output is an entangled state, one that cannot be written in a product form. (Einstein: “Spooky action at a distance.”) x y H

28 Al Aho Language translators s in S, a language translator maps s into a sentence Given a source language S, a target language T, and a sentence s in S, a language translator maps s into a sentence t in T that has the same meaning as s.

29 Al Aho Translation of programming languages Compiler source program target program input output

30 Al Aho Phases of a compiler Semantic Analyzer Interm. Code Gen. Syntax Analyzer Lexical Analyzer Code Optimizer Code Gen. source program token stream syntax tree annotated syntax tree interm. rep. interm. rep. target program Symbol Table Alfred V. Aho, Monica S. Lam, Ravi Sethi and Jeffrey D. Ullman Compilers: Principles, Techniques, & Tools Addison Wesley, 2007

31 Al Aho The programming languages and translators course at Columbia 1.Theory principles of modern programming languages fundamentals of compilers fundamental models of computation 2.Practice a semester-long programming project in which students work in small teams to create and implement an innovative little language of their own design. This project teaches computational thinking as well as project management, teamwork, and communication skills that are useful in all aspects of any career.

32 Al Aho The project schedule Week Task 2 Form a team and design an innovative new language 4 Write a whitepaper on your proposed language modeled after the Java whitepaper 8 Write a tutorial patterned after Chapter 1 and a language reference manual patterned after Appendix A of Kernighan and Ritchie’s book, The C Programming Language 14 Give a ten-minute presentation of your language to the class 15 Give a 30-minute working demo of your compiler to the teaching staff 15 Hand in the final project report

33 Al Aho Some of the languages created Producing applications for an Android cell phone Configuring a wireless sensor network Turning data into music Giving advice on what to wear Generating code for a quantum computer

34 Al Aho

35 Al Aho A compiler for a quantum computer Front End Technology Independent CG+Optimizer Technology Dependent CG+Optimizer Technology Simulator quantum source program QIRQASMQPOL QIR: quantum intermediate representation QASM: quantum assembly language QPOL: quantum physical operations language quantum circuit quantum circuit quantum device quantum mechanics K. Svore, A. Aho, A. Cross, I. Chuang, I. Markov A Layered Software Architecture for Quantum Computing Design Tools IEEE Computer, 2006, vol. 39, no. 1, pp Computational abstractions

36 Al Aho LIQUi|>: language-integrated quantum operations LIQUi|> is a software architecture and toolsuite for quantum computing Includes a programming language, optimization and scheduling algorithms, and quantum simulations Translates quantum algorithms written in a high-level programming language into the low-level instructions for a quantum device Developed by Dave Wecker and Krysta Svore of the Quantum Architectures and Computation group at Microsoft Research Platform for the 2016 Microsoft Quantum Challenge Dave Wecker and Krysta Svore LIQUi|>: A Software Design Architecture and Domain-Specific Language for Quantum Computing arXiv: v1 [quant-ph] 18 Feb 2014

37 Al Aho Towards more reliable software How can we get reliable software from unreliable programmers?

38 Al Aho IEEE Spectrum Software Hall of Shame 2004UK Inland RevenueSoftware errors contribute to $3.45 billion tax-credit overpayment 2004J Sainsbury PLC [UK]Supply chain management system abandoned after deployment costing $527M 2002CIGNA CorpProblems with CRM system contribute to $445M loss 1997U. S. Internal Revenue Service Tax modernization effort cancelled after $4 billion is spent 1994U. S. Federal Aviation Administration Advanced Automation System canceled after $2.6 billion is spent YearCompanyCosts in US $ R. N. Charette, Why Software Fails, IEEE Spectrum, September 2005

39 Al Aho Software errors in scientific Papers Five papers published in Science, the Journal of Molecular Biology, and the Proceedings of the National Academy of Sciences were retracted because of software errors. Zeeya Merali Computational science: … Error... why scientific programming does not compute Nature 467, , 13 October 2010

40 Al Aho NASA’s Mars Science Laboratory Mars Science Laboratory (MSL) is a robotic space probe mission launched by NASA on November 26, 2011 It successfully landed Curiosity, a robotic Mars rover, in the Gale Crater on August 5, 2012 MSL depends on millions of lines of software working correctly

41 Al Aho Earth Mars Mercury Venus Sun 26 November August 2012 a trip of 350 million miles Mission to Mars…

42 Al Aho Destination: Gale Crater an old streambed 12 x 4.3 mile landing ellipse

43 Al Aho How do you make sure that it works? 43

44 Al Aho And what about the software? 3 million lines of C code 120 parallel threads ( VxWorks tasks) 2 CPUs (1 spare) 5 years development time, with a team of 40 software engineers (< 10 lines of code per hour) 1 customer, 1 use: it has to work the first time How do you get it right? Gerard Holzmann

45 Al Aho Getting it right Some of the things done differently from previous missions: 1. Defined a new risk-based Coding Standard with tool-based compliance checks 2. Introduced a Certification program for flight software developers 3. Introduced routine use of strong Static Source Code Analysis tools 4. Defined a new Code Review process and Tool (scrub), integrated with static analysis 5. Made use of formal analysis for key subsystems with Logic Model Checking

46 Al Aho The Spin model checker Developed by Gerard Holzmann at Bell Labs starting in 1980 Spin has been used worldwide for the formal verification of multi- threaded software applications Available as an open-source software verification tool Spin was used to help verify the software in NASA’s Mars Science Laboratory

47 Al Aho Verifying concurrent code: what is the state-of-the-art? a small example : manual proof (a few months) proof sketch: 5 pages, 7 Lemmas, 5 Theorems 2004: new proof with PVS theorem prover (3 months) 2006: +CAL model & TLA+ proof (a few days) Is it any easier today? 2000: manual proof (a few months) proof sketch: 5 pages, 7 Lemmas, 5 Theorems 2004: new proof with PVS theorem prover (3 months) 2006: +CAL model & TLA+ proof (a few days) Is it any easier today?

48 Al Aho Today this verification takes seconds $ verify dcas.c..report assertion violation $ 1.this takes C code as input it uses the modex model-extractor to generate a formal model mechanically, and then runs the Spin model-checker to check if the assertion can be violated 2.all steps together take about 10 seconds 3.the verification step itself takes a fraction of that Gerard Holzmann

49 Al Aho Cutting to the chase In the first (Earth) year on the surface of Mars the previous mission lost 26 days of operation to software bugs. In the first year on Mars the MSL mission lost 1 day to a single bug.

50 Al Aho Telling lessons learned by PLT students “During this course we realized how naïve and overambitious we were, and we all gained a newfound respect for the work and good decisions that went into languages like C and Java which we’ve taken for granted for years.” “Designing a language is hard and designing a simple language is extremely hard!”