Dichotomies: Software Research vs Practice Peter Lee Carnegie Mellon University HCMDSS Workshop, June 2005 Peter Lee Carnegie Mellon University HCMDSS Workshop, June 2005

“We are like the barber-surgeons of earlier ages, who prided themselves on the sharpness of their knives and the speed with which they dispatched their duty  either shaving a beard or amputating a limb. Imagine the dismay with which they greeted some ivory-towered academic who told them that the practice of surgery should be based on a long and detailed study of human anatomy, on familiarity with surgical procedures pioneered by great doctors of the past, and that it should be carried out only in a strictly controlled bug-free environment, far removed from the hair and dust of the normal barber’s shop.”  Sir Tony Hoare

Normal vs radical design Many of the most promising research ideas today require re-inventing (parts of) the world.  M. Jackson

“It’s hard to read through a book on the principles of magic without glancing at the cover periodically to make sure it isn’t a book on software design.”  Bruce Tognazzini

Dichotomies State of research vs practice radical vs normal design inspiration vs process verification/analysis vs testing “shall not” vs “shall” specifications intrinsic vs extrinsic properties State of research vs practice radical vs normal design inspiration vs process verification/analysis vs testing “shall not” vs “shall” specifications intrinsic vs extrinsic properties

Some “radical” concepts Use of formal (usually mathematics or logic based) specification languages Systematic derivation of implementations from specifications, or else formal analysis/verification of implementations against specifications Emphasis on provable soundness of methods Use of formal (usually mathematics or logic based) specification languages Systematic derivation of implementations from specifications, or else formal analysis/verification of implementations against specifications Emphasis on provable soundness of methods

Example: PCC Define possible machine states S and semantics Step(S) Define the “safety policy” on states, SP(S) Define safe execution: Safe(S) = ∀ n. SP(Step n (S)) Prove safety, P : Safe(S) Deliver proof-carrying code, (S 0, P) Define possible machine states S and semantics Step(S) Define the “safety policy” on states, SP(S) Define safe execution: Safe(S) = ∀ n. SP(Step n (S)) Prove safety, P : Safe(S) Deliver proof-carrying code, (S 0, P)

Implications Formal specification involves inspiration and technology, not just carefully managed software process Verification/derivation sometimes entails use of “radical” programming languages and specially- trained programmers Provable soundness implies conservatism, either less ambitious programs/components or theorems Formal specification involves inspiration and technology, not just carefully managed software process Verification/derivation sometimes entails use of “radical” programming languages and specially- trained programmers Provable soundness implies conservatism, either less ambitious programs/components or theorems

“If you have ‘process’ without ‘inspiration,’ all you end up with is well-documented crap.”  John C. Sommerer

Bridging the radical and normal In recent years, major advances in type theory, model checking, code certification, and automated theorem proving have reinvigorated interest in practical applications SLAM, Praxis, ESC, PCC,... Potentially lots of low-hanging fruit in the area of medical device software Potentially also a context for fundamental (less directly applied) research In recent years, major advances in type theory, model checking, code certification, and automated theorem proving have reinvigorated interest in practical applications SLAM, Praxis, ESC, PCC,... Potentially lots of low-hanging fruit in the area of medical device software Potentially also a context for fundamental (less directly applied) research

Medical devices Medical devices are interesting: many have yet to be invented, and hence re- invention of the software technology may be more of a possibility their embedded nature can make them smaller, less ambitious in some design requirements, and hence more susceptible to radical methods they have to work correctly, and so perhaps in some cases there is no choice Medical devices are interesting: many have yet to be invented, and hence re- invention of the software technology may be more of a possibility their embedded nature can make them smaller, less ambitious in some design requirements, and hence more susceptible to radical methods they have to work correctly, and so perhaps in some cases there is no choice

Dichotomies in specifications How we think about specifying our systems can have powerful consequences Example: Current practice is based primarily on “shall” statements completeness? missing behavior in normal vs failure modes missing “shall not” statements much of the “low-hanging fruit” is in tools and methods for verifying “shall not” statements How we think about specifying our systems can have powerful consequences Example: Current practice is based primarily on “shall” statements completeness? missing behavior in normal vs failure modes missing “shall not” statements much of the “low-hanging fruit” is in tools and methods for verifying “shall not” statements

Intrinsic vs extrinsic properties Most of the research on radical methods today focuses on intrinsic properties of software e.g., “this program shall not instruct the pump to give more than 2 doses of insulin per day” But as a practical / engineering matter, we need also to understand the extrinsic properties e.g., “this device will not kill the patient through an overdose of insulin” Most of the research on radical methods today focuses on intrinsic properties of software e.g., “this program shall not instruct the pump to give more than 2 doses of insulin per day” But as a practical / engineering matter, we need also to understand the extrinsic properties e.g., “this device will not kill the patient through an overdose of insulin”

Extrinsic properties Intrinsic properties are extremely powerful, providing what amounts to an “atomic” or “genomic” view of software But we may need, also, a science of the extrinsic, for much the same reason we need the laws of thermodynamics [Analogy due to Mary Shaw] This is particularly important in future medical devices, which will need to work in networked environments, with knowledge of human system Intrinsic properties are extremely powerful, providing what amounts to an “atomic” or “genomic” view of software But we may need, also, a science of the extrinsic, for much the same reason we need the laws of thermodynamics [Analogy due to Mary Shaw] This is particularly important in future medical devices, which will need to work in networked environments, with knowledge of human system

Summary High confidence software requires radical concepts, and this has been the primary focus of much of today’s most important research Medical device software may be a good context for bridging the dichotomy between radical and normal software development In addition to its traditional focus on intrinsic properties of software, a science of the extrinsic should also be considered High confidence software requires radical concepts, and this has been the primary focus of much of today’s most important research Medical device software may be a good context for bridging the dichotomy between radical and normal software development In addition to its traditional focus on intrinsic properties of software, a science of the extrinsic should also be considered