1. 1 The Formal Verification of SPIDER Lee Pike Department of Computer Science Indiana University, Bloomington lepike@indiana.edu

2. 1 Thanks to ● Steven Johnson, Indiana University, Bloomington ● The National Institute of Aerospace ● The NASA LaRC Formal Methods Team, especially Paul Miner

3. 1 Overview ● SPIDER Overview ● Reasoning about Faults ● The Old vs. New Interactive Consistency (IC) Protocol ● SPIDER Formal Verification Goals & Future Work ● References

4. 1 SPIDER Overview Why? ● Develop a fault-tolerant architecture based on an ultra-reliable bus ● Scalable ● Handle a large number of possibly-simultaneous faults, specifically transient faults from electromagnetic effects. ● Provide reintegration services ● Case study for the FAA ● Developed in accordance with RTCADO-254: Design Assurance Guidance for Airborne Electronic Hardware. ● Provide a test-bed for techniques in the specification and verification of safety-critical electronic systems. These sort of architectures are the foundation of tomorrow's X-by wire safety-critical systems.

5. 1 SPIDER Overview What? ● Scalable Processor-Independent Design for Electromagnetic Resilience

6. 1 SPIDER Overview What? ● Scalable Processor-Independent Design for Electromagnetic Resilience ● Processor Elements (PEs) PE

7. 1 SPIDER Overview What? ● Scalable Processor-Independent Design for Electromagnetic Resilience ● Processor Elements (PEs) ● Reliable Optical BUS (ROBUS) ● Time Division Multiple Access (TDMA) bus ● Maintains Synchrony between PEs. ● Prevents Babbling Idiots & PE-to-PE interference ● The services of the ROBUS are the focus of the verification effort. ROBUS PE

8. 1 ROBUS Overview Topology BIU1 to PE BIU2 BIU3 RMU1 RMU2 ROBUS RMU3 ● n Bus Interface Units (BIUs) ● m Redundancy Management Units (RMUs) ● The BIUs and RMUs are called nodes. ● Every BIU and RMU is directly connected. ● No two BIUs are directly connected. Similarly for the RMUs.

9. 1 ROBUS Overview Services (Protocols) ● Interactive Consistency Purpose: Reliably broadcast messages between PEs. ● Clock Synchronization Purpose: Maintain synchrony between all nodes and PEs. ● Distributed Diagnosis Purpose: Convict faulty nodes in the ROBUS. The focus of this talk is Interactive Consistency.

10. 1 Global Fault Classifications ● Good Not faulty node d d d

11. 1 Global Fault Classifications ● Good Not faulty ● Benign Broadcasts only detectably faulty messages node garbag e

12. 1 Global Fault Classifications ● Good Not faulty ● Benign Broadcasts only detectably faulty messages ● Symmetric Broadcasts the same arbitrary message to all node d'

13. 1 Global Fault Classifications ● Good Not faulty ● Benign Broadcasts only detectably bad messages ● Symmetric Broadcasts the same arbitrary message to all ● Asymmetric (Byzantine) Arbitrarily sends arbitrary messages node d' d'' d

14. 1 Local Fault Information Each Node Maintains ● Accusations A node accuses other nodes based on the messages it receives as well as indirect information.

15. 1 Local Fault Information Each Node Maintains ● Accusations A node accuses other nodes based on the messages it receives as well as indirect information. ● Convictions Periodically, the distributed diagnosis protocol is executed; nodes exchange accusations to produce convictions. ● NOTE: While a good node knows that all good nodes have the same convictions, it does not know that all good nodes have the same accusations.

16. 1 Local Fault Information Each Node Maintains ● Accusations A node accuses other nodes based on the messages it receives as well as indirect information. ● Convictions Periodically, the distributed diagnosis protocol is executed; nodes exchange accusations to produce convictions. ● NOTE: While a good node knows that all good nodes have the same convictions, it does not know that all good nodes have the same accusations. ● Eligible Voters For each BIU, the set of RMUs that it neither accuses nor convicts. Similarly for each RMU.

17. 1 Interactive Consistency Protocol External View ● Purpose: Reliably communicate data between processing elements (PEs) over the ROBUS. ROBUS PE

18. 1 Interactive Consistency Protocol External View ● A PE sends its data to the ROBUS. ROBUS PE sende r data in

19. 1 Interactive Consistency Protocol External View ● The IC Protocol is executed in the ROBUS. ROBUS PE...IC Protocol...

20. 1 Interactive Consistency Protocol External View ● The ROBUS broadcasts data back out to the PEs. ROBUS PE sende r data out...IC Protocol... data out

21. 1 Old Interactive Consistency Protocol Internal View BIU1 to PE BIU2 BIU3 RMU1 RMU2 sender data in ROBUS RMU3

22. 1 1. A BIU broadcasts data to the RMUs. If the BIU is good, the same value is broadcast to all RMUs. BIU1 to PE BIU2 BIU3 RMU1 RMU2 sender data in ROBUS data RMU3 data

23. 1 2. For each good RMU, if it receives data that isn't detectably faulty, then it passes the data received back to each BIU. Otherwise, source_error is sent. BIU1 to PE BIU2 BIU3 RMU1 RMU2 ROBUS RMU3 similarly for RMUs 2 and 3 data or source_err or data or source_err or data or source_err or RMU1 good

24. 1 3. Each BIU eliminates from its EV those RMUs that sent detectably faulty messages. BIU1 to PE BIU2 BIU3 RMU1 RMU2 ROBUS RMU3 2 1 3 BIUs 2 and 3 do likewise d d garbage RMU1 good RMU2 benign faulty

25. 1 4. For each BIU, it votes on the majority data sent from each RMU in its EV. BIU1 to PE BIU2 BIU3 RMU1 RMU2 ROBUS RMU3 2 1 3 BIUs 2 and 3 do likewise vote = d d d

26. 1 5. IF the majority of RMUs sent the same data, then it is sent to the BIU's PE. ELSE source_error is sent to the BIU's PE. BIU1 to PE BIU2 BIU3 RMU1 RMU2 ROBUS RMU3 BIUs 2 and 3 similarly send data vote = d d

27. 1 IC Protocol Guarantees ● Validity If the broadcasting BIU is good, not convicted, and sends data d, then the result of the vote for a good BIU is be d. ● Agreement Any two good BIUs vote the same result for the broadcasted value (even if the sender is asymmetric!).

28. 1 Old Assumptions to ensure guarantees hold Environment Assumptions The Maximum Fault Assumption (MFA): 1. There are more good BIUs than symmetric + asymmetric BIUs. 2. Similarly for the RMUs. 3. There are either no asymmetric BIUs or no asymmetric RMUs.

29. 1 Old Assumptions to ensure guarantees hold Environment Assumptions The Maximum Fault Assumption (MFA): 1. There are more good BIUs than symmetric + asymmetric BIUs. 2. Similarly for the RMUs. 3. There are either no asymmetric BIUs or no asymmetric RMUs. System Assumptions ● Symmetric Agreement If a node is not asymmetric, then all good nodes assign it the same accusation. ● Good Trusting Good nodes aren't accused by good nodes. ● Conviction Agreement All good nodes have the same convictions.

30. 1 Validity Proof Sketch Assume the broadcasting BIU is good and sends data d. BIU1 BIU2 BIU3 RMU1 RMU2 sender good ROBUS d d RMU3 d

31. 1 Validity Proof Sketch BIU1 BIU2 BIU3 RMU1 RMU2 ROBUS RMU3 similarly for RMUs 2 and 3 d d d Thus, all good RMUs send d back to the BIUs. RMU1 good

32. 1 Validity Proof Sketch BIU1 BIU2 BIU3 RMU1 RMU2 ROBUS RMU3 2 1 3 d d Each good BIU filters out the bad messages received. By the MFA, most of its EV then contains good RMUs. garbage similarly for BIUs 2 and 3

33. 1 Validity Proof Sketch BIU1 BIU2 BIU3 RMU1 RMU2 ROBUS RMU3 2 1 3 vote = d d d Since all good RMUs sent d, the result of the vote yields d. q.e.d.

34. 1 Agreement Proof Sketch BIU1 BIU2 BIU3 RMU1 RMU2 sender asym ROBUS d d'' RMU3 d' Either the broadcasting BIU is asymmetric or not. Suppose it is.

35. 1 Agreement Proof Sketch ROBUS BIU1 BIU2 BIU3 RMU1 RMU2 RMU3 Then no RMU is asymmetric, by the MFA. So every RMU sends the same data to every BIU. 2 1 3 x z y BIUs 2 and 3 receive the same values

36. 1 Agreement Proof Sketch ROBUS BIU1 BIU2 BIU3 RMU1 RMU2 RMU3 Since no RMU is asymmetric, by symmetric trusting, the EV of each BIU is the same. Thus, the result of the vote for each BIU is the same. 2 1 3 x z y BIUs 2 and 3 receive the same values

37. 1 Agreement Proof Sketch BIU1 BIU2 BIU3 RMU1 RMU2 sender not asym ROBUS d d RMU3 d For the other case, suppose the sending BIU is not asymmetric.

38. 1 Agreement Proof Sketch ROBUS BIU1 BIU2 BIU3 RMU1 RMU2 RMU3 Most of the RMUs are good, by the MFA. Since all good RMUs received the same values, they send the same values. RMU1 good RMU3 good BIU1 good BIU3 good x x

39. 1 Agreement Proof Sketch ROBUS BIU1 BIU2 BIU3 RMU1 RMU2 RMU3 By good trusting, no good BIU accuses a good RMU. Since most RMUs are good, there are a majority of good RMUs in the EV of each good BIU, after filtering benign RMUs. RMU1 good RMU3 good 2 1 3x 2 1 3x x x BIU1 good BIU3 good

40. 1 Agreement Proof Sketch ROBUS BIU1 BIU2 BIU3 RMU1 RMU2 RMU3 Thus, the result of the votes will be the same for all good BIUs. q.e.d. RMU1 good RMU3 good 2 1 3x 2 1 3x x x BIU1 good BIU3 good

41. 1 New Assumptions to reason about reintegration Environment Assumptions The Dynamic Maximum Fault Assumption (DMFA): 1. For each good BIU, its EV consists of more good RMUs than symmetric + asymmetric RMUs. 2. Similarly for good RMUs. 3. Either no asymmetric RMU is in the EV of a good BIU or no asymmetric BIU is in the EV of a good RMU.

42. 1 New Assumptions to reason about reintegration Environment Assumptions The Dynamic Maximum Fault Assumption (DMFA): 1. For each good BIU, its EV consists of more good RMUs than symmetric + asymmetric RMUs. 2. Similarly for good RMUs. 3. Either no asymmetric RMU is in the EV of a good BIU or no asymmetric BIU is in the EV of a good RMU. System Assumptions ● Symmetric Agreement If a node is not asymmetric, then all good nodes assign it the same accusation. ● Good Trusting Good nodes aren't accused by good nodes. ● Conviction Agreement All good nodes have the same convictions.

43. 1 Agreement Breaks! Under the New Assumptions (courtesy of Wilfredo) ROBUS BIU1 BIU2 BIU3 RMU1 RMU2 RMU3 Suppose the sender is asymmetric, but is in no EV of all good RMUs. Suppose there is an asymmetric RMU in the EV of both good BIUs. This satisfies the DMFA. asym good & accuses BIU2 good & accuses BIU2 sender asym d d' d'' good & trusts all

44. 1 Agreement Breaks! Under the New Assumptions ROBUS BIU1 BIU2 BIU3 RMU1 RMU2 RMU3 The two good RMUs relay the values received, and since RMU3 can relay arbitrary data, it sends d to BIU1 and d' to the other. asym good & accuses BIU2 good & accuses BIU2 sender asym 2 1 3 d d 2 1 3 d d' good & trusts all d'

45. 1 Agreement Breaks! Under the New Assumptions ROBUS BIU1 BIU2 BIU3 RMU1 RMU2 RMU3 The result of the votes of BIU1 and BIU2 differ. Agreement is violated! asym good & accuses BIU2 good & accuses BIU2 sender asym 2 1 3 d d 2 1 3 d d' good & trusts all d' vote = d vote = d'

46. 1 Revised IC Protocol In the new IC Protocol, the RMUs relay source_error when ● They receive bad messages and ● They accuse the sender.

47. 1 Revised IC Protocol In the new IC Protocol, the RMUs relay source_error when ● They receive bad messages and ● They accuse the sender. The revised IC protocol satisfies both validity and agreement (verified in PVS).

48. 1 Formal Verification Why Level 3 Verification? ● A math proof is proof enough, right? ● Level 3 verification can require significant time to complete. In other words...

49. 1 Using PVS

50. 1 Formal Verification Why Level 3 Verification? ● A math proof is proof enough, right? ● Level 3 verification can require orders of magnitude more time to complete than level 1 or level 2 verification. But... ● Proofs for fault-tolerant protocols for distributed architectures are tedious and large (there are nearly 400 lemmas & theorems in our current unfinished set of proofs). ● Proofs are not checked by a community of mathematicians like other mathematical results are. In other words...

51. 1 You don't have to be a Laurel or Hardy to make an oversight in an informal proof. Small changes in assumptions can obviate guarantees.

52. 1 Some Goals & Current Work in verifying SPIDER ● Robust Specifications/Proofs ● Hold for arbitrary configurations of SPIDER ● Hold for all accusation & conviction policies satisfying the system requirements

53. 1 Some Goals & Current Work in verifying SPIDER ● Robust Specifications/Proofs ● Hold for arbitrary configurations of SPIDER ● Hold for all accusation & conviction policies satisfying the system requirements ● Specification/Proof “Reuse” (Economic specs/proofs)

54. 1 Some Goals & Current Work in verifying SPIDER ● Robust Specifications/Proofs ● Hold for arbitrary configurations of SPIDER ● Hold for all accusation & conviction policies satisfying the system requirements ● Specification/Proof “Reuse” (Economic specs/proofs) ● Specification/Proof Hierarchy ● Property specifications ● Relational specifications ● Functional composition specifications ● State machine specifications

55. 1 References ● SPIDER Homepage: http://shemesh.larc.nasa.gov/fm/fm-now-spider.html. ● PVS Homepage: http://pvs.csl.sri.com/. ● Butler, Ricky et al. NASA Langley's Research and Technology-Transfer Program in Formal Methods. 2000. Available athttp://shemesh.larc.nasa.gov/fm/fm- welcome.html. ● Rushby, John. Formal Methods and Digital Systems Validation for Airborne Systems. NASA Contractor Report 4551. 1993. Available at: http://www.csl.sri.com/papers/csl-93-7/.