1. Programmable Logic Devices
Building the Case for
Software-style Assurance
Kalynnda Berens
July 30, 2003
NASA Glenn Research Center

2. What is Programmable Logic
Programmable Logic Controllers (PLC)
Programmable Logic Devices
Field Programmable Gate Array (FPGA)
Application Specific Integrated Circuit (ASIC)
System-on-chip (SOC)
Complex PLD (CPLD)
others
July 30, 2003
NASA Glenn Research Center

3. The Hardware/Software Boundary
BIOS/bootstrap
Operating system
Applications
Programmed
Easily changed
Can “do anything”
Cannot be 100%, exhaustively tested
Programmable Logic Controllers
Special purpose computer (process control)
Uses LadderLogic, other languages for programming
Firmware
Software residing in non-volatile storage
SOC
Reconfig. Computing
Programmable Logic Devices
FPGA
CPLD
PAL
ASIC
Designed with HDL
Compiled/Programmed
May be reprogrammable in the field
Cannot be 100%, exhaustively tested
Electronic Hardware
ICs
Microprocessor
A/D, D/A
Sensors
Off-the-shelf components
Exhaustively Tested by Vendor
July 30, 2003
NASA Glenn Research Center

4. SAIC @ NASA Glenn Research Center
Pushing the Limits
System-on-Chip (SOC)
Combine microprocessor/input/output, often FPGA for programmability
Reconfigurable Computing
Morphware, Configware, Flowware
In NASA Strategic Technology Plan
FPGAs
30,000 to over a million gates
Complex interactions
July 30, 2003
NASA Glenn Research Center

5. SAIC @ NASA Glenn Research Center
Complexity
Types of faults
Incomplete specifications
Design and Implementation Errors (Common mode)
Unexpected or unanticipated combinations of valid operating states.
Unintended interactions
Unknown defects in tools (design or verification)
July 30, 2003
NASA Glenn Research Center

6. Hardware/Software Differences
Most PL cannot be changed once “burned” (programmed). FPGAs can be programmed on-the-fly.
Software execution is serial – one instruction after another
PL execution is parallel – multiple simultaneous signals and processes
PL designed, verified, tested by engineers
July 30, 2003
NASA Glenn Research Center

7. Assurance: Product and Process
Activity
Product
Process
Eng. QA
Requirements Specification X R
Design Documentation
Requirements, Design Analyses
Inspections, Walkthroughs
Simulation W
Testing
Planning (Risk, Management, Development, QA)
Configuration Management
Audits
July 30, 2003
NASA Glenn Research Center

8. SAIC @ NASA Glenn Research Center
Current PL Process
Design from system requirements
Functional Simulation
Includes “corner cases”
Testing (unit and system)
Simulation and unit test usually performed by design engineer
May perform code coverage measurement
Verification takes 70% of design task
July 30, 2003
NASA Glenn Research Center

9. NASA PL Assurance Activities – from the user’s point of view
Yes No
Review source 15 31
Witness Programming 9 38
Witness Testing 16
Verify Version 12 34
Audit development 11
Audit CM 32
July 30, 2003
NASA Glenn Research Center

10. NASA PL Assurance Activities – from QA’s point of view
Project SA
Other QA
Other
None
PL Testing 9 1 11
Test Witness 6 2 3 22
Code Review 5
Witness Burn
CM Audit 4
Devel. Audit
FCA
PCA
VDD
Safety Verif. 8 32
1 Vendor or contractor 2 Safety personnel
July 30, 2003
NASA Glenn Research Center

11. SAIC @ NASA Glenn Research Center
What are others doing?
Hardware/software co-verification
Industry/Military practices still open issue – tough nut to crack
ESA – starting to address FPGA/ASIC through reports and guidance
FAA – DO-254 for Complex Electronic Hardware, calls for design process assurance
July 30, 2003
NASA Glenn Research Center

12. PL-related Standards and Guidelines
IEC “Functional Safety of Electrical/Electronic/ Programmable Electronic Safety-related Systems”
DO-254 – “Design Assurance Guidelines for Airborne Electronic Hardware”
IEC – “PLC Programming Languages”
IEC – “Functional Safety - Safety Instrumented Systems For The Process Industry Sector”
IEE SEMSPLC – “Software Engineering Methods for Safe Programmable Logic Controllers”
July 30, 2003
NASA Glenn Research Center

13. PL-related Standards and Guidelines (continued)
European Space Agency
Space Product Assurance – ASIC Development
VHDL Modeling Guidelines
FPGA report
EWICS TC7 Guidelines for the use of Programmable Logic Controllers in Safety-related Systems
July 30, 2003
NASA Glenn Research Center

14. SAIC @ NASA Glenn Research Center
FAA and DO-254
Complex Electronic Hardware includes FPGA, CPLD, and ASIC
DO-254 required for Levels A and B (highest criticality)
Defines Hardware Life-Cycle Processes:
Planning Process
Hardware Design Processes
Requirements, Design, Implementation, Production, Test
Validation Process
Verification Process
Configuration Management Process
Process Assurance
Certification Liaison Process
July 30, 2003
NASA Glenn Research Center

15. SAIC @ NASA Glenn Research Center
DO-254 at Langley
Case study on applying DO-254 to SPIDER1
Implemented process assurance
monitoring the development activities to assure they are in accordance to plans
Placed conceptual design in CM
Used Formal Methods
1 Scalable Processor-Independent Design for Electromagnetic Resilience
July 30, 2003
NASA Glenn Research Center

16. FPGA Lessons Learned European Space Agency, 2002
Reviewed FPGA’s in ESA/NASA missions
Extensive use in critical systems with little thought to SEU
Design and verification by same individual
Insufficient verification due to inadequate stimuli selection
Test only – simulation often skipped
Non-engineers “blessing” design
FPGA’s are “the software of the hardware world”
Encourage engineers to quickly get to hardware test, ignoring good design practice
July 30, 2003
NASA Glenn Research Center

17. Safety-Related Complex Electronic Systems, 2000
Simulation alone is not adequate. Exhaustive list of possible failures not possible.
Strengthen system/subsystem tests
Consider origin of faults
Errors of specification, design, production
Internal faults
External faults
Quality of vendor-supplied soft core or macro libraries is not guaranteed
Synthesis tools can generate faults
High fault coverage in test is mandatory
July 30, 2003
NASA Glenn Research Center

18. SAIC @ NASA Glenn Research Center
What do we know?
PLCs use software, just the purpose and language differ
FPGAs and other Programmable Logic devices are very complex
Process assurance provides additional value in conjunction with product assurance
Process assurance currently not applied to most PLD development
July 30, 2003
NASA Glenn Research Center

19. SAIC @ NASA Glenn Research Center
What don’t we know?
Industry and Military QA Best Practices
What level of process assurance is required
Who should do QA on programmable logic
Hardware QA
More likely to understand PL or quickly learn
Need to learn process assurance activities
Software QA
Familiar with process assurance
Would need to learn PL hardware and language
How to integrate process assurance in NASA
Software CMM implementation may provide guide
July 30, 2003
NASA Glenn Research Center

20. Software->Hardware Assurance
Inspection of HDL code and schematics
Validated as low-cost, high-probability of catching errors for hardware
Walkthroughs
Independent test team
Formal Methods
Complexity measurements
Traceability
Change impact analysis
CM tools and processes
Functional, code coverage analysis
QA monitoring of development process
July 30, 2003
NASA Glenn Research Center

21. Hardware->Software Assurance
Simulation, test beds are standard operating procedure
Testing against boundary conditions (“corner cases”)
Wide variety of available tools for verification
July 30, 2003
NASA Glenn Research Center

22. SAIC @ NASA Glenn Research Center
Next steps
Goal is not to provide the answers to how PL is assured, but to set the parameters for constructive discussion within NASA and provide a common information base
Issue Paper on this topic
Process Assurance guidance for Hw QA
PL/Hardware guidance for Sw QA
July 30, 2003
NASA Glenn Research Center

23. SAIC @ NASA Glenn Research Center
Please Take the Survey!
If you have industry/military QA or engineering contacts, please me at:
July 30, 2003
NASA Glenn Research Center