1. Anatomy of 4GL Disaster CS524 - Software Engineering I Fall I, 2007 - Sheldon X. Liang, Ph.D. Nathan Scheck CS524 - Software Engineering I Fall I, 2007 - Sheldon X. Liang, Ph.D. Nathan Scheck

2. Sources  Object-Oriented & Classical Engineering, 7th Edition –Stephen R. Schach  Software Runaways Lessons Learned from Massive Software Project Failures –Robert L. Glass  Wikipedia.org  Object-Oriented & Classical Engineering, 7th Edition –Stephen R. Schach  Software Runaways Lessons Learned from Massive Software Project Failures –Robert L. Glass  Wikipedia.org

3. Introduction  In 1985, an upgrade to the New Jersey DMV database system was activated.  It was written using “Ideal”, a new 4GL.  Hundreds of thousands of errors resulted, along with that many dollars spent in fixing them.  The only viable solution was to rewrite some of the core components using Cobol.  In 1985, an upgrade to the New Jersey DMV database system was activated.  It was written using “Ideal”, a new 4GL.  Hundreds of thousands of errors resulted, along with that many dollars spent in fixing them.  The only viable solution was to rewrite some of the core components using Cobol.

4. The Real Question  Anatomy of a 4GL disaster?  Anatomy of a human disaster?  Anatomy of a 4GL disaster?  Anatomy of a human disaster?

5. What Happened?

6. The DMV Needs an Upgrade  In 1983, the New Jersey DMV knew their DB systems needed upgrading.  Two options for implementation:  Price Waterhouse (consulting firm)  DMV’s Division of Systems and Communications (SAC)  In 1983, the New Jersey DMV knew their DB systems needed upgrading.  Two options for implementation:  Price Waterhouse (consulting firm)  DMV’s Division of Systems and Communications (SAC)

7. Implementation Option: Price Waterhouse  Completely new system.  $6.5 million, plus $8.5 million for hardware.  DMV Director and deputy (Snedeker and Kline) support Price Waterhouse.  “Price Waterhouse’s greatest strength is in project management and control.”  Price Waterhouse would be involved in the design anyway, so why not let them implement it?  Completely new system.  $6.5 million, plus $8.5 million for hardware.  DMV Director and deputy (Snedeker and Kline) support Price Waterhouse.  “Price Waterhouse’s greatest strength is in project management and control.”  Price Waterhouse would be involved in the design anyway, so why not let them implement it?

8. Implementation Option : Price Waterhouse  Can get things done faster than SAC.  SAC thought not to have enough qualified personnel.  “[SAC’s] greatest weakness is project management and control.”  Can get things done faster than SAC.  SAC thought not to have enough qualified personnel.  “[SAC’s] greatest weakness is project management and control.”

9. Implementation Option: DMV’s SAC  Update current systems to work more effectively, and add new modules.  $2 million, plus $3 million for hardware.  Price Waterhouse could help design.  SAC employees are already familiar with the systems.  Price Waterhouse does not have implementation experience.  Update current systems to work more effectively, and add new modules.  $2 million, plus $3 million for hardware.  Price Waterhouse could help design.  SAC employees are already familiar with the systems.  Price Waterhouse does not have implementation experience.

10. An External Opinion  The governor’s office asked for advice from the Science Management Corp. (SMC), another consulting firm with knowledge of the DMV’s systems.  They recommended against Price Waterhouse taking charge of the whole project.  Price Waterhouse would only be involved in the design phase.  (The federal government actually requires that the design firm not be involved in implementation.)  The governor’s office asked for advice from the Science Management Corp. (SMC), another consulting firm with knowledge of the DMV’s systems.  They recommended against Price Waterhouse taking charge of the whole project.  Price Waterhouse would only be involved in the design phase.  (The federal government actually requires that the design firm not be involved in implementation.)

11. The Verdict  Price Waterhouse chosen.  Why?  Nobody is exactly sure.  They wanted it done “quickly”.  Price Waterhouse made a few generous political contributions.  Project manager: A lawyer.  Price Waterhouse chosen.  Why?  Nobody is exactly sure.  They wanted it done “quickly”.  Price Waterhouse made a few generous political contributions.  Project manager: A lawyer.

12. Implementation: Plan  “Ideal” to be the primary programming language.  4GL  Introduced eight months earlier by ADR.  Price Waterhouse’s contract stated that they had evaluated Ideal to determine that it was suited for the job.  “Ideal” to be the primary programming language.  4GL  Introduced eight months earlier by ADR.  Price Waterhouse’s contract stated that they had evaluated Ideal to determine that it was suited for the job.

13. Conceptual Problems: Speed  Ideal is a 4GL.  Simplifies coding at the expense of processing speed.  More than three times slower than Cobol.  Fine under the right circumstances, but not good for heavy multi-user data processing.  Ideal lacks DB indexing.  Ideal is a 4GL.  Simplifies coding at the expense of processing speed.  More than three times slower than Cobol.  Fine under the right circumstances, but not good for heavy multi-user data processing.  Ideal lacks DB indexing.

14. Conceptual Problems: Interfacing  Ideal does not allow computer-to- computer interfacing.  The DMV project required communication between over 50 computers.  “Price Waterhouse never seemed to grasp the issue. It didn’t seem to realize the system would not run on a dedicated DMV machine.”  Ideal does not allow computer-to- computer interfacing.  The DMV project required communication between over 50 computers.  “Price Waterhouse never seemed to grasp the issue. It didn’t seem to realize the system would not run on a dedicated DMV machine.”

15. Implementation: Problems  16 months from deadline - Price Waterhouse: “The uncertainties associated with the use of Ideal represent an acceptable risk.”  15 months - “The support necessary from ADR is slow and difficult to obtain.”  14 months - ADR gets copy of DB design, says nothing about potential problems.  16 months from deadline - Price Waterhouse: “The uncertainties associated with the use of Ideal represent an acceptable risk.”  15 months - “The support necessary from ADR is slow and difficult to obtain.”  14 months - ADR gets copy of DB design, says nothing about potential problems.

16. Implementation: Problems  12 months - “The satisfactory resolution of ADR-related technical problems would remove a major issue threatening timely completion of the project.”  9 months - ADR informs Price Waterhouse and SAC of two potential problems:  Slow response times  Limitation on number of online users  8 months - System tests show transactions taking more than twice as long as acceptable.  12 months - “The satisfactory resolution of ADR-related technical problems would remove a major issue threatening timely completion of the project.”  9 months - ADR informs Price Waterhouse and SAC of two potential problems:  Slow response times  Limitation on number of online users  8 months - System tests show transactions taking more than twice as long as acceptable.

17. Implementation: Problems  7 months  Price Waterhouse outlines plan to limit number of online terminals to 200, rather than 1,000.  A few modules will be delayed, and some of the code will be written in Cobol.  7 months  Price Waterhouse outlines plan to limit number of online terminals to 200, rather than 1,000.  A few modules will be delayed, and some of the code will be written in Cobol.

18. Launch!  All old systems turned off. (PW did not plan for enough processor power to run in parallel.)  New systems turned on.  Log on could take one hour.  Response times could take three minutes.  Nightly batch updates could take days.  All old systems turned off. (PW did not plan for enough processor power to run in parallel.)  New systems turned on.  Log on could take one hour.  Response times could take three minutes.  Nightly batch updates could take days.

19. Epilogue  After a lawsuit, Price Waterhouse agreed to convert as much of the system as necessary to Cobol at their expense.  The update was still not complete as of six months later.  After a lawsuit, Price Waterhouse agreed to convert as much of the system as necessary to Cobol at their expense.  The update was still not complete as of six months later.

20. Summary  What went wrong?  Wrong tool for the job (4GL for heavy data processing)  Overly-optimistic management  Ignoring warning signs  Lack of quality assurance reviews or early feasibility testing.  What went wrong?  Wrong tool for the job (4GL for heavy data processing)  Overly-optimistic management  Ignoring warning signs  Lack of quality assurance reviews or early feasibility testing.

21. Summary  “It’s a story of an organization and individuals abandoning established management and data processing practices and, it appears, common sense, in a rush to collect on the promise of the new generation of programming languages.”