1. PowerPoint Presentation: Richard H. Baum, Ph.D. DeVry Institute of Technology 9th Edition Structured COBOL Programming Nancy Stern Hofstra University Robert A. Stern Nassau Community College “Copyright @ 2000 John Wiley & Sons, In. All rights reserved. Reproduction or translation of this work beyond that permitted in Section 117 of the 1976 United States Copyright Act without the express permission of the copyright owner is unlawful. Request for further information should be addressed to the permissions Department, John Wily & Sons, Inc. The purchaser may make back-up copies for his/her own use only and not for distribution or resale. The Publisher assumes no responsibility for errors, omissions, or damages, caused by the use of these programs or from the use of the information contained herein.”

2. Structured COBOL Programming, Stern & Stern, 9th Edition CHAPTER 13 Sequential File Processing

3. Structured COBOL Programming, Stern & Stern, 9th Edition OBJECTIVES To familiarize you with: 1. Master file processing concepts. 2. Sequential update procedures using disk as a master file. 3. How sequential disk files may be updated in place with a REWRITE statement.

4. Structured COBOL Programming, Stern & Stern, 9th Edition CONTENTS Systems Overview of Sequential File Processing –Master Files –Typical Master File Procedures: A Systems Overview Sequential File Updating - Creating a New Master File –The Files Used –The Ordering of Records for Sequential Updates –The Procedures Used for Sequential Updates –Self-Test

5. Structured COBOL Programming, Stern & Stern, 9th Edition CONTENTS Validity Checking in Update Procedures –Checking for New Accounts –Checking for Delete Codes and Deleting Records from a Sequential Master File –Checking for Sequence Errors Update Procedures with Multiple Transaction Records for Each Master Record

6. Structured COBOL Programming, Stern & Stern, 9th Edition CONTENTS Sequential File Updating - Rewriting Records on a Disk –The REWRITE Statement for a Disk File Opened as I-O –Using an Activity-Status Field for Designating Records to Be Deleted –The EXTEND Option for Adding Records to the End of a Sequential File

7. Structured COBOL Programming, Stern & Stern, 9th Edition SYSTEMS OVERVIEW OF SEQUENTIAL FILE PROCESSING

8. Structured COBOL Programming, Stern & Stern, 9th Edition Master Files In most companies, a master file will be stored on a magnetic disk. The features of magnetic media such as disk that make it ideally suited for storing master file data include the following: 1. Disks can store billions of characters. 2. Disk drives can read and write data very quickly. 3. Disk records can be any size.

9. Structured COBOL Programming, Stern & Stern, 9th Edition Master Files Although tapes were once used for storing files, most companies today use disks because disks are more versatile. A tape is more often used for backing up a file in case the original disk file becomes unusable. –One main reason why tapes are not as prevalent as disks for master file processing is that tapes can be accessed only sequentially, thus limiting their usefulness.

10. Structured COBOL Programming, Stern & Stern, 9th Edition Master Files Disk files, however, can be organized for either sequential or random access. If a master file is always processed sequentially, it may be stored on either disk or tape. But if a file needs to be read, or processed, randomly, or in some sequence that cannot be predetermined, then it would be stored on disk and organized for random access.

11. Structured COBOL Programming, Stern & Stern, 9th Edition Typical Master File Procedures

12. Structured COBOL Programming, Stern & Stern, 9th Edition Designing a Master File The following are elements to consider when designing a master file: 1. The first field or fields should be key fields that uniquely identify the record. 2. Where possible, key fields should consist of numbers (e.g., Social Security Number rather than Employee Name, Part Number rather than Part Description, etc.). 3. Secondary key fields (e.g., Name) should follow primary key fields in a record.

13. Structured COBOL Programming, Stern & Stern, 9th Edition Designing a Master File 4. Fields should appear in a master file in order of importance (e.g., Employee Name and Address before Birth Date, etc.). 5. Be sure that fields are large enough to accommodate the data (e.g., a 10- position Last Name field, for example, is not likely to be large enough). 6. Use coded fields where possible to save space (e.g., codes are used for Marital Status, Insurance Plan Option, etc.).

14. Structured COBOL Programming, Stern & Stern, 9th Edition Creating a Master File When a new system is implemented, or used for the first time, a master file must be initially created. –This procedure can be performed by entering all master file data using a keyboard or other data entry device. The data is then recorded on a disk, or tape, which becomes the new master file. –Creating a master file is a one-time procedure. That is, once the master file is created, changes to it are made by a different procedure.

15. Structured COBOL Programming, Stern & Stern, 9th Edition Creating a Master File The primary objective of a program that creates a master file is ensuring data integrity. A master file is only useful if it contains valid and reliable data; hence, a creation program must be designed so that it minimizes input errors. When a master file is created, a control listing or audit trail is also produced that prints the data stored on the new master file, and necessary control totals.

16. Structured COBOL Programming, Stern & Stern, 9th Edition Creating a Transaction File After a master file is created, a separate procedure is used to make changes to it. Change records are stored in a file referred to as a transaction file, which is also typically stored on disk. –Changes to an accounts receivable master file or changes to a payroll master file would usually be stored in a disk transaction file. File design considerations, as specified for master files, should be applied to the transaction file as well. –The transaction file should be validated to ensure data integrity.

17. Structured COBOL Programming, Stern & Stern, 9th Edition Updating a Master File The process of making a master file current is referred to as updating. –The master file is updated by incorporating the changes from the transaction records. This chapter emphasizes techniques used for performing sequential updates for master files. – Sequential updates that process transaction records stored in sequence in a file, rather than entered randomly and processed interactively, are accomplished via batch processing.

18. Structured COBOL Programming, Stern & Stern, 9th Edition Reporting from a Master File The purpose of maintaining a master file is to store data that will provide users with meaningful output. –Such output, which is generally in the form of reports are frequently scheduled, which means they are prepared on a regular basis. Sales reports, customer bills, and payroll checks are examples of output from master files that are prepared on a regularly scheduled basis. Reports can also be prepared on demand. That is, they are requested and produced whenever the need arises.

19. Structured COBOL Programming, Stern & Stern, 9th Edition SEQUENTIAL FILE UPDATING: CREATING A NEW MASTER FILE

20. Structured COBOL Programming, Stern & Stern, 9th Edition The File Used Master files that have been designed and created for sequential processing can be updated by reading in the master file along with a transaction file and creating a new master file. –This means that the sequential update procedure uses three files. Most often, a fourth print file that produces a control listing or audit trail is created as well for printing all changes made, any errors found, and totals. –Any program written to update a master file should create a control listing that specifies all changes made to the master.

21. Structured COBOL Programming, Stern & Stern, 9th Edition The Files Used Input Master File The input master file is current through the previous updating period. That is, if updates are performed weekly, the input master file is the file that was created the previous week as the master. We call this file OLD-MASTER because it does not contain changes that have occurred since the previous update.

22. Structured COBOL Programming, Stern & Stern, 9th Edition The Files Used Input Transaction File The transaction file contains data to be used for updating the master file that we call OLD-MASTER. The input transaction file contains all changes that have occurred since OLD- MASTER was created. We call this file TRANS-FILE.

23. Structured COBOL Programming, Stern & Stern, 9th Edition The Files Used Output Master File The output master file becomes the new master as a result of the updating procedure. The output master file will integrate data from the OLD-MASTER and the TRANS- FILE. We call this file NEW-MASTER. Note that for the next week's update run, this NEW-MASTER becomes OLD-MASTER.

24. Structured COBOL Programming, Stern & Stern, 9th Edition The Files Used Control Listing or Audit Trail Recall that a print file or control listing is usually created during a sequential file update. Such a print file would list: (1) changes made to the master file (2) errors encountered during processing (3) totals to be used for control and checking purposes.

25. Structured COBOL Programming, Stern & Stern, 9th Edition The Ordering of Records for Sequential Updates OLD-MASTER contains master information that was complete and current through the previous updating cycle. The TRANS-FILE contains transactions or changes that have occurred since the previous updating cycle. –These transactions or changes must be incorporated into the master file to make it current.

26. Structured COBOL Programming, Stern & Stern, 9th Edition The Ordering of Records for Sequential Updates The NEW-MASTER will include all OLD- MASTER data in addition to the changes stored on the TRANS-FILE that have occurred since the last update. The NEW-MASTER will be on the same medium (e.g., disk) as the OLD-MASTER, since the current NEW- MASTER becomes the OLD-MASTER for the next update cycle.

27. Structured COBOL Programming, Stern & Stern, 9th Edition The Ordering of Records for Sequential Updates In a sequential master file, all records are in sequence by a key field, depending on the type of master file, such as account number or social security number) –This key field uniquely identifies each master record. We compare the key field in the master to the same key field in the transaction file to determine if the master record is to be updated; –This comparison requires both files to be in sequence by the key field.

28. Structured COBOL Programming, Stern & Stern, 9th Edition The Use of HIGH-VALUES for End-of-File Conditions. HIGH-VALUES means ``all bits on,'' which is not a printable character and which has a nonnumeric value greater than all 9s. Using it on an end-of-file condition will ensure correct file handling regardless of the actual values that the account numbers can assume. This is because an incoming account number will always compare ``less than'' HIGH-VALUES.

29. Structured COBOL Programming, Stern & Stern, 9th Edition QUESTIONS?

30. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 1. What do we call the major collection of data pertaining to a specific application? Solution: A master file

31. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 2. Changes to be made to a master file are placed in a separate file called a ____ file. Solution: transaction

32. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 3. In a sequential update procedure, we use three files called ____, ____, and ____. Solution: the old master file - current through the previous updating cycle; the transaction file; the new master file - which incorporates the old master data along with the transaction data

33. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 4. ( T or F ) In a sequential update procedure, all files must be in sequence by the same key field. Solution: T

34. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 5. In a sequential update procedure, the key field in the transaction file is compared to the key field in the ____. Solution: old master file

35. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 6. In Question 5, if the key fields are equal, a ____ procedure is performed. Describe this procedure. Solution: regular update; transaction data is added to the master data, a new master record is written, and records from the old master file and the transaction file are read.

36. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 7. In Question 5, if the transaction key field is greater than the master key field, a ____ procedure is performed. Describe this procedure. Solution: no update; a new master record is created directly from the old master record and a record from the old master file is read.

37. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 8. In Question 5, if the transaction key field is less than the master key field, a ____ procedure is performed. Describe this procedure. Solution: new account or error; if it is a new account, move transaction data to the new master and write; if it is an error, an error message is printed. In either case, a transaction record is then read.

38. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 9. The statement READ TRANS-FILE AT END MOVE HIGH-VALUES TO T-ACCT- NO moves ____ if there are no more records in TRANS-FILE. To contain HIGH-VALUES, T-ACCT-NO must be defined with a PIC of ____. Solution: a value of all bits on to the T- ACCT-NO field. (Any subsequent comparison of T-ACCT-NO to an actual M- ACCT-NO will cause a ``>'' condition); Xs because HIGH-VALUES is a nonnumeric figurative constant.

39. Structured COBOL Programming, Stern & Stern, 9th Edition SELF-TEST 10. ( T or F ) Disk files can be created for either sequential or random access. Solution: T

40. Structured COBOL Programming, Stern & Stern, 9th Edition VALIDITY CHECKING IN UPDATE PROCEDURES

41. Structured COBOL Programming, Stern & Stern, 9th Edition Checking for New Accounts To verify that a TRANS-REC is a new account, we usually include a coded field in the TRANS-REC itself that specifies the record as a new account. For example: 1-5 T-ACCT-NO 6-11 AMT-TRANS-IN-CURRENT-PER 9999V99 12-99 FILLER 100 CODE-IN (1 = NEW-ACCT; 2 = REGULAR- UPDATE) –Thus, if T-ACCT-NO IS LESS THAN M-ACCT- NO, we would process the transaction record as a new account only if it also contains a 1 in CODE-IN.

42. Structured COBOL Programming, Stern & Stern, 9th Edition Checking for Delete Codes and Deleting Records from a Sequential Master File One type of update function not considered in our previous illustrations is that of deleting master records. Since accounts may need to be deleted if customers give up their charge privileges or have not paid their bills, there must be some provision for eliminating specific records from the master file during an update.

43. Structured COBOL Programming, Stern & Stern, 9th Edition Checking for Delete Codes and Deleting Records from a Sequential Master File We may use the technique of a coded transaction field as described earlier to accomplish this. We could add a code of '3' to indicate that a record is to be deleted. For example: 1-5 T-ACCT-NO 6-11 AMT-TRANS-IN-CURRENT-PER 9999V99 12-99 FILLER 100 CODE-IN (1 = NEW-ACCT; 2 = UPDATE- CODE; 3 = DELETE-THE-RECORD)

44. Structured COBOL Programming, Stern & Stern, 9th Edition SUMMARY: How Transaction Records Are Processed A. T-KEY = M-KEY 1. Delete the master record if T-CODE indicates deletion. 2. Change or update the master record if T- CODE indicates update. 3. Process the transaction record as an error if T-CODE indicates new record.

45. Structured COBOL Programming, Stern & Stern, 9th Edition SUMMARY: How Transaction Records Are Processed B. T-KEY < M-KEY 1. Add the transaction record to the master file if T-CODE indicates a new record. 2. Process the transaction record as an error if T-CODE does not indicate a new record. C. T-KEY > M-KEY 1. Rewrite the master record as is.

46. Structured COBOL Programming, Stern & Stern, 9th Edition UPDATE PROCEDURES WITH MULTIPLE TRANSACTION RECORDS FOR EACH MASTER RECORD

47. Structured COBOL Programming, Stern & Stern, 9th Edition Update Procedures for Multiple Transactions The following is the Procedure Division for updating a file where multiple transactions per master record are permitted. 100-MAIN-MODULE. PERFORM 800-INITIALIZATION-RTN PERFORM 200-COMP-RTN UNTIL M-ACCT-NO-IN = HIGH-VALUES AND T-ACCT-NO-IN = HIGH-VALUES PERFORM 900-END-OF-JOB-RTN STOP RUN.

48. Structured COBOL Programming, Stern & Stern, 9th Edition Update Procedures for Multiple Transactions 200-COMP-RTN. EVALUATE TRUE WHEN T-ACCT-NO-IN = M-ACCT-NO-IN PERFORM 300-REGULAR-UPDATE WHEN T-ACCT-NO-IN < M-ACCT-NO-IN PERFORM 400-NEW-ACCOUNT WHEN OTHER PERFORM 500-NO-UPDATE END-EVALUATE.

49. Structured COBOL Programming, Stern & Stern, 9th Edition Update Procedures for Multiple Transactions 300-REGULAR-UPDATE. MOVE OLD-MASTER-REC-IN TO NEW-MASTER-REC-OUT PERFORM 550-ADD-AND-READ-TRANS UNTIL T-ACCT-NO-IN NOT = M-ACCT-NO-IN WRITE NEW-MASTER-REC-OUT PERFORM 600-READ-MASTER.

50. Structured COBOL Programming, Stern & Stern, 9th Edition Update Procedures for Multiple Transactions 400-NEW-ACCOUNT. MOVE SPACES TO NEW-MASTER-REC-OUT MOVE T-ACCT-NO-IN TO ACCT-NO-OUT MOVE AMT-TRANS-IN TO AMOUNT-DUE-OUT WRITE NEW-MASTER-REC-OUT PERFORM 700-READ-TRANS. 500-NO-UPDATE. WRITE NEW-MASTER-REC-OUT FROM OLD-MASTER-REC-IN PERFORM 600-READ-MASTER. 550-ADD-AND-READ-TRANS. ADD AMT-TRANS-IN TO AMOUNT-DUE-OUT PERFORM 700-READ-TRANS

51. Structured COBOL Programming, Stern & Stern, 9th Edition Update Procedures for Multiple Transactions 600-READ-MASTER. READ OLD-MASTER-IN AT END MOVE HIGH-VALUES TO M-ACCT-NO-IN END-READ. 700-READ-TRANS. READ TRANS-FILE-IN AT END MOVE HIGH-VALUES TO T-ACCT-NO-IN END-READ.

52. Structured COBOL Programming, Stern & Stern, 9th Edition Update Procedures for Multiple Transactions 800-INITIALIZATION-RTN. OPEN INPUT OLD-MASTER-IN TRANS-FILE-IN OUTPUT NEW-MASTER-OUT PERFORM 600-READ-MASTER PERFORM 700-READ-TRANS. 900-END-OF-JOB-RTN. CLOSE OLD-MASTER-IN TRANS-FILE-IN NEW-MASTER-OUT.

53. Structured COBOL Programming, Stern & Stern, 9th Edition The Balanced Line Algorithm for Sequential File Updating Another alternative for updating a master file with any number of transactions and checks for a variety of error conditions is a technique called the balanced line algorithm. This method is viewed by many as the most efficient and effective sequential file updating method. –The version of the balanced line algorithm illustrated in the text has been described by Barry Dwyer in an article entitle "One More Time--How to Update a Master File" ACM, vol. 24, no.1 (January, 1981), pp. 3-8

54. Structured COBOL Programming, Stern & Stern, 9th Edition SEQUENTIAL FILE UPDATING: REWRITING RECORDS ON A DISK

55. Structured COBOL Programming, Stern & Stern, 9th Edition The REWRITE Statement for a Disk File Opened as I-O Both disk and tape can be organized sequentially and both can use sequential update procedures. Disks, however, unlike tape, can serve as both input and output during the same run. Thus, it is possible to read a disk record, make changes directly to the same record, and rewrite it or update it in place. –A disk file, for example MASTER FILE, can be opened as I-O, which means records from the disk will be accessed, read, changed, and rewritten: OPEN I-O MASTER-FILE

56. Structured COBOL Programming, Stern & Stern, 9th Edition Using an Activity-Status Field for Designating Records to be Deleted If a sequential disk file is to be updated by rewriting records directly on it, we need a different procedure for deleting records. One common technique is to begin each record with a one-character activity- status code that precedes the key field. –The activity-status code would have a specific value for active records, and a different value if the record is to be deactivate.

57. Structured COBOL Programming, Stern & Stern, 9th Edition CODED FIELDS TO DESIGNATE RECORDS AS ACTIVE OR INACTIVE 1. Include an ACTIVITY-STATUS code as the first position in the record. 2. Set the ACTIVITY-STATUS code to a value designating the record as active. 3. Change the ACTIVITY-STATUS code to a value designating the record as inactive only if it is to be deleted. 4. Before printing or other processing, check first to see if the record is active.

58. Structured COBOL Programming, Stern & Stern, 9th Edition The EXTEND Option Records may be added to the end of a sequential disk (or tape) file by using the following OPEN statement: OPEN EXTEND file-name When the OPEN EXTEND statement is executed, the disk pointer is positioned at the end of the file, immediately after the last record.

59. Structured COBOL Programming, Stern & Stern, 9th Edition The EXTEND Option A WRITE statement, then, will add records to the end of this file. – If all records to be added have key fields in sequence that are greater than those currently on the master, then the entire file will be in the correct order. If the records that are added are not in sequence, the file must be sorted before it is processed again.

60. Structured COBOL Programming, Stern & Stern, 9th Edition The EXTEND Option The following is a chart of permissible input/output statements depending on how a sequential file was opened: Statement INPUT OUTPUT I-O EXTEND READ X X WRITE X X REWRITE X

61. Structured COBOL Programming, Stern & Stern, 9th Edition CHAPTER SLIDES END HERE CHAPTER SUMMARY COMES NEXT

62. Structured COBOL Programming, Stern & Stern, 9th Edition CHAPTER SUMMARY A. Sequential updating by creating a new master. Use three files: an incoming master file, a transaction file with change records, and a new output master file that will incorporate all the changes. Use these techniques: 1. All files to be processed must be in sequence by the same key field.

63. Structured COBOL Programming, Stern & Stern, 9th Edition CHAPTER SUMMARY 2. A record is read from each file and specified routines are performed depending on whether or not the key fields match. 3. The transaction record could have a coded field to determine: a. What type of update is required. b. If the master record is to be deleted. c. If the transaction is a new account.

64. Structured COBOL Programming, Stern & Stern, 9th Edition CHAPTER SUMMARY 4. The end-of-job test for each file must be processed individually. By moving HIGH-VALUES to the key field of the file that ends first, we can be assured that the other file will always compare low and hence will continue to be processed. The job is terminated only after both input files have been processed. HIGH- VALUES can only be moved to a key field that has been defined as alphanumeric.

65. Structured COBOL Programming, Stern & Stern, 9th Edition CHAPTER SUMMARY B. Sequential updating by rewriting a disk. As an alternative to the preceding, records on a sequential disk can also be updated by rewriting them in place, if the disk file is opened as I-O. – A backup file should always be created in this case. C. Records can be added to the end of a disk file if we code OPEN EXTEND file- name.