Inside Module 5 Working with Files Page Copying files 2 Working with ordinary disc files 3 Defining fields 4 New or existing files 7 Writing records 10 Listing records 13 Printing reports 18 Calculating field totals 20 Sorting and removing duplicates 21

Copying files Copying an MPE file >input file1 >output file2 >xeq Copying an entire dataset to an MPE file >base store >get m-customer >output custdump >exit Copying an MPE file is one of the simplest Suprtool tasks. Input Specifies the file to be copied Output Defines the output filename Xeq Performs this task You can copy an entire dataset to an file to create a backup. Base Tells which database to read Get Tells which dataset to read Exit Gets you back to the OS Although the MPE/iX Copy command is a bit faster than Suprtool/iX for copying an MPE file, it does not work on datasets.

MPE files vs HP-UX files File system differences Input command requires more information on HP-UX Reclen Linefeeds MPE and HP-UX files systems are considerably different beasts. The MPE file system has many different types of files and also provides more information about the different attributes of a file. So much so, that in order to open an input file for processing you need only to specify the filename. >input myfile On HP-UX, you must specify the Record Length and whether or not the file has Linefeeds or not, so the syntax is slightly different. >input myfile,reclen 80,lf The above command tells Suprtool to open a file called myfile and that the record length is 80 bytes and each record has a line feed on the end. If the file myfile does not have linefeeds the syntax would be: >input myfile, r 80,nolf Suprtool on HP-UX self-describing files can be opened without this information added as the meta data about the file and it’s structure is stored in a “.sd” file.

Working with ordinary disc files A source of input files can be ordinary disc files, such as MPE, KSAM, tape files or POSIX files You can select, extract, and sort these files Listing the Custdump file >input custdump >list char >xeq Input This command tells Suprtool to use Custdump as the source file and read its contents; the Input command is used for disc files and has the same record-range options as the Get command. List Char Displays the contents of a file on the screen. The Char option converts nonprinting binary characters into “.” so that Suprtool doesn't treat the binary data as escape sequences, which can make your screen go crazy. Char also shows the starting word offset of each line, but the offsets are in octal (e.g. type = %34 to convert to decimal). Xeq Performs the task

POSIX filespace vs MPE filespace MPE filenames have three parts: file.group.account file.group account is assumed file group and account are assumed POSIX filenames can have any number of parts: ./file assumed to be in the current directory ../file in the next level ‘up’ ./Letters/PersonalStuff/Mom-1999-04-15 /ACCOUNT/GROUP/FILE Suprtool can use POSIX files anywhere it can use MPE files HP-UX files are similar to MPE’s POSIX files There is one gotcha with POSIX filenames: they can be very long, much longer than the usual 27 (or 36 if you count the lockword) characters for a regular MPE file. If you find that Suprtool or one of its components (Suprlink or STExport) tells you that the POSIX filename is too long, just use a file command. For example, :file x = /APACHE/PUB/htdocs/webecom/202020-ytd.html output *x On HP-UX you can refer to in a similar manner as POSIX files, you just cannot use file equations as this concept does not exist on HP-UX.

Define the record structure Use the DEFINE command to describe the layout of a flat file >input custdump >define account,17,8 >define lastname,37,16 >define credit,13,4,int;item credit,decimal,2 >extract account,lastname,credit >sort account >if credit > 2000.00 >list standard >xeq May 06, 1996 22:28 File: CUSTDUMP Page 1 ACCOUNT LASTNAME CREDIT 00010003 Melander 2500.00 00010005 Coyle 2400.00 Define Tells Suprtool where the fields are in the file, their size and type. Once this is done, you can use other Suprtool commands (e.g., Extract, Sort, and If) to operate on these fields. Syntax DEFINE filename, start, length [, type] The start position is the byte offset within the record, starting with number 1 in the first byte. The length is the number of bytes occupied by the field. The type options are byte (default), integer, packed, or zoned. Test results To check the accuracy of a Define command, perform an Extract on the defined fields followed by List Standard. If the columns of the listing make sense, you did a good job. If the first character is missing, your starting offset for the field is probably off by 1. Invisible fields Define is a lifesaver when you have invisible fields in a database schema. For example, you may have an X80 field called “filler” which really contains a few integers, a string, and a zoned-decimal field. This is especially common in PowerHouse applications. The Define command lets Suprtool access these hidden fields. If the database is still open, you must select new field names which do not duplicate those in the database. Alternately, you can close the database.

Let Suprtool maintain field names Use the INPUT command to tell Suprtool that a file has the same structure as a dataset >input custdump = m-customer >item credit-rating,decimal,2 >extract cust-account,name-last,credit-rating >sort cust-account >if credit-rating > 2000.00 >list standard >xeq May 06, 1996 22:38 File: CUSTDUMP Page 1 CUST-ACCO NAME-LAST CREDIT-RATING 10003 Melander 2500.00 10005 Coyle 2400.00 Syntax INPUT filename[=dataset] [(startrecord/[end])] [(#count)] Field names Unless you have changed the dataset structure since creating the file, all Suprtool commands can use the original field names. That means you don’t have to create alternate names for each field; you can simply continue using the names from the database. Leading zeros Notice that the customer account number had leading zeros in the previous example. In this example, however, there are no leading zeros because the field type has been specified as zoned decimal. List Produces formatted listing of records The input filename=dataset does not currently work on HP-UX. Further file structuring solutions are available with self-describing files. Learn more about them in "Working with Suprlink" (Module 5).

To create, or not to create -- that is the option Default is to create a new, permanent file >output custdump TEMP creates a temporary file >output foo,temp APPEND adds data to an existing file >output blabla,append ERASE overwrites an existing file >output subfile,erase You can write to the following types of files: New file This is the default option Existing file If you want to write your output file to an existing file, you must specify the Erase or Append option with the Output command. Use Set Ignore On if you want to ignore duplicate keys while overwriting a KSAM file. Temporary file You have to specify the Temp option to save a file as a temporary file. Temporary files do not exist on HP-UX, so therefore this option is not available.

To squeeze, or not to squeeze, that is the option Sometimes output file capacity (limit) is set higher than the number of records (EOF) Sometimes the limit is squeezed to the EOF to save disc space You control it with SET SQUEEZE ON or OFF To reserve space for appending later, use SET SQUEEZE OFF >get m-customer >if state-code="BC" >out foo >x IN=20, OUT=12. CPU-Sec=1. Wall-Sec=1. >get m-customer >if state-code="AL" >out foo,append >x Error: There is no room to APPEND records. The concept of File Limits do not exist in the HP-UX file system, so therefore the above is not relevant.

To squeeze, or not to squeeze, what is the default? If you specify Set Squeeze On or Off, Suprtool will do what you say If you don’t specify, Suprtool makes up its own mind The Output file will be squeezed except in these cases: input is a file, not a dataset output option is Append or Erase output option is Ask or Num,Query Here's the actual SPL source code: if set'squeeze'flag = squeeze'always then get'out'disposition.(12:1) := 1 else if set'squeeze'flag = squeeze'never then get'out'disposition.(12:1) := 0 else if in'is'a'file or out'domain <> domain'new or out'option = out'ask or out'option = out'num'query or out'is'ksam then get'out'disposition.(12:1) := 0 else get'out'disposition.(12:1) := 1;

Writing records The OUTPUT command determines where your output records go and in what format >output customer,num,data Select one of these output formats: Data (default) - records are identical to input format Key - records contain only the sort keys Num - records contain 32-bit input record number The obvious default is to write your selected data to the output file. In addition to this data, or instead of this data, you can write the sort keys, or the record numbers of the input record.

More common record formats Additional formats of the OUTPUT command: Num,Key Num,Data Query ancient self-describing Link improved self-describing Num,Query Ask ASK report writer ASCII human-readable Display computer-readable PRN import to PC program Num,Key 4-byte integer record number plus sort keys Num,Data 4-byte integer record number plus data record Query Self-describing output file Link New format self-describing file Num,Query Query numbers format Ask COGELOG ASK select file ASCII Convert numeric to ASCII Display Convert numeric to display (zoned-decimal) PRN Personal computer format

Numrecs controls size of output file Limit the number of records selected Limit the size of the sort scratch files Limit the size of the output file if input is a dataset Specify the number of records in a tape file Use percentage >100 with SET SQUEEZE OFF to create output file bigger than input file. This provides space for appending records. >numrecs 100 >numrecs 100000 >numrecs 10% >numrecs 200% Numrecs performs a number of functions, all related to the number of records selected, processed, written, sorted, or reserved in the output file. You can limit the number of records that the If command selects by specifying Numrecs, which also sets the size of the sort scratch file. When reading records from a dataset, you usually do not want to create an output file that can hold all the input records because you are only selecting some of the input records with the If command. The Numrecs command limits the output file capacity to the number of records specified. The number of records stored on a tape is unknown so Suprtool just assumes that there are no more than 10,000. Numrecs allows you to specify this number more accurately. You can set the Numrecs to a specific number or to a percentage of the number of records in the input file. Specify a size larger than 100% if you are creating an output file which will have to have a lot more records appended later. Don’t forget Set Squeeze Off.

Listing records Listing refers to displaying the records in either a dump format or as simple reports Use the LIST command to produce formatted listings of selected records >list octal,char >list decimal,record >list standard >list hex,char,labels Dump format Octal Char is the default format for records with an unknown structure Simple reports This is a formatted listing with field names and field values converted into ASCII Print listing Use the List Record command to send a listing to the attached printer. Suprtool uses Record mode on your terminal or PC to List to the attached printer. If the attached printer is a LaserJet, you may want to use the Set PCL command to enable a particular font or orientation. Report title You can easily change the title of a report >get m-customer >list title “Sample Report” date 2 >xeq Date formats 0 = dd mmmyy 20 May96 1 = yy/mm/dd 96/03/20 2 = mm/dd/yy 03/20/96 3 = dd/mm/yy 20/03/96 4 = mmm dd, ccyy May 20, 1996 Time formats 0=none 1=24-hour 23:02 2=a.m./p.m. 11:02 p.m. Set PCL is described a little later in this module.

List format of nonself-describing files >in catalog.pub.sys(12/12) >list >exit >IN CATALOG.PUB.SYS (12) >OUT $NULL (12) 00000: 030460 020127 071157 067147 020166 067554 10 Wrong vol 00006: 072555 062440 067556 020114 042145 073043 ume on LDev 00014: 056056 020040 040556 067564 064145 071040 . Another 00022: 060566 060551 066141 061154 062440 024131 available (Y 00030: 027516 024477 020040 020040 020040 020040 /N)? 00036: SAME TO: 000043 00044: 030060 030061 031460 030060 00013000 Defaults Disc files, tape files, and KSAM files with unknown formats default to an Octal Char dump. Both the offset in the left most column and the actual data are displayed in octal. Same to This term indicates that the last data value shown (020040) is repeated until word number 43 (decimal number 35) corresponding to bytes 70 and 71.

Some List options for reports ONEPERLINE NONAME NOSKIP STANDARD format DUPLEX printing HEADINGS NOREC Oneperline Puts each extracted field on its own line Noname Suppresses printing field names next to each field value Noskip Removes the blank line that is usually printed between records Standard Represents a set of predefined format options Duplex Invokes double-sided printing on a LaserJet IID or IIID Headings Allows you to use your own headings instead of the default field names Norec Suppresses printing of input and output record numbers List Oneperline is useful for mailing labels and preparing program input.

Listing one field per line >get m-customer >list oneperline >xeq >GET M-CUSTOMER (1) >OUT $NULL (0) NAME-FIRST = Terry NAME-LAST = Coyle STATE-CODE = AL CUST-STATUS = 30 List Without the Oneperline option, the List command puts as many records as possible on each line. >GET M-CUSTOMER (1) >OUT $NULL (0) NAME-FIRST = Terry NAME-LAST = Coyle STATE-CODE = AL CUST-STATUS = 30 Oneperline This option is useful for creating input to menu-driven programs (e.g., Xpress Config program). You can do this by creating an output file with the combined fields and using it as input to List Oneperline.

Preparing program input by combining List options Combine LIST options to format input to other programs >get m-customer >extract name-last, name-first, city, state-code >list norec, noskip, noname, oneperline >file suprlist=myinput,new;save;dev=disc; rec=-80,,f,ascii;nocctl >xeq Run the program with the file as input :run dataload.prog;stdin=myinput or.... :run dataload.prog < myinput Extract List only the fields that the program asks for Norec Don’t list the input and output record numbers Noskip Don’t skip a line between each input record Noname Don’t show the field name in front of each data value Oneperline Put each extracted field on its own line in the list File The File command directs the output of the List command to the specified file, which will be created with the specified characteristics Example The Dataload program repeatedly prompts for four pieces of data and stops when it reaches end-of-file. :run dataload.prog Enter last name: Jones Enter first name: Michael Entry city: Redmond Enter state code: WA Enter last name: Smith Enter first name: Ann ... Enter last name: That’s all folks! END OF PROGRAM To create input to interactive programs, you can extract constants that represent commands or menu choices.

Printing reports The LIST command writes to an output file called Suprlist, which defaults to $stdlist Override the default using a file command :file suprlist;dev=laser155 Listing to a LaserJet SET PCL command indicates page orientation and font type Set PCL 0 Indicates printer is not a LaserJet Set PCL 1 Landscape, Lineprinter font, 175 columns If you are printing to a LaserJet, this command is recommended. It selects 175 column output in Landscape mode. You may want to add this command to the file Suprmgr.Pub.Sys so that every time Suprtool runs, PCL gets set to 1. Only add this to Suprmgr if all your List commands are for LaserJets. Set PCL 2 Landscape, Courier font, 100 columns Set PCL 3 Portrait, Courier font, 80 columns Set PCL 4 Portrait, Lineprinter font, 132 columns Set PCL 5 Portrait, Courier, 80 columns, A4 paper Set PCL 6 Landscape, Lineprinter, 223 columns, legal List Duplex Allows you to print on both sides of the paper if you have a LaserJet IID or IIID. Portrait duplex pages are printed head-to-head, and Landscape duplex pages are printed head-to-toe so that you can insert the listings in a 3-hole binder. This is a function of the LaserJet printer, not a Suprtool feature.

Printing mailing labels Use the EXTRACT command with LIST ONEPERLINE to produce mailing labels >get m-customer >extract " " {blank line} >extract " " {blank line} >extract customer-name >extract street-address(1) >extract street-address(2) >extract street-address(3) >extract " " {blank line} >list oneperline,noname,noskip,norec >xeq Labels The labels in this example have the following format: Extract Each Extract command corresponds to one line in the label. A label is made for every input record selected. List versus The List Oneperline command differs from the Output Output command by putting only one field on each line instead of multiple fields. This is the format of the List Oneperline command: Makita 3/8" Var. Speed Drill #DP3730 50511501 This is the format of the Output command: Makita 3/8" Var. Speed Drill #DP3730 505115 <blank line> <blank line> Dave Patterson 1234 Main Street Anytown, New York 10101-2292 <blank line>

Calculating totals in numeric fields The TOTAL command provides an easy way to sum the contents of one or more numeric fields in selected records >if state-code = "BC" >total sales-total By default, the result is printed to $stdlist or can be redirected to another device If you are using the Suprtool2 interface from a programming language, the total amount is returned to the calling program in the workspace Example >get d-sales >if purch-date >= 19930101 and & >>purch-date <= 19931031 >total sales-qty >total sales-tax,2 >total sales-total,2 >x Totals (FRI, MAY 12, 1996, 1:31 PM): SALES-QTY 11+ SALES-TAX 148.81+ SALES-TOTAL 1211.68+ IN=8, OUT=8. CPU-Sec=1. Wall-Sec=1.

Sorting records Suprtool can sort in several ways On any field On any part of an input record, not just previously defined fields According to multiple sort keys (e.g., primary, secondary) Ascending or descending order MPE files require a DEFINE command to define the field or use the KEY command Single sort key Ascending order is the default. In this example, the records are arranged from lowest to highest customer number: >sort cust-number In the following example, the default sorts records starting with the oldest transaction (assuming the purch-date item collates properly): >sort purch-date Descending order Use the Desc option to sort in descending order. >sort purch-date, desc Specifying descending order means the highest values come first. Multiple sort keys >sort cust-account {primary sort key} >sort purch-date,desc {secondary sort key} Arbitrary field Define any position in the input record as a field to sort on. For example, >define area-code,phone-number,3,byte >sort area-code or >key 11,4,double

Working with duplicate records DUPLICATE [ NONE | ONLY ] [ RECORD | KEYS [ n ] ] >duplicate none record >duplicate none keys 1 >duplicate only record >duplicate only keys DUPLICATE NONE KEYS [ n ] [ COUNT ] [ TOTAL field [ field... ] ] >duplicate none keys count >duplicate none keys total sales-qty sales-value >duplicate none keys count total sales-qty sales-value You may want to create an output file with only unique values. Or you may have an input file that is not supposed to contain any duplicate entries, and you want to check if there are any duplicates. The Duplicate command operates on the principle that every record is either an “original” or a duplicate. Suprtool determines whether a record is an original or a duplicate by looking at the records as they come out of the sort module. In any sequence of records with the same sort key, the first record is the “original”, and all the rest are the duplicates. That first record would be selected using Duplicate None. The duplicates would be selected with the Duplicate Only command. The Keys keyword on the Duplicate command refers to sort keys, not IMAGE keys. By default, Duplicate uses all the sort keys to determine if a record is a duplicate of the previous record. You can tell Suprtool to use fewer sort keys to make this determination. For example, if you need the output sorted by account-number and invoice-date, but want to use only account-number to determine “duplicate-ness”, you would specify keys 1.

Discarding duplicates from the output file Remove duplicates to get a list of unique values or records Based on the whole record or the sort key(s) 10003 112.07 19931015 505 10003 166.00 19931015 505 10003 219.10 19931015 505 10016 159.42 19931021 505 10020 224.15 19931001 505 10020 167.13 19931028 505 Input file - 6 records >sort cust-account >dup none keys 10003 112.07 19931015 505 10016 159.42 19931021 505 10020 224.15 19931001 505 Output file - 3 records Dup None Keys Suprtool determines whether a record is an “original” or a duplicate by looking at the sort keys of successive records. You must be sorting so that Suprtool knows what the key fields are. Dup None Record Suprtool looks at the whole record to see if it is a duplicate of the previous record. If you do not have a Sort command, then Suprtool assumes that the file is already sorted. It does not check whether the file is sorted, so the results would be unpredictable if the file isn’t sorted.

Saving only the duplicates Remove “originals” to get a list of duplicate values or records Exact opposite of DUPLICATE NONE 10003 112.07 19931015 505 10003 166.00 19931015 505 10003 219.10 19931015 505 10016 159.42 19931021 505 10020 224.15 19931001 505 10020 167.13 19931028 505 Input file - same 6 records >sort cust-account >dup only keys Dup Only Keys You must be sorting so that Suprtool knows what the key fields are. If a record has the same sort keys as the previous record, it would be copied to the output file. Otherwise, the record would be discarded. Dup Only Record This command is similar to Dup Only Keys, but the whole record is used to identify a duplicate record. You do not have to be sorting with this command. Suprtool assumes that the input file is already sorted. 10003 166.00 19931015 505 10003 219.10 19931015 505 10020 167.13 19931028 505 Output file - the other 3 records

Counting records DUPLICATE COUNT can tell you how many records have the same key 10003 112.07 19931015 10003 166.00 19931015 10003 219.10 19931015 10016 159.42 19931021 10020 224.15 19931001 10020 167.13 19931028 Input file - 6 records >get d-sales >sort cust-account >duplicate none keys count Output file - 3 records 10003 112.07 19931015 3 10016 159.42 19931021 1 10020 224.15 19931001 2

Totaling records DUPLICATE TOTAL calculates a field total for all records with the same key >get d-sales >sort cust-account >duplicate none keys total sales- total 10003 112.07 19931015 10003 166.00 19931015 10003 219.10 19931015 10016 159.42 19931021 10020 224.15 19931001 10020 167.13 19931028 Input file - 6 records Output file - 3 records 10003 112.07 19931015 497.17 10016 159.42 19931021 159.42 10020 224.15 19931001 391.28

Exercises Duplicates, Duplicates, Duplicates, Duplicates Exercise 1 Create a list of all the states/provinces in which we have customers Exercise 2 List all the dates on which we made more than one sale Bonus Exercise 3 List all the sales made on the dates in Exercise 2 HINT: Requires two passes, and the TABLE command Tables are described in "Selecting Records with Suprtool" (Module 3).

How to check Suprtool results Use the :SHOWJCW command to check the Job Control Word (JCW) after a task has been completed On MPE V and MPE/iX, the SUPRTOOLOUTCOUNT JCW contains the number of records written to the output file (up to 65,535 maximum) On MPE/iX, the SUPRTOOLFULLCOUNT variable also contains the output count (no limit) On HP-UX, -oc option puts count into .stoutcount System JCW Suprtool sets the system JCW to a fatal state when a task fails in a batch job. The job then aborts unless the :run command is preceded by a :continue. !continue !run suprtool.pub.robelle;info="use task"& !;parm=4 !if jcw >= fatal then Suprtooloutcount !run suprtool.pub.robelle base bb;get xx;if ...;xeq;exit !if suprtooloutcount <> 0 ! run report23.pub.accting !else ! tellop There were no transactions! !endif HP-UX delfile file98a suprtool -oc << \!EOD in abstest;out file98a,link;exit !EOD if [ 'cat .stoutcount' -ne 2 ]; then echo Incorrect number of records selected cat .stoutcount fi It is Suprtool, not Suprlink, that sets the Suprtool@Count JCW. 28,916

Summary Copy a dataset or a file Define new fields Select a set of records Produce listings Specify record formats Sorting records Checking for duplicates