Copyright 2007, Information Builders. Slide 1 Modify Internals Cesare Petrizio Information Builders

Copyright 2007, Information Builders. Slide 2

Copyright 2007, Information Builders. Slide 3 Agenda  MODIFY Internals  Database Creation  Analysis Phase  Transaction Processor  Transaction Processor Activation  Database Processor  Database Actions  Database Physical Structure  Limits  Database Logical Structure  File Directory Table  File Control Block  Pointers  MODIFY Overview

FOCUS Database  CREATE FILE filename  Initial MODIFY FILE filename  REBUILD/REBUILD or REBUILD/REORG  HOLD FORMAT FOCUS A FOCUS Database is established by:

FOCUS Database  MVS/TSO  Database to be used in SU process  Database to be used as part of MAINTAIN or COMBINE CREATE FILE is required for a FOCUS (or FUSION) database if:

Copyright 2007, Information Builders. Slide 6 MODIFY Internals Transaction Processor Analysis File Transactions Log Files Screen Transactions Database Processor Database BINs

Copyright 2007, Information Builders. Slide 7 Analysis n Open and CHECK Master File Descriptions n Verify DBA access to file(s) n Determine I/O access modules required n Check DBA and set up access restrictions at SEGMENT and FIELD levels. n Read and parse the FOCEXEC n Establish VALIDATE statements for VALUE restrictions n Set up memory for Scratch Pad Area if required n Establish DATREC and LINREC n Establish Case Processing tables n If COMPILE, create FOCCOMP and exit.

Copyright 2007, Information Builders. Slide 8 MODIFY Internals Analysis Work Areas  DATREC – DATabase-RECord  Retrieval of Database Values  Maximum size – 32K  LINREC – LINe RECord  Retrieval of Transaction Values  Maximum size – 32K  COMPUTE and VALIDATE fields are extensions to the transaction

Copyright 2007, Information Builders. Slide 9 MODIFY Internals Analysis Work Areas  RECDIS - RECord Description  Field Activation Table  Maximum 3072 fields  CRTBUF – CRTform BUFfer  Last display of CRTFORM  Divided into Screens

Copyright 2007, Information Builders. Slide 10 MODIFY Internals Analysis - Work Areas  SEGACT – SEGment ACTivation  Segment Activation Table  Established per CASE  Bits to control  Activation  Move to DATREC  MATCH/NOMATCH  SEGTYPE and MATCH logic

Copyright 2007, Information Builders. Slide 11 MODIFY Internals Analysis – Special Conditions  SEGACT- SEGment ACTivation  COMBINE – SYSTEM99 root segment of each database is constructed in FOCSORT  Indexed MODIFY –  Index is treated as root segment  If the index is not the key of its segment, key(s) must also be MATCHed  No access to segments higher in the hierarchy  UPDATE or REJECT only at entry level (no INCLUDE OR DELETE)  Full access to MODIFY subcommands at lower level segments

Copyright 2007, Information Builders. Slide 12 MODIFY Internals Analysis - Work Areas  SPA (Scratch Pad Area)  Width: (Storage Length of fields * 2) + 1  # Rows: Max of REPEAT n Default 5 rows Rounded up to nearest 1K  Fields:  Every field mentioned in any HOLD REPEAT  Every field mentioned in any GETHOLD REPEAT  Every field mentioned in CRTFORM field(index)

Copyright 2007, Information Builders. Slide 13 MODIFY Internals Order of Processing  No Cases  Transaction Subroutine  Non-Position Related Commands  MATCH/NEXT to ALL segments  TOP CASE only  Transaction Subroutine  Non-Position Related Commands  MATCH/NEXT to segments specified  Other Cases  Non-position Related Commands in order coded  MATCH/NEXT to segments specified  If MATCH/NEXT is coded, all GOTO commands are considered ON MATCH/NOMATCH GOTO …

MODIFY Internals Order of Processing - MODIFY Premise Action COMPUTE field/format = ; YY COMPUTE/VALIDATE field/format = expression YYYY Same name and format as other field YNYYNN TOP CASE YNYN GOTO/IF... GOTO/PERFORM YY MATCH/NEXT prior in case YN Non-Positional TYPE/COMPUTE/REPEAT... Y Analysis Phase, extend LINREC YYY After first transaction subcommand YY ON MATCH/NOMATCH... Y Prior to MATCH/NEXT commands YYY Order coded Y

Copyright 2007, Information Builders. Slide 15 MODIFY Internals Default MATCH/NOMATCH Actions  MATCH * (MATCH all active fields)  MATCH * KEYS (MATCH all active keys)  MATCH requires a MATCH to all keys:  WARNING – MATCH ASSUMED  Default ON NOMATCH - INCLUDE  Default ON MATCH - REJECT

Copyright 2007, Information Builders. Slide 16 MODIFY Internals Transaction Processor Analysis File Transactions Log Files Screen Transactions Database Processor Database BINs

Copyright 2007, Information Builders. Slide 17 MODIFY Internals Transaction Processing  FREEFORM  Default Transaction Type  Comma-delimited  Maximum LRECL 256  Value Carry-over  Comma Dollar (,$) to end transaction MODIFY FILE CAR DATA ON CCARS END ENGLAND,JAGUAR,V12XKE AUTO,SEDAN,$ CAR=JENSEN,$

Copyright 2007, Information Builders. Slide 18 MODIFY Internals Transaction Processing  PROMPT  Creates comma-delimited file  PROMPT field field field SEG.field  END in response to prompt terminates transaction processing  PROMPT COUNTRY 5(CAR MODEL)  Converts to FREEFORM MODIFY FILE CAR PROMPT COUNTRY CAR MODEL MATCH COUNTRY ON NOMATCH INCLUDE DATA END

Copyright 2007, Information Builders. Slide 19 MODIFY Internals Transaction Processing  FIXFORM  FIXFORM field/format field/format field/Cn Xn  Xn is ignored at the end of any FIXFORM except in TOP CASE  Default format type is A (alpha)  Upon reading the transaction, values are moved to LINREC and with necessary conversion.  Invalid format gives FORMAT rejection  FORMAT rejection if not in ACCEPT list.  Read is performed:  TOP CASE  FIXFORM ON  When required to fulfill request

Copyright 2007, Information Builders. Slide 20 MODIFY Internals Transaction Processing  FIXFORM FROM mastername  Mastername must describe a single segment SUFFIX = FIX FILE  Actual is used to indicate the input format  All fields are conditional  FIELDNAME matches to FIELDNAME or ALIAS  Used in HOLD FORMAT FOCUS FOC$HOLD.FEX MODIFY FILE newfile FIXFORM FROM FOC$HOLD DATA ON FOC$HOLD END

Copyright 2007, Information Builders. Slide 21 MODIFY Internals Transaction Processing  CRTFORM {LINE n} {UPPER/LOWER} {TYPE n} {WIDTH n}  Default Width – 78 columns  Default Height – TYPE lines  Non-conditional field  Blank is acceptable value and will update database  <field Conditional field  Blank is considered not present.  Creates FIXFORM transaction

Copyright 2007, Information Builders. Slide 22 MODIFY Internals CRTFORM – Source of D., T. fields Henry’s Famous Table Database FieldRe-COMPUTECOMPUTE Field MATCH DATREC Active? LINREC NOMATCH Defaults Not Active DATREC

MODIFY Internals Source of Fields Values on CRTFORM Premise Source INVALID Y CRTFORM NOCLEAR Y Index > HOLDCOUNT Y Y D.Field or T.field YYYYY Database field YYNYY Compute NNYYY Field ACTIVE YN MATCH YN DATREC YY LINREC YY CRTBUF YY Defaults YY Blanks Y

Copyright 2007, Information Builders. Slide 24 MODIFY Internals Transaction Processor Analysis File Transactions Log Files Screen Transactions Database Processor Database BINs Activation

Copyright 2007, Information Builders. Slide 25 MODIFY Internals Activation  FIELD Activation  Controls UPDATE  Field is ACTIVE When :  Present on a Transaction  Left side of a COMPUTE  ACTIVATE issued  Field is Deactivated When:  UPDATE is performed  INCLUDE / DELETE at that level  Navigation to TOP Case  DEACTIVATE issued

Copyright 2007, Information Builders. Slide 26 MODIFY Internals Activation  SEGMENT Activation  Controls  MATCH  INCLUDE  Any field on SEGMENT is Activated  Segments lower in the hierarchy activated  Segment is Deactivated When:  INCLUDE / DELETE at that level  Navigation to TOP Case  Movement to different segment instance,same level.  DEACTIVATE issue

MODIFY Internals FIELD and SEGMENT Activation FIXFORM field/format or CRTFORM Y FIXFORM field/Cformat or CRTFORM <fieldYY Field value = blankNYNYN CRTFORM or Y CRTFORM <field or <T.fieldYYY Transaction value = database valueNY COMPUTE field= expression;Y ACTIVATE fieldYY Field already ACTIVEYN Field ActivatedYYYYYYY SEGMENT ActivatedYYYYYYY Value Moved to LINRECYYYYY Default Moved to LINRECYYY DATREC Value moved to LINRECY

MODIFY Internals FIELD and SEGMENT Deactivation INCLUDEY UPDATEYY DELETEY MOVE TO DIFFERENT SEGMENT INSTANCEY DEACTIVATEY DEACTIVATE RETAINY TOP CASEY All fields on SEGMENT DeactivatedY Field DeactivatedYYYYYY SEGMENT DeactivatedYYYYY Default Values to LINRECY

Copyright 2007, Information Builders. Slide 29 MODIFY Internals Transaction Processor Analysis File Transactions Log Files Screen Transactions Database Processor Database BINs

Copyright 2007, Information Builders. Slide 30 MODIFY Internals MATCH  MATCH keyfield(s)  Using INDEX  SEGTYPE = S1 or SH1  Root Segment  CRTFORM or Trans value < last trans  Beginning of Chain  SEGTYPE = ‘ ‘  SEGTYPE = S[H]n,  Trans Value < last trans  Current Position  SEGTYPE = S0  SEGTYPE = S[H]n  Trans Value > last trans

Copyright 2007, Information Builders. Slide 31 MODIFY Internals NOMATCH  SEGTYPE = S[H]n  Prior to searched value  SEGTYPE = S0  End-of-Chain  SEGTYPE = blank  End-of-Chain  Explicit MODIFY through index MODIFY FILE fn.index  NO position at NOMATCH

MODIFY Internals MATCH SEGTYPE = S[H]1YYY SEGTYPE = S0Y SEGTYPE = blankY SEGTYPE = S[H]nYYY Top SegmentYYY Root key indexedYYY CRTFORMYNN Current Transaction : last Trans<><>= Start Beginning of ChainYY Start Current PositionYYY Use IndexYY NOMATCH at END-OF-CHAINYY NOMATCH Prior to searched valueYYYYY

Copyright 2007, Information Builders. Slide 33 MODIFY Internals Movement to DATREC - local mode  MATCH keys  ON MATCH CONTINUE  No action (“passing through”)  ON MATCH COMPUTE/CRTFORM/VALIDATE  Move segment instance to DATREC  Take required action  MATCH WITH-UNIQUES key  Retrieves host instance + ALL unique child instances  ON MATCH COMPUTE/CRTFORM/VALIDATE  Unique segments are considered “logical extensions of the parent”

Copyright 2007, Information Builders. Slide 34 MODIFY Internals Movement to DATREC  NEXT field  Entire instance moved to DATREC  Key fields are copied to LINREC  COMPUTE/VALIDATE RETURN/I1 = FIND (val IN filename)  Index page read into BINs (if necessary)  Only RETURN is populated (1 or 0)  COMPUTE/VALIDATE RETURN = LOOKUP([SEG.crfield] condition);  Crfield - JOIN segment or [DKU/KU]  JOIN and COMBINE are mutually exclusive  Value for field(s) moved to DATREC and LINREC  Crfield value taken from LINREC

MODIFY Internals MATCH MATCHYYY SUYNNNY ON MATCH COMPUTEY NEXTY FINDY LOOKUPYY MOVE FROM DATABASE TO BINSYYYYY MOVE FROM BINS TO DATRECYYYYY MOVE INDEX PAGE TO BINSY SEND DATREC TO USERYY MOVE KEYS FROM DATREC TO LINRECYYY MOVE NON-KEYS FROM DATREC TO LINRECYY

Copyright 2007, Information Builders. Slide 36 MODIFY Internals Transaction Processor Analysis File Transactions Log Files Screen Transactions Database Processor Database BINs Database actions

Copyright 2007, Information Builders. Slide 37 MODIFY Internals Database Actions  INCLUDE  Values moved from LINREC To BINS  All Active Child Segments INCLUDED  Non-SU deleted space may be re-used  UPDATE  Specified ACTIVE fields are updated from LINREC to BINS  MATCH WITH-UNIQUES UPDATE might be INCLUDE of UNIQUE segment  DELETE  Segment and all descendant segments are deleted

Copyright 2007, Information Builders. Slide 38 MODIFY Internals INDEX Actions  INCLUDE  New value and address are moved to index page  Full index pages require index page split  UPDATE  Index searched and old value deleted  New value and address moved to index page  Full index pages require index page split  DELETE  Index searched and old value deleted

Copyright 2007, Information Builders. Slide 39 MODIFY Internals Transaction Counter  Transaction Counter is incremented in first PHYSICAL case with data collection.  CHECK nnnn  After nnnn transactions, COMMIT is issued  Default CHECK (no COMMIT specified in MODIFY)  LOCAL  CHECK 10 CRTFORM  CHECK 1,000 FIXFORM  SU  CHECK 1 CRTFORM  CHECK 1,000 FIXFORM  SU – CHECK saves all users changes

Copyright 2007, Information Builders. Slide 40 MODIFY Internals Transaction Counter  Changes are made into BINs  SU and COMMIT  Local User changes to Local Bins  SU, not COMMIT  Local User changes sent to Sink and sink BINs  “Forced CHECK”  Move from BINs to Database  If SHADOW is ON for that database, move to shadow pages.  CHECK or COMMIT  Move any additional pages from BINS to database  If SHADOW = ON, update SHADOW directory

Copyright 2007, Information Builders. Slide 41 QUESTIONS Thanks for Coming