Load Profile Training

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Agenda – Day One  Introductions  Review of training goals  Review of Innovative Record Structure  Anatomy of a Data Load  Load Tables (m2btab)  Review of standard load profiles  Global Variable Functions

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Agenda – Day Two  Questions from day one  Special Processing Functions (%)  Command Functions (#)  Translation Tables (m2bmap)  Data analysis  The m.marcload.local file  Test loading records  Review and support from Innovative

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Training Goals  Understand the database structure  Understand existing load profiles  Be familiar with issues to consider for new load profiles  Update and create load tables (m2btab)  Create translation tables (m2bmap)  Add menu options for loading records  Test load profiles before production

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Topics Not Covered in This Training  Text to MARC conversion programs  Loading non-MARC records  Loading Contact, Course, Checkin Cards, Invoice, License, Program, Resource, Session or Vendor records  Complex data manipulation  Export tables

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Record Structure  Types of records  csdirect.iii.com (see chart)  Innovative Guide and Reference no  The Getting Started gsm.iii.com

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Each Record Type Contains:  Variable-Length Fields M>I>C>T and M>I>C>M (Page 7 and IGR page )  Fixed-Length Fields (Page 14 and IGR page )

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Anatomy of a Data Load Load Table (m2btab) Loading prog. (marc2inn) Trans. File (bibtemp) Holding symbol table m.marcload.local Data Exchange Load Interface Translation Table (m2bmap) Fixed-length fields Fixed-length values/codes Variable-length fields Record Templates Indexing prog. (control) Indexes Database Index rules Hi/Lo watermark

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Load Tables (m2btab)  Control how records are loaded  Fields in the incoming record that do not match load table criteria will not be loaded  Create separate load tables for different loading requirements  Data must be in MARC format

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Standard Load Tables  Authority records  m2btab.a (p. 140)  m2btab.anam (p. 142)  m2btab.asub (p. 144)  Bibliographic and Item records  m2btab.batch (p. 130)  m2btab.lti (p. 137)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Standard Load Tables  Bibliographic and Order records  m2btab.order (p. 133)  Patron records  m2btab.p (p. 146)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Overview of Load Tables  Commenting out lines with “#”  Twelve data elements  Global Variable Functions  Special Processing Functions (%)  Command Functions (#)  “#” has 2 meanings in load tables

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Comment Lines in Load Tables  Any line that begins with “#” is a Comment  Include a brief summary of load table, identify the author and creation date  It is strongly recommended that you add a Comment to every load table you create  Commented lines are not read by the loading program

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 1 – MARC Tag 690 |699|+|0|0|b|d|0|y|N|0|%first=“y”  MARC tags and indicators to be loaded  Can be set as a regular expression  Can load from any MARC field and the Leader

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 2 – New MARC Tag 690| 699 |+|0|0|b|d|0|y|N|0|%first=“y”  Re-map the data from the tag in element one to the new MARC tag in element two  Cannot be a regular expression  If indicators are specified, all indicators must be present

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 3 – Subfields 690|699| + |0|0|b|d|0|y|N|0|%first=“y”  Indicates which subfields to load from MARC field in Element 1, MARC tag  Unless specified, subfields are loaded in the order that they appear in the incoming data

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 4 - Offset 008|| | 35 |3|b| |24|n|N|0|lang(b) 690|699| + |0|0|b|d|0|y|N|0|%first=“y”  Used for loading data from positional fields, e.g. Leader, 006, 007, 008  Counting begins at 0  Loading data from single subfield, refers to beginning of subfield  Must be set to “0” when loading the entire field (usual setting)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 5 – Byte Count 008|| |35| 3 |b| |24|n|N|0|lang(b) 690|699|+|0| 0 |b|d|0|y|N|0|%first=“y”  Number of bytes to load starting from offset in Element 4  Counting begins at 1  Must be set to “0” when loading the entire field (usual setting)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Example of MARC 008 field s1991####nyu###########001#0# eng # Language code at byte 35 of 008 field: Offset = 35 Byte count = 3

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 6 – Record Type 690|699|+|0|0| b |d|0|y|N|0|%first=“y”  Determines the record type to load the MARC data into for this line  This can be set to:  “b” (Bibliographic) or “i” (Item) or  “o” (Order) or “c” (Holdings/Checkin) or  “a” (Authority) or “p” (Patron)  Will always correspond with Element 11, Pass Indicator

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 7 – Variable Field Group Tag 690|699|+|0|0|b| d |0|y|N|0|%first=“y”  Indicates the variable field group tag to load data into  Each line can map to a variable field or a fixed field, but not both  This should be set to “space” if using the line to load a fixed-length field

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 8 – Fixed Field No. 690|699|+|0|0|b|d| 0 |y|N|0|%first=“y”  Indicates the fixed-length field to load data into  Fixed-length fields must be referenced by line numbers. See page 14  If the line is used to load a variable- length field then this element should be “0”

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 9 – MARC/Non-MARC Indicator 690|699|+|0|0|b|d|0| y |N|0|%first=“y”  Determines if the incoming field is loaded in MARC format or not  Works in conjunction with trigger  Load lines for fixed-length fields always have a non-MARC indicator, “n”  Variable fields usually have a MARC indicator, “y”

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 10 - Permanence 690|699|+|0|0|b|d|0|y| N |0|%first=“y”  Used for loading during Interactive hardwired Serial Port Interface  Determines whether that setting is retained or not for subsequent loads  Usually set to “N” for most lines in a load table since Serial Port Interface is not used much

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 11 – Pass Indicator 690|699|+|0|0|b|d|0|y|N| 0 |%first=“y”  Determines at which pass the line is read during the load  Will always correspond with Element 6, Record Type  Each new record type loaded requires a new pass  Primary record type is always pass 0

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Element 12 – Function or Note 690|699|+|0|0|b|d|0|y|N|0|%first=“y”  Global Variable Function  Command Function (#)  Special Processing Function (%)  Text to be inserted  Any other beginning character is a comment and is not read by the loading program

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Training Exercise 1 “Creating and Evaluating Load Table Entries” Turn to page 120 and complete training exercise 1

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Global Variable Functions aka Triggers  Specified in 998 or 999 lines that have in Element 12,  Controls how the incoming record is loaded  Options are controlled on a “global” level and affect all records loaded

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Global Variable Functions  Used to control the following  overlay options  default record templates used  holdings symbol processing  field protection  automatic dates in records  and many more load options...

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Commonly Used Triggers  These triggers are found in most load tables: (not found in Authority load tables) (not found in Patron load tables)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  This trigger identifies the primary record to be created during the load  Will always correspond with Element 11, Pass Indicator  This can only be set to:  “b” for Bibliographic records  “a” for Authority records  “p” for Patron records

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  This trigger is used to determine which records can have MARC fields  The settings of this trigger corresponds with Element 9, MARC/Non-MARC Indicator  Typically this is set for Bibliographic, Item, Holdings/Checkin and Authority records (and not Patron records)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  This trigger adds a message to the top of the Output Messages display when a record load starts in Data Exchange  Usually used to specify types of records loaded

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  This trigger is used to specify the load table (m2btab file) during interactive loading  For all load tables (except Authority load tables) this trigger equals the m2btab’s file extension  Authority load tables do not have this does the same for those tables

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  This trigger is used to set the Authority load table to use  In Authority load tables, this should equal the m2btab’s file extension  In Bibliographic load tables, this should equal “a”  This trigger is not found in Patron load tables

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Updating/Creating Load Profiles  X > Load Profile Maintenance  Naming conventions  Options for editing load tables (m2btab)  Verifying load tables and checking for errors  Saving changes to load tables  Restoring load tables if mistake is made

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Practice Exercise 1 “Copying and Editing an Existing Load Table (m2btab)” Turn to page 111 and complete practice exercise 1

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Most Commonly Used Overlay Triggers

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  Sets the overlay match point  Match point must be an indexed variable-length field  Provisions can be made for specific matches on field group tag, MARC tag and index tag  Alternate match points can be set

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  Sets the action to be taken if a match is found or not found  Possible actions are:  Insert a new record  Overlay an existing record  Link records to a Bibliographic record  Reject the record

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  Specifies fields to protect from overlay  Can be set for variable-length (V) and/or fixed-length fields (F)  Variable-length fields can be protected based on field group tag and MARC tag  Variable-length fields have a variety of protect options (:d and :k)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Overlay Based on Innovative Record Number is used to overlay using the Innovative record number as the match point  Options include trigger overrides triggers

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Overview of Overlay Triggers MOST USED LESS USED

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Overlay using Innovative record number as match? Add record to the database Overlay existing record Do not add record to the database to overlay record Yes BEGIN of record load END of record load 1 1 “t” “r” No “i”

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Overlay using Indexed Control Number as Match 1 or 2 matches Additional match 0 matches Add record to the database rules and overlay existing record Do not add record to the database 2 2 BEGIN of Record load Reject Insert END of record load 1 1 match? No Reject Overlay Yes Insert

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Training Exercise 2 “Practice Applying Overlay Triggers” Turn to page 122 and complete training exercise 2

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Busy Record Handling Triggers  Loading program will not overlay “records in use by system”  Two triggers control the handling of “busy” records: – insert busy records? – save busy records to file that can be loaded later

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Automatic Date Selection Triggers  Load system date into fixed-length fields - automatic Cat. Date - automatic Order Date - automatic Recv Date

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  Innovative staff can construct a Holding Symbol table to issue loading instructions for:  Bibliographic Call Number  Bibliographic Location code  6XX fields based on 2nd indicator values (e.g., value of 0 to load LCSH only)  Holding symbol processing is activated trigger

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  Fixed-length field values are assigned from the default Record Template when no value for the fixed field is present in the incoming record  The format (“pass 0, pass 1”)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative  Linked (aka attached) records are loaded based on settings trigger  The format of this @link=“i:1:#1”

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Additional Triggers (applicable if your library has multiple accounting or serials units)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Practice Exercise 2 “Specifying Record Templates, the Cat. Date Field and Creating Linked Records” Turn to page 116 and complete practice exercise 2

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Special Processing Functions (%)  Specified in load lines that have “%” in Element 12, e.g. %008=“y”  MARC tag is processed in a special way when loaded  Examples of usage:  001 and 008 field processing  Encrypt patron PINS  Foreign currency conversion  Activate a translation table (m2bmap)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. %001  This function has five parts: %001(bcode,char,skip,start,valid)  bcode - RLIN code to Bib Code 1; set to “n” since RLIN db no longer used  char - valid characters for 001 field  skip - number of characters to skip  start - valid chars. for start of field  valid have an OCLC prefix?

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Commonly Used Special Processing Functions  %008 - places Leader after 008  %bracket - place [ ] around GMD  %encryptpin – encrypt patron PIN  %foreign – foreign currency processing  %replace – single character replacement  %map - activates a translation table (m2bmap)  %first and %last – select first or last MARC tag occurrence

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Training Exercise 3 “Using Special Functions” Turn to page 123 and complete training exercise 3

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Command Functions (#)  Specified in 998 or 999 lines that have “#com” in Element 12  For example:  Activated trigger during OCLC interactive downloading (not batch)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. More on Command Functions  Functions to:  Specify fixed-length field information  Activate triggers from command line  Create linked records  Can new command functions be created locally?  See the Innovative Guide and Reference for more information about these commands

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Practice Exercise 3 “Troubleshooting Load Tables 101” Turn to page 117 and complete practice exercise 3

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Translation Tables (m2bmap)  Used to change the data in incoming record during the load, e.g. change ABC to 123  Translations are stored in an m2bmap file  Activated in m2btab with the special processing function %map  More than one m2bmap file can be used in a load table

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Data Elements in Translation Tables |  Comparison expression is a string or regular expression to search for in incoming data file  If match is found, comparison expression will be replaced by the replacement expression

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Data Elements in Translation  Global instructions appear at top of file and start  They affect all lines in the m2bmap file

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Example of Simple Juv Ref|mjref South|s SJUV|sjuv ant|manth JUV|mjuv

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Using Regular Expressions  Regular Expressions can be used in comparison expressions  These make use of “meta-characters” listed on page 24  More information on regular expressions can be found in any basic UNIX manual

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Syntax for Regular Expressions  Comparison expressions (.*)$0|d(.*)$1  Replacement expressions \0|d19\1

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Tips on Using Translation Tables  Order of lines in a map file is important  longest matching field should appear first  Avoid unwanted case changes  Loading data into fixed-length fields  byte count in Element 5 must be > largest comparison  Must activate in load table  Specify m2bmap in Element 12 using %map trigger

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Creating and Updating Translation Tables X > Load Profile Maintenance E > Edit m2btab and m2bmaps T > TRANSLATION Tables (m2bmap) N > Create a NEW m2bmap file  Use %map trigger to activate translation table in m2btab  Must load records to test m2bmap file

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Training Exercise 4 “Translation Tables (m2bmap) ” Turn to page 124 and complete training exercise 4

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Data Analysis before Profiling  Issues to consider before Profiling - MARC tags to load - Creating new records or overlay existing records - Remapping of MARC data - Loading linked records

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Use Data Exchange to Transfer and Prep Files  Launch Millennium Cataloging  Go | Data Exchange  Select Process: Load records via locally-created load profiles (.local)  Data file must be transferred with “.lfts” file extension (use “Get FTS” or “Get PC” buttons to transfer files to server)  “Prep” file to create “.lmarc” file for loading

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Use Data Exchange to View Records  Click on “.lfts” or “.lmarc” file  Click “View” button  Viewing features include:  “Count” button (counts number of records and MARC tags)  “Search” button (to find records that match specific criteria)

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. The m.marcload.local file  What does this file do?  Data elements in this file  Edit this file to create “Load” buttons for your new load tables X > Load Profile Maintenance L > Edit m.marcload.local change –fzzz to –f  Test new “Load” buttons in Data Exchange

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Practice Exercise 4 “Applying Required Changes to an Existing Load Table (m2btab)” Turn to page 118 and complete practice exercise 4

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Test Load Files Using Data Exchange  Admin | Clear Initials  Admin | Reset Initials  Go | Data Exchange  Select Process: Load records via locally-created load profiles (.local)  Click once on “.lmarc” file  Click correct “Load” button  Click “Test” button at the bottom of screen

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Practice Exercise 5 “Overlaying of Bibliographic Records” Turn to page 119 and complete practice exercise 5

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Review: Creating a New Load Table (m2btab)  Plan load table settings  View MARC records to be loaded  Copy an existing load table  Verify and check load table for errors  Restore load table if errors are made

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Review: Using Translation Tables (m2bmap)  m2bmaps are optional  But if you need one, remember to activate the m2bmap in the m2btab file  Specify m2bmap in Element 12 using %map trigger

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Review: Editing m.marcload.local  Add m2btab file extension or brief description to Element 1 to distinguish between “Load” buttons  For example: C > LOAD a MARC file (.pbursar)  Use Clear Initials and Reset Initials to activate new “Load” buttons in Data Exchange

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Support from Innovative  Consultation   csdirect.iii.com  Technical Assistance  Profiling Services  Product Code 830 

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Review of Training Goals  Understand the database structure  Understand existing load tables (m2btab)  Be familiar with issues to be considered for new load profiles  Update and create load tables  Create simple translation tables (m2bmap)  Add “Load” buttons to Data Exchange  Test load profiles before production

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces. Trainer Evaluation  Workshop Evaluation Form is available on csdirect.iii.com  Select “Contact Us” in upper right corner  "Feedback" from left column  "Workshop Feedback" from right column  Please take a few minutes to fill out the evaluation. Thank you!

Copyright Innovative Interfaces. Not to be duplicated or distributed without permission of Innovative Interfaces.