Managing SX.e and TWL with scripts and MARC 02/12/04 Jeremiah Curtis

Assumptions You should understand: 1.MARC -- What it is and generally what it does 2.Unix -- what it is and basic commands 3.Shell Scripts -- what they are and what they are used for 4.Basic understanding of TWL (for the TWL specific portions – you can nap during these portions if you are not concerned with TWL)

Once upon a time… … customers as well as NxTrend personnel identified the need to have a standard set of scripts MARC would generate the standard set Everyone would be happy

The Shadow... NxTrend support identified needs that the MARC scripts did not address The product evolved, but MARC was not well suited to this change So, there became two sets of scripts –GFI for Go For It –MARC

Confusion Reigned... There was confusion –Which scripts should be used? –Which were getting installed?

An Alliance was Forged... The decision came that there should be only one set of scripts MARC should generate these scripts MARC should be flexible enough to handle future needs

Harmony was Restored... All scripts will be generated by MARC and are stored as templates No longer hand editing scripts Script issues will begin by regenerating all scripts New scripts are easily added and existing ones easily updated

The End

The Changes The next section describes the current set of scripts

New Scripts and New Names Many new scripts have been added to MARC’s script base Many more scripts had a name change to standardize names and functions

The Progress Admin Server These scripts help manage the Progress Admin and Name Servers admin.start admin.shut admin.query nameserver.query

The Database These scripts help manage the database brokers dbbroker.start dbbroker.shut dbbroker.query These are database utilities dbstats truncate.bi

The Application Broker These scripts help manage the Progress Application Brokers appbroker.start appbroker.shut appbroker.query

Parameter Files The following are a new set of parameter files for client connections client.pf editor.pf batch.pf singleuser.pf connect.pf

The Environment These scripts help manage this particular environment nxt.env nxtall.start nxtall.shut

Enterprise To start SX.enterprise, now use sxe sxe sxee

The Report Manager These are the Report Manager scripts rptmgr (or rptmgr.start) rptmgr.shut

Binary Dump and Load These scripts are for Dump and Loads easybdsareabd bddriver.psareabddriver.p bddumpall.pbddumparea.p bdloadall.pbdloadarea.p bdscript.psareabdscript.p

After Imaging These scripts are for AI aimage.start aimage.shut aimage.backedup aimage.list aimage.archive aimage.new aistats rollforward.ai truncate.ai

Backups These are new standard backup scripts backup.data backup.full backup.online backup.files

RxServer RxServer scripts are now generated by MARC rxsfax.sh rxsprint.sh out.sh

Other Scripts These don’t fit anywhere else rdclean sassi.sh shutuser showuser config_rpt

TWL TWL scripts have been mostly rewritten MARC generates all of the needed TWL scripts TWL scripts now follow a standard naming convention

Additional TWL Application Broker These scripts help manage the Application Broker that manages TWL connections to the SX.enterprise database twlentappbroker.start twlentappbroker.shut twlentappbroker.query TWL also uses the normal appbroker scripts which are generated from TWLMARC. These appbrokers are for connections to the TWL database

TWL RF Units These scripts are used to start the TWL application on RF units twlrf twlentrf twlentrf.pf

Additional TWL Environment Since TWL integrates with SX.enterprise, an additional environment is available using: twlsecondary.env

TWL End of Day These scripts are used to run TWL End of Day processing. These are usually called from the backup scripts The first script performs processing that requires a single-user connection to the DB while the second performs tasks using a normal multi-user connection to the DB twleod1 twleod2

TWL Interfaces The following scripts are used to manage the TWL interfaces twlsend.starttwlrcvpack.sh twlsend.shtwlrcv.sh twlsend.shuttwlrcv.shut twlsend.querytwlrcv.query

The Templates The next section describes the new Script Generator in MARC

MARC Generate Scripts MARC Databases -> Configure -> Generate Scripts Select the database(s) to generate scripts for Select the scripts to generate

Script Template All scripts generated by MARC are stored as templates These templates contain the form of the final script Critical portions of a template contain tags These tags get substituted with real values to generate a script

MARC Generate Scripts All of the templates reside in subdirectories under /rd/marc/scripts/template All templates must end with a.tpl These subdirectories are: base dg hp ibm sco intchan taxware twl

All of the templates in base are added to the list One system directory is included based on the selected machine type in Installations -> Configure (e.g. ibm ) If optional products are setup in MARC, then those templates are also included in the list (e.g. twl ) The custom directory is always checked first and overrides a standard template of the same name MARC Generate Scripts

Script templates may also be in /rd/marc/scripts/cust in the same subdirectories Copy the template from the directory under /rd/marc/scripts/template Custom Scripts

Template Tags There are two types of template tags –System tags (defined by MARC) –Custom tags (defined by the user) Tags are names enclosed in ‘ ’ –Example: The convention is to use tag names that are all capital letters

System Tags System tags are defined during the installation of MARC System tags contain values that MARC can get from the information in its own database System Tags can not be edited directly with the tag editor

Custom Tags Users may use the Tag Editor to create and edit tags. Custom tags are created by users or may be in some delivered scripts (e.g. rxsprint.sh) Values for the tags can be assigned in two ways: –By answering the prompt when MARC finds an undefined tag –By using the tag editor in Databases -> Configure -> Tag Editor

The Tag Editor The tag editor allows the user to: –Create new custom tags –Modify existing custom tags –Delete custom tags

Example Template Here is a simple template echo “Hello, world!” Top two lines are reserved for formatting SCRIPTNAME is what the script will be called DESTDIR tells where the script will get created

Script Naming All scripts should be prefaced with the environment name Use ENV before the script name to substitute in the environment name in the MARC that is generating the script

Destinations The destination directory can be a path The destination can be one of these predefined directories BIN OPSYS RD EXEC

Building A Script Here is a simple template “helloworld.sh.tpl” echo “Hello, world!” Should be created in the../cust/base directory Generating script to /rd/bin/

Script Templates - Summary All scripts will be generated by MARC and are stored as templates Should no longer hand edit scripts Script issues will begin by regenerating all scripts New scripts are easily added and existing ones easily updated

MARC Enhancements The next section describes some changes in Progress 9 and changes in MARC 10.1

First, Some Background With the advent of Progress 9, there are some new concepts to understand MARC 10.1 also contains many changes to accommodate the new version of Progress as well as a few enhancements

Storage Areas The Progress database has changed to incorporate a new concept of storage areas Storage areas are a logical grouping of tables Storage areas are also in their own files on disk and have extents Oh, and by the way, all Progress 9 databases are multi-volume and will have extents

Progress Admin and Name Servers MARC can now configure the ports used by the Progress Admin and Name Servers –Admin Server Port –Admin Server Admin Port –Name Server Port Allows for multiple installs of Progress –Live: /rd/dlc –Test: /test/rd/dlc

Installation Configure Screen New Environment Name Online Backup Directory AI Save Directory Layout Changes

Database Configuration With storage areas, MARC has the ability to configure extents for the storage areas Each storage area can be configured to have a certain number of fixed extents plus the size for the fixed extents

AppServer Changes The AppServer source code is not on the Unix server anymore Therefore the AppServer is not compiled from MARC anymore The AppServer is pushed from the Staging Client

TWL Related Changes There are now two AppServer configurations –twlappsrv –twlsxappsrv TWL warehouse locations now only configure the name of the location

Backup Configuration Backup configuration have been completely rewritten Configuring backups have changed –They work –Tape Device –AI –Online Backup Script templates are copied for the type of backup needed

Shutuser The Disconnect User function has been completely rewritten MARC now calls the shutuser script –The script is generated by MARC –It works –There is a lookup available (F12) to see users logged in

Dump and Load There are two types of dump and load –For an upgrade which included a Progress 9 upgrade, a dump and load is needed to put the data into the correct storage areas –For maintenance, a dump and load is used to realign data and improves performance –There are different functions for each type of dump and load

Maintenance Dump and Load Reworked in MARC 10.1 –A storage area can be dumped and loaded –The dump and load is based on scripts proven to work

Script Generation The script generation module in MARC has been completely rewritten Script generation is now template-based No hacking required Script generation is logged –scriptgen log –systemtag.lst

Questions ? ? ? ? ? ? ? ? ? ? ? ?