How to use the job list Version 16 - 18 Yoel Kortick

All of the information in this document is the property of Ex Libris Ltd.  It may NOT, under any circumstances, be distributed, transmitted, copied, or displayed without the written permission of Ex Libris Ltd. The information contained in this document has been prepared for the sole purpose of providing information for the members of the MELI users group.  The material herein contained has been prepared in good faith; however, Ex Libris disclaims any obligation or warranty as to its accuracy and/or suitability for any usage or purpose other than that for which it is intended. © Ex Libris Ltd., 2006 How to use the job list

Introduction The job list, also known as Job Daemon, is a file containing specific jobs which can be run on a regular basis via fixed parameters The job list can contain two types of jobs: ALEPH programs UNIX procedures As we will see, the job list is similar to the UNIX crontab, but contains many advantages. How to use the job list

Accessing the job list The job list may be edited via: UTIL E 16 / 1 (UTIL E 15 is used for other job list purposes discussed later) vi $alephe_tab/job_list How to use the job list

Accessing the job list How to use the job list

Defining the job list We will begin with defining ALEPH programs in the job list Let us assume that we run the service “Courtesy Notices (cir-10)” on a regular basis How to use the job list

Defining the job list – ALEPH program We send the service one time from the services: How to use the job list

Defining the job list – ALEPH program Then we check the parameters of the service via the log file in $alephe_scratch. Here the relevant log file is called usm50_p_cir_10.00200 il-aleph02-18(1) USM50-YOELK>>cd $alephe_scratch il-aleph02-18(1) USM50-YOELK>>ls -lrt *p_cir_10* -rw-rw-r-- 1 aleph exlibris 28791 Oct 24 08:27 usm50_p_cir_10.00200 How to use the job list

Defining the job list – ALEPH program If the file will be opened (in editor) the parameters will be visible. -------------------------------------------------------------- Load: /exlibris/aleph/a18_1/alephe/tab/tab100 Load: /exlibris/aleph/a18_1/usm50/tab/tab100 ^[[2J^[[H^[[0m ALEPH/LINUX, Copyright Ex Libris. version 18 revision 01 copy 1, 15-Oct-2006 start USM50,cir10_out,Y,,Y,,030,00,N,2,N,N,3, procedure=p_cir_10 Fixed param: USM50,cir10_out,Y,,Y,,030,00,N,2,N,N,3, setenv p_active_library "USM50" setenv p_file_out "cir10_out" Now we will take these parameters and add them to the job_list file How to use the job list

Defining the job list – ALEPH program We open job_list either via UTIL E 16 or via vi editor and add the following line: 3. Send to batch queue? Y/N 4. Name of target (more later) 6. Name of job (ALEPH program) ! 2 3 4 5 6 7 !!-!!!!!!!!-!-!!!!!!!!!!!!!!!!!!!!-!!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!-!!!!!! 02 08:56:10 Y USM50 p_cir_10 USM50,cir10_out,Y,,Y,,030,00,N,2,N,N,3, 1. Day Sunday=00 Monday=01 Tuesday =02 Wednesday=03 Thursday=04 Friday=05 Saturday=06 2. Time 5. Library name 7. Job parameters (taken directly from $alephe_scratch log file) * See table header and remainder of presentation for additional information How to use the job list

Defining the job list – ALEPH program After editing the job list the job daemon must be reactivated via UTIL E 15 / 1 How to use the job list

Defining the job list – ALEPH program The job was sent for Tuesday (02 in column 1) for time 08:56 and 10 seconds (column 2). It was sent and the log file appears in $alephe_scratch: il-aleph02-18(1) USM50-YOELK>>cd $alephe_scratch il-aleph02-18(1) USM50-YOELK>>ls -lrt *p_cir_10* -rw-rw-r-- 1 aleph exlibris 28791 Oct 24 08:27 usm50_p_cir_10.00200 -rw-rw-r-- 1 aleph exlibris 27221 Oct 24 08:58 usm50_p_cir_10.00211 The job we sent via the services The job we sent via the job list. It was sent at 08:56 and 10 seconds and finished at 08:58 How to use the job list

Viewing the job list log Any time a job is sent via the job list it updates a file “jobd.log” in the $alephe_scratch directory il-aleph02-18(1) USM50-YOELK>>cd $alephe_scratch il-aleph02-18(1) USM50-YOELK>>ls -lrt jobd.log -rw-rw-r-- 1 aleph exlibris 2563 Oct 24 09:05 jobd.log The file “jobd.log” may be viewed via “vi” editor or via UTIL E 15 / 4 How to use the job list

Viewing the job list log Here is the section in file “jobd.log” for the p_cir_10 which we sent to be run Tuesday at 08:56 and 10 seconds: Current time: Tuesday 24 October 2006 08:56:10 [1161672970] Performing Jobs: ---------------- USM50 p_cir_10 USM50,cir10_out,Y,,Y,,030,00,N,2,N,N,3, Wait for: 110 seconds Current time: Tuesday 24 October 2006 08:58:01 [1161673081] How to use the job list

Using job_list.conf In addition to $alephe_tab/job_list, another file exists called $alephe_tab/job_list.conf. job_list.conf is used for the scheduling of “regular” jobs that run via the job daemon. For example, a specific job may be sent every day except Saturday and Sunday. Instead of putting this job five times into job_list, we can use job_list.conf to make a parameter meaning “Monday to Friday”. How to use the job list

Using job_list.conf $alephe_tab/job_list.conf may be edited via the vi editor or via UTIL E 16 / 5 How to use the job list

Using job_list.conf Typical setup in job_list.conf ! 1. Code ! 2. W(eekly) ! 3. Flags (Sun - Sat) Y/N ! Define daily procedure slot: !1 2 3 4 5 !!----!-!!!!!-!!!!!-!!!!! W1 W NYYYYYN W2 W YYYYYNN W3 W YYYYYYY W1 means every day except Sunday and Saturday W2 means every day except Friday and Saturday W3 means every day How to use the job list

Using job_list.conf Thus if we have this job in job list and the previous setup in job_list.conf W3 09:45:00 Y hacolbised USM50 p_cir_51 USM50,summary_letter_overdue,Y,,Y,,,,00,00,N,N,O,4, The job will run every day at 09:10, because W3 means every day How to use the job list

Using print daemon with job_list It is possible to define a “target” in column 4 of job_list. This “Target” is a suffix will be added to the output file of the service. If a print daemon is running which has the name of the target, then the printout will be printed to default printer of the PC on which the print daemon is running. For more information on the print daemon see accompanying document “How_to_set_up_and_use_print_daemon.doc” How to use the job list

Using print daemon with job_list In this example we have print daemon “hacolbised” in column 4 for job “p_cir_51” which runs every day at 09:45. Column 4 is the “target”, which is equivalent to the “Print ID” when running a service. W3 09:45:00 Y hacolbised USM50 p_cir_51 USM50,summary_letter_overdue,Y,,Y,,,,00,00,N,N,O,4, Same thing How to use the job list

Using print daemon with job_list When the job runs the log file in $alephe_scratch will get the suffix of the “target” from the job_list: il-aleph02-18(1) USM50-YOELK>>cd $alephe_scratch il-aleph02-18(1) USM50-YOELK>>ls -lrt *hacolbised -rw-rw-r-- 1 aleph exlibris 26424 Oct 24 09:45 usm50_p_cir_51.00220.hacolbised When a report is printed it too will get the suffix of the “target” from the job_list: How to use the job list

Using date parameters in the job_list We may wish to run a retrieval of records which will include the last week only. Here we run p_ret_01 to retrieve records updated between 18/10/2006 and 24/10/2006 How to use the job list

Using date parameters in the job_list If we were to put these dates in the job_list then the dates would be hardcoded: always 18/10/2006 and 24/10/2006 USM01,,last_week,000000000,999999999,,00,00000000,99999999,20061018,20061024,,AND,NOT,,,,,,,,,,,,,,,,,,,,,,00000000,99999999,MASTER, This would not be good because we want it to always be between today and 7 days ago. We want to add it one to job_list and have job_list always “know” that we mean “today” and “one week ago”. How to use the job list

Using date parameters in the job_list Thus we could put this in job_list: ! 2 3 4 5 6 7 !!-!!!!!!!!-!-!!!!!!!!!!!!!!!!!!!!-!!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!-!!!!!! 02 10:30:00 Y hacolbised USM01 p_ret_01 USM01,,last_week,000000000,999999999,,00,00000000,99999999,%DATE-D007,%DATE,,AND,NOT,,,,,,,,,,,,,,,,,,,,,,00000000,99999999,MASTER, Here is the syntax: %DATE = today (day job runs) %DATE-D007 = today minus 7 days %DATE+D003 = today plus 3 days %DATE+M003 = today plus 3 months %DATE-M002 = today minus 2 months How to use the job list

Using date parameters in the job_list Thus if we have this in job_list: ! 2 3 4 5 6 7 !!-!!!!!!!!-!-!!!!!!!!!!!!!!!!!!!!-!!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!-!!!!!! 02 10:30:00 Y hacolbised USM01 p_ret_01 USM01,,last_week,000000000,999999999,,00,00000000,99999999,%DATE-D007,%DATE,,AND,NOT,,,,,,,,,,,,,,,,,,,,,,00000000,99999999,MASTER, And we run the job on October 24th, these will be the parameters: USM01,,last_week,000000000,999999999,,00,00000000,99999999,20061017,20061024,,AND,NOT,,,,,,,,,,,,,,,,,,,,,,00000000,99999999,MASTER, setenv p_update_date_from "20061017" setenv p_update_date_to "20061024" How to use the job list

Using date parameters in the job_list It is also possible to add a date suffix to the output file in the job_list. This will prevent various files from overwriting each other. If each day a file named “last_week” was sent to the print directory, it would always overwrite the file of the previous day. How to use the job list

Using batch queue in the job_list It is also possible to send ALEPH jobs to the batch queue via the job list This is done by putting a Y in column 3 We may send two jobs, and not want the second job to start until the first job finishes. The first job may build an input file for the second job, and thus the second job would not work if it started before the first job finishes. How to use the job list

Using job queue in the job_list Here we run the service p_print_01 with input file “last_week” and output file “last_week_report” How to use the job list

Using job queue in the job_list We check the parameters in the $alephe_scratch log file start USM01,last_week,last_week_report,037,00, procedure=p_print_01 Fixed param: USM01,last_week,last_week_report,037,00, setenv p_active_library "USM01" setenv p_file_name_in "last_week" setenv p_file_name_out "last_week_report" setenv p_format_type_x "037" setenv p_level "00" How to use the job list

Using job queue in the job_list The we add the parameters to job_list along with the job p_print_01 !!-!!!!!!!!-!-!!!!!!!!!!!!!!!!!!!!-!!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!-!!!!!! 02 10:59:00 Y hacolbised USM01 p_ret_01 USM01,,last_week.%DATE,000000000,999999999,,00,00000000,99999999,%DATE-D007,%DATE,,AND,NOT,,,,,,,,,,,,,,,,,,,,,,00000000,99999999,MASTER, 02 10:59:00 Y hacolbised USM01 p_print_01 USM01,last_week.%DATE,last_week_report.%DATE,037,00, Here we have p_ret_01 making output file which will be called “last_week[.date]” and it will retrieve records created between [today] and [today – 7 days] We then run p_print_01 using input file “last_week[.date]” which was created from p_ret_01 and then it makes output file “last_week_report[.date] If p_print_01 runs every day then the output file will not overwrite previous files because it always has date suffix. How to use the job list

Using job queue in the job_list !!-!!!!!!!!-!-!!!!!!!!!!!!!!!!!!!!-!!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!-!!!!!! 02 10:59:00 Y hacolbised USM01 p_ret_01 USM01,,last_week.%DATE,000000000,999999999,,00,00000000,99999999,%DATE-D007,%DATE,,AND,NOT,,,,,,,,,,,,,,,,,,,,,,00000000,99999999,MASTER, 02 10:59:00 Y hacolbised USM01 p_print_01 USM01,last_week.%DATE,last_week_report.%DATE,037,00, Both jobs, p_ret_01 and p_print_01 are both set to run at 10:59. However, column 3 has a Y and thus they are sent to the batch queue. As a result, p_print_01 will not start before p_ret_01 finishes. How to use the job list

Using job queue in the job_list We see p_ret_01 running via UTIL C / 1 *** util_c_01 - check USM01 batch queue *** 22459 ? S 0:04 /exlibris/aleph/a18_1/aleph/exe/lib_batch USM01 4622 ? S 0:23 /exlibris/aleph/a18_1/aleph/exe/rts32 ue_01_a USM01.a18_1 4623 ? S 0:15 /exlibris/aleph/a18_1/aleph/exe/rts32 ue_01_z0102_index USM01.a18_1 4633 ? S 0:09 /exlibris/aleph/a18_1/aleph/exe/rts32 ue_01_word_index USM01.a18_1 4744 ? S 0:02 /exlibris/aleph/a18_1/aleph/exe/rts32 ue_08_a USM01.a18_1 C 5844 ? S 0:00 csh -f /exlibris/aleph/a18_1/aleph/proc/p_ret_01 USM01,,last_week.20061024,000000000,999999999,,00,00000000,99999999,20061017,20061024,,AND,NOT,,,,,,,,,,,,,,,,,,,,,,00000000,99999999,MASTER, hacolbised Enter to continue We see p_print_01 waiting in queue via UTIL C / 7 *** util_c_07 - USM01 jobs waiting for exec *** Jobname Target Entry Status ------------------------------ -------------------- ----- -------------------------- p_print_01 hacolbised 00345 Holding until 20061024 00 press return for main menu How to use the job list

Using job queue in the job_list Both jobs complete and the output file, with target name, goes to $data_print il-aleph02-18(1) USM01-YOELK>>cd $data_print il-aleph02-18(1) USM01-YOELK>>ls -lrt last_week_report.20061024.hacolbised -rw-rw-r-- 1 aleph exlibris 12269 Oct 24 11:01 last_week_report.20061024.hacolbised How to use the job list

Using job queue in the job_list If the print daemon “hacolbised” will be running with library USM01: How to use the job list

Using job queue in the job_list Then the file will automatically print to the default printer and then get sent to a sub directory with the date in which it was printed: il-aleph02-18(1) USM01-YOELK>>cd $data_print il-aleph02-18(1) USM01-YOELK>>ls -lrt | tail -1 drwxrwxr-x 2 aleph exlibris 4096 Oct 24 11:09 save-2006-10-24/ il-aleph02-18(1) USM01-YOELK>>ls -lrt save-2006-10-24/ total 12 -rw-rw-r-- 1 aleph exlibris 12269 Oct 24 11:01 last_week_report.20061024.hacolbised-11-09-27 For more information on the print daemon see accompanying document “How_to_set_up_and_use_print_daemon.doc” How to use the job list

Non ALEPH jobs in the job_list It is also possible to send UNIX scripts via the job_list. It is advantageous to use the job_list for these types of scripts instead of the UNIX crontab because all of the jobs (ALEPH and non ALEPH) will be together. This will prevent conflicts such as a backup script running while an indexing script is running. When using non ALEPH scripts in job list the “send to queue” column (3) should be N. How to use the job list

Non ALEPH jobs in the job_list Here we have a simple non ALEPH script which performs UNIX command “df –k”, send the results to a file, copies file to $alephe_scratch, and also emails the file to a recipient: il-aleph02-18(1) USM01-YOELK>>cat /exlibris/aleph/a18_1/alephe/private/script touch /exlibris/aleph/a18_1/alephe/private/dfk df -k >> /exlibris/aleph/a18_1/alephe/private/dfk cp /exlibris/aleph/a18_1/alephe/private/dfk /exlibris/aleph/a18_1/alephe/scratch cat /exlibris/aleph/a18_1/alephe/private/dfk | mail -s "dfk on ALEPH server" "yoel.kortick@exlibris.co.il" The script may be run from the command line as follows: il-aleph02-18(1) USM01-YOELK>>csh -f /exlibris/aleph/a18_1/alephe/private/script The command here may be put in the job list to run regularly – example in next slide How to use the job list

Non ALEPH jobs in the job_list The command may be put in the job list to run regularly: ! 2 3 4 5 !!-!!!!!!!!-!-!!!!!!!!!!!!!!!!!!!!-!!!!!!!!!!!!!!> ! 2 3 4 5 6 7 !!-!!!!!!!!-!-!!!!!!!!!!!!!!!!!!!!-!!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!-!!!!!! W3 11:30:00 N df-k_log csh -f /exlibris/aleph/a18_1/alephe/private/script And every day at 11:30 the script will run: il-aleph02-18(1) USM01-YOELK>>cd $alephe_scratch/ il-aleph02-18(1) USM01-YOELK>>ls -lrt | tail -2 -rw-rw-r-- 1 aleph exlibris 1908 Oct 24 11:30 dfk -rw-rw-r-- 1 aleph exlibris 0 Oct 24 11:30 df-k_log_18393 How to use the job list