1. Indexing Innovations
14.2 Seminar
14.1 Seminar - Filing Procedures

2. Session Agenda Filing and word breaking procedures
Indexing procedures - new features:
Parallel processing
Updating a group of indexes
New indexing routines
Filing Procedures
14.1 Seminar - Filing Procedures

3. Parallel Processing
Filing Procedures
14.1 Seminar - Filing Procedures

4. Parallel Processing
Problems with indexing batch routines in the early versions of ALEPH:
Long run time
Computer resources not fully utilized -
single process per stage
No recoverability – if indexing failed,
the whole building process needed to
be rerun
Filing Procedures
14.1 Seminar - Filing Procedures

5. Parallel Processing In 14.2 all the index creation jobs
(with the exception of
p_manage_27)
enable parallel processing.
Filing Procedures
14.1 Seminar - Filing Procedures

6. Parallel Processing
Optimal utilization of computer resources (Large databases - multiple processors)
Certain stages of index creation can be split into several cycles – this allows you to divide the workload among different processors
Indexing is much quicker
Filing Procedures

7. Parallel Processing – Tracking
Assignment progress table: good control of indexing stages ? ?
0004 ?
0005 ?
0006 ?
+ success
? in process
- not processed
Filing Procedures

8. Parallel Processing - Recovery
If:
database tables need to be enlarged
not enough disk space - intermediate files
not enough disk space - sort
general disaster
You do not have to rerun the whole process!
Filing Procedures

9. Parallel Processing - Recovery
Recovery stages:
identify last successful section
change “in process” signs (?) to “not processed” sign (-)
rerun discrete stage scripts:
For example:
p_manage_01_a
p_manage_01_c
p_manage_01_d
p_manage_01_d1
Filing Procedures

10. Parallel Processing – Main Features
Indexing is quicker
Tracking is easier
Recoverability is possible
Filing Procedures

11. Updating a Group of Indexes
Filing Procedures
14.1 Seminar - Filing Procedures

12. Updating a Group of Indexes
p_manage_01 and p_manage_02 have a new feature allowing you to update a specific group of indexes.
Col.8 defines a group of headings/word indexes for updating:
11 W F A WRD WYR
11 W F A WRD WLN
11 W LOC## o WRD WCL
11 W 041## abdefg A WRD WLN
Filing Procedures
14.1 Seminar - Filing Procedures

13. Updating a Group of Indexes
This option is only available when the program is run from the prompt command line. It is not available from the Web Services.
The following is an example of the way in which the program should be run for fields that belong to group B:
csh -f p_manage_02 USM01,1, , ,B,1,0,00,
csh -f p_manage_01 USM01,1, , ,B,1,0,00,
Filing Procedures
14.1 Seminar - Filing Procedures

14. Z0102 – COUNTERS FOR LOGICAL BASES
Filing Procedures
14.1 Seminar - Filing Procedures

15. z0102 Pre-14.2 – Problem: Solution: Filing Procedures
Scanning logical bases which are less than 50% of the total database is very inefficient
(slow, irrelevant unlinked headings )
Solution:
There is a new index z0102 which ‘divides’ z01 into sections in accordance with the existing logical bases.
Filing Procedures
14.1 Seminar - Filing Procedures

16. z0102 Filing Procedures Example of z0102 record:
14.1 Seminar - Filing Procedures

17. z0102 When a logical base is being browsed,
the system uses the Z0102 table to “decide” whether to display the heading (Z01) without having to retrieve the documents attached to the heading, Read them, and then “decide”.
Filing Procedures
14.1 Seminar - Filing Procedures

18. z0102
Structure:
A record is built for each Z01 and each logical base,
giving the filing text and sequence (in order to
make the SCAN more efficient) and a counter of the
number of relevant docs. Records are built for "see"
reference headings, as well as for preferred headings.
The record does not include pointers to the doc records;
this is still done by Z02.
Filing Procedures
14.1 Seminar - Filing Procedures

19. z0102 Filing Procedures Run p_manage_32 to create z0102
Building the table:
Run p_manage_32 to create z0102
Run p_manage_34 to update z0102
- p_manage_32 runs on all Z01 records and builds Z0102.
When p_manage_02 is run, p_manage_32 should be run
directly afterwards.
- p_manage_34 runs on Z01 records that have been "touched"
since the last time 32 or 34 were run. It should be run on
a regular basis -- i.e. nightly, listed in the job_list
(UTIL E/15/1).
Filing Procedures
14.1 Seminar - Filing Procedures

20. z0102 Z01 records that have been "touched“… Filing Procedures
- Z01 has a new field, Z01-UPDATE-Z0102.
- p_manage_02 set this flag to "Y".
p_manage_32 and _34 set this flag to "N".
update of z01 sets Z01-UPDATE-Z0102 is set to"Y".
- p_manage_34 re-indexes Z01 records that have
Z01-UPDATE-Z0102 = "Y".
Filing Procedures
14.1 Seminar - Filing Procedures

21. z0102 Restrictions: Filing Procedures
Z0102 is used only for the WEB OPAC browse
A new switch in the WEB OPAC defines which tables are involved in BROWSE.
If TAB10-Z0102-IN-USE = ‘Y’ – browse is performed by z0102
If TAB10-Z0102-IN-USE = ‘N’ –z0102 does not participate in BROWSE
Presently, there is no online update of z0102.
Filing Procedures
14.1 Seminar - Filing Procedures

22. New Batch Jobs for AUT Enrichment
Pre – 14.2 :
AUT enrichment and correction of BIB after initial conversion or re- indexing is very time-consuming (it takes up to several days).
Solution:
New batch jobs for AUT enrichment and correction of BIB libraries.
These batch jobs will replace the background running of ue_08 after
a re-indexing of the z01 indexes.
p_manage_102: enrich the BIB z01 index from the entire AUT
library
p_manage_104: reset the Z01 created from regular indexing to
"-CHK-" status
p_manage_103: send Z07 records to all potential "corrected”
BIB docs.
Filing Procedures
14.1 Seminar - Filing Procedures

23. New Batch Jobs for AUT Enrichment
p_manage_102: enrich the BIB z01 index from the
entire AUT library
p_manage_104: reset the Z01 created from regular
indexing to "-CHK-" status
p_manage_103: send Z07 records to all potential
"corrected" BIB docs.
Filing Procedures
14.1 Seminar - Filing Procedures