By default, ARCHIVE processes files and job variables in save and restoration runs serially, i.e. two or more FILES or JOBVAR statements are always processed consecutively.
However, within one FILES or JOBVAR statement, particularly in the case of a system backup, it is possible to write to or read from several save volumes simultaneously (parallel processing).
Parallel processing
In ARCHIVE, parallel processing is generally important only for the system support. Normally only he/she has to deal with quantities of files large enough to make it worthwhile. However, the option of using parallel processing is available to any user.
Parallel processing in ARCHIVE is controlled by means of the DRIVES operand and, in the case of save runs, also via the FILES statement.
As many parallel runs are started as were specified in the DRIVES operand. Each parallel run is processed by one subtask. These subtasks are each assigned one device, i.e. as many parallel runs may be specified as devices are available. Each parallel run is identified with a subsave number (0,1,...). The subsave number of the first parallel run is 0.
The subtasks receive the names of the files to be processed in “packets” of information from the main task.
Each subtask processes its packet and then requests another. In the case of save runs, which files a packet contains can be controlled by means of the NAME operand in each FILES statement. This takes place as follows:
In save runs, depending on the distribution of the files to the volumes of the pubset ARCHIVE forms up to four packets for the files to be saved for each user and catalog ID; in addition, the files can be distributed to several packets by means of several NAME operands within a FILES statement.
The packets within a FILES statement are then processed in parallel by several ARCHIVE subtasks.In restoration and import runs as well as when copying save data, the division into packets is performed automatically, based on the way the data was saved to the volumes. Only the same number of subtasks may be active as were used during saving.
It may happen that a run is aborted before all the FILES statements have been executed, e.g. if the operator rejects the tape assignment with tsn.NO. In this case, the files which have not yet been transferred and were specified explicitly in the NAME operand are marked NOT PROCESSED in the ARCHIVE report.
All that has been said about the FILES statements also applies analogously to the JOBVAR statement.
Example 1
/SET-LOGON-PARAMETERS TSOS /START-ARCHIVE . . . *FILES *SAVE ...,DRIVES=3
Up to four packets are created for each user ID of the system. All the user IDs of the system are saved in parallel on 3 tape devices by 3 subtasks.
Example 2
/SET-LOGON-PARAMETERS TSOS /START-ARCHIVE . . . *FILES NAME=$ABC. *SAVE DIRECTORY=...,DRIVES=2
In this run the files of the $ABC ID are distributed to up to 4 packets and saved to two magnetic tape devices by two subtasks. However, if the pubset consists of just one volume, only one packet is generated. As a result of the DRIVES=2 operand this means that two subtasks are started, but one of these has nothing to do. Consequently no parallel processing takes place.
Serial processing
The individual FILES statements are processed in strict consecutive order: all files specified in one FILES statement are transferred before the next FILES statement is processed.
The generated subtasks wait if no further packet can be created for them from the first FILES statement.
Example 1
/SET-LOGON-PARAMETERS TSOS /START-ARCHIVE . . . *FILES NAME=$A. *FILES NAME=$B. *FILES NAME=$C. *SAVE DIRECTORY=...,DRIVES=3
In this run the three IDs are processed serially. If the files of the various IDs are distributed evenly over the various volumes of the pubset, the three subtasks are utilized evenly. However, if, for example, the pubset consists of only one volume, only one packet can be formed within the FILES statement and two subtasks are in the wait state.
Example 2
/SET-LOGON-PARAMETERS TSOS /START-ARCHIVE . . . *FILES NAME=$A. *FILES NAME=$B.,NAME=$C.,NAME=($D.AA,THRU=$D.KK9) *SAVE ....,DRIVES=3
In this save run, two subtasks remain in the wait state until the third subtask has completely saved the user ID A. Only when all the files of the user ID A. have been processed do the three subtasks process the files of the second FILES statement in parallel.
When tapes are read in, the whole tape is wound from the beginning for each FILES statement. Users should therefore consider whether or not it is really a good idea to use a number of different FILES statements.
Example 3
/SET-LOGON-PARAMETERS TSOS /START-ARCHIVE . . . *FILES NAME=HUGO1 *FILES NAME=HUGO2 *RESTORE DIRECTORY=NONE,FROM=(RGA001)...
This sequence of statements results in tape RGA001 being searched until the file HUGO1 is found. Next the tape is searched a second time for the file HUGO2. In order to find both tapes in a single search, the following sequence of statements must be entered:
*FILES NAME=(HUGO1,HUGO2) *RESTORE DIRECTORY=NONE,FROM=(RGA001)...
The FILES statements are processed in strict consecutive order and therefore no coupling of the NAME operands in the individual FILES statements takes place. Files specified in more than one FILES statement (even implicitly) are also processed more than once.
Example 4
If the system support wishes to save the entire public space area with the exception of the user IDs $TSOS and $SYSDUMP, the FILES statement then becomes
*FILES EXCEPT=($TSOS.,$SYSDUMP.),FROM=PUBLIC
Under $TSOS the missing NAME operand has the effect of implicitly causing all the files in public space to be addressed. In the example the $TSOS and $SYSDUMP. files are excluded by means of the EXCEPT operand. In contrast to this, the following sequence of statements causes all the files in public space to be saved twice, with the exception of the files under the user IDs $TSOS and $SYSDUMP. Both these user IDs would be saved only once.
. . . *FILES EXCEPT=$TSOS.,FROM=PUBLIC *FILES EXCEPT=$SYSDUMP.,FROM=PUBLIC *SAVE ... . . .
Summary
Parallel processing only takes place within a single FILES statement, i.e. between the individual user IDs or NAME operands of the FILES statement in the case of SAVE runs.
A sequence of FILES statements is always processed in strict serial order. In RESTORE and IMPORT runs and when copying save data this means that the input volumes are searched again from the beginning for each FILES statement. Information in the NAME or EXCEPT operand is processed separately for each FILES statement.