Reorganization of version backups

Reorganization is used to save space on the respective backup storage level (S1 or S2). There are obsolete file versions (file versions which should not be kept any longer according to file attribute number of backup versions (NUM-OF-BACKUP-VERS), and there are files that have been deleted for a longer time from the pubsets file catalog and are now ready to be removed from the version backup archive. The reorganization of version backup archives is provided via the statement REORGANIZE-VERSION-BACKUP. The steps on reorganizing a version backup archive are the following:

1. CHECK-CATALOGED-FILES
   The statement CHECK-CATALOGED-FILES has three purposes:
   1. it updates the number of backup versions (NUM-OF-BACKUP-VERS) in the files records of the archive directory from the catalog entries of the TSOSCAT.
   2. CHECK-CATALOGED-FILES checks if the files residing in the archive directory still exist on the corresponding pubset. If a file was deleted from the pubset in the meantime, the file is marked as deleted and the current date is inserted as a deletion-date in the file record in the directory.
   3. The deletion date is removed, in case a file was restored or created again.
   It is recommended that the HSMS administrator create reports during the check runs to be able to inform the users about deleted files. Additionally, the user can check this situation by using HSMS statement SHOW-ARCHIVE. With the SHOW-ARCHIVE statement the user can see which files will be lost during the next reorganization: SHOW-ARCHIVE provides information on which files are deleted from S0, the deletion date and which are additionally marked for deletion from the archive. If the file was deleted unintentionally from the pubset, the user may restore the file. 
   If a CHECK-CATALOGED-FILES finds that a file record has a deletion date but exists again in the TSOSCAT of the corresponding pubset, the deletion date is erased from the file record.
2. MODIFY-ARCHIVE FILE = *MARK-FOR-DELETION(…) 
   In order for a file to be deleted from the archive during reorganization, it must first be marked for deletion. This flag is set with MODIFY-ARCHIVE FILE = *MARK-FOR-DELETION(...) and is only possible after a certain period of time (SECURE-PERIOD) has elapsed since the file was deleted from the pubset. The SECURE-PERIOD is a property of the version backup archive. It is 180 days by default. The files are now “marked for deletion”, but the deletion is still not performed. In case the HSMS administrator wants to remove files from the archive although they still exist on the pubset, he must use the force option to mark the files for deletion:  MODIFY-ARCHIVE FILE = *FORCE-DELETION(...)
3. After that, a reorganization of the archive needs to be performed by REORGANIZE-VERSION-BACKUP statement. Now obsolete file versions and files that are marked for deletion are removed from the archive.

Hint: In case CHECK-CATALOGED-FILES had been done previously and/or some files had been marked for deletion with MODIFY-ARCHIVE FILE=*MARK-FOR-DELETION(…)/*FORCE-DELETION(...) and no reorganization followed, during BACKUP-FILE-VERSIONS or CHECK-CATALOGED-FILES the deletion date and the mark for deletion of specified existing (again) files are reset. Corresponding hints are given within the report.

The user can change the value of the number of backup versions in the file catalog at any time. However, the change will not be transferred to the archive directory until the next version backup or the next call of the CHECK-CATALOGED-FILES statement and only if the file is specified in the FILE-NAMES operand. This means that the updating of information functions or reorganization runs takes effect from this moment on.

Save directory processing during reorganization:

During reorganization of a version backup archive, it is possible to save the archive directory. Due to this process the archive directory, which is updated during the reorganization process, can also be written to the save file. This function is activated by specifying SAVE-DIRECTORY = * YES. If a recovery is necessary, the archive directory can be restored by import. The attributes of an archive are also saved in the archive directory. If the archive definition is lost, the archive can be restored from the archive directory using the CREATE-ARCHIVE statement (RECONSTRUCT-ARCHIVE = * YES operand).

Specifics of saving directory for the reorganization process:

• The archive directory is saved during additional call of ARCHIVE from HSMS after the reorganization processing has finished successfully.

• If the reorganization process itself has finished with errors and no output save file was created, the directory is not backed up. The warning message HSM032C is issued in the report. In the context of error handling a roll-back is started to revert the taken actions as far as possible. During a reorganization run at first save files are deleted that contain only obsolete file versions. These save files are not restored during a roll-back process. Thus the current directory may differ from the directory content before the run.

• A reorganization process sets a lock flag into the directory that prohibits using //RVB and //BFV simultaneously within the same archive. The directory is saved with the flag activated inside, unlike when saving directory during //BFV. In order to remove the flag from the directory after importing the directory for using it, an administrator can run //MODIFY-ARCHIVE statement with specifying DIRECTORY-LOCK=*UNLOCK-VERSION-DIR:

         //MDA ARCHIVE-NAME=<archive_name>,DIRECTORY-LOCK=*UNLOCK-VERSION-DIR


• The version backup directory is not distributed to different volumes. Saving of the version backup directory and the volume used for this are displayed in the report.

Examples:

//BACKUP-FILE-VERSIONS FILE-NAMES=:SSF1:FILE.*,SAVE-OPTIONS=*PARAMETERS(SAVE-DIRECTORY=*YES)
//RVB ARCHIVE-NAME=VERSION.1,SAVE-DIRECTORY=*YES


An unexpected situation occurred and version backup directory was deleted, but now we could import the directory saved in the save file

//IMF FILE-NAMES=*DIRECTORY,SAVE-FILE=*BY-VOLUME(SAVE-FILE-ID= <sfid>,VOLUMES=<volume>,DEVICE-TYPE=*STD)


The attributes of an archive are also stored in the archive directory. If the archive definition has been inadvertently deleted the archive can be restored from the archive directory using the CREATE-ARCHIVE statement:

//CREATE-ARCHIVE ARCHIVE-NAME=<archive_name>,DIRECTORY-NAME=<directory>(NEW-DIRECTORY=*NO(RECONSTRUCT-ARCHIVE=*YES))

If now we run, for example, the BFV command of this archive, it will end with an HSM032A error in report:  

//BACKUP-FILE-VERSIONS FILE-NAMES=<file_names>,SAVE-OPTIONS=*PARAMETERS(SAVE-DIRECTORY=*YES)


HSM032A A VERSION BACKUP IS CURRENTLY NOT POSSIBLE, AS A REORGANIZATION RUN IS EXECUTED FOR THE SAME ARCHIVE. 
     ? Possible reasons:
     - It is not possible to simultaneously perform reorganization
       and version backup within the same archive.
     - If you are using the recovered directory after //RVB with saving directory processing, the lock flag was saved with the directory options.
     RESPONSE : - Wait for the processing completion and try again. - Using //MDA command to unlock directory

If we run //RVB instead of //BFV on this step, we will see a HSM032B error message in the report. This is due to the fact that at previously // RVB with SAVE-DIRECTORY = *YES  lock- flag also was stored in the archive directory.

Remove flag with MODIFY-ARCHIVE statement: 

//MDA ARCHIVE-NAME=<new_archive_name>,DIRECTORY-LOCK=*UNLOCK-VERSION-DIR