Because of the continual growth of disk storage capacity and of data that needs to be available online, the disk and file sizes have been increased from the approx. 32 GB previously supported in BS2000. The limits are as follows:
The maximum capacity of a pubset or volume set is about 4 TB.
The maximum capacity of a single disk is about 2 TB.
The maximum file size is about 4 TB (which is the maximum size of a pubset or volume set minus SVL, F5 label and system files).
This means that within the operating system, 4-byte block numbers and 4-byte counters must be used systematically for file sizes and disk sizes.
Converting 3-byte fields to 4-byte fields affects all components, products and applications that work directly or indirectly with these fields. The TPR interfaces have been modified accordingly, but not all of the TU user interfaces can compatibly support these large files.
Preconditions
There are two pubset types for large objects (large files and volumes):
LARGE-OBJECTS pubsets without large files:
This pubset type allows large volumes, but limits the file size allowed to 32 GB. This means that the pubset may contain volumes >= 32 GB, but does not necessarily currently contain such volumes.
From the point of view of the user, these pubsets behave (almost) as conventional pubsets.
LARGE-OBJECTS pubsets with large files
This pubset type allows large volumes and large files, but they do not necessarily exist.
It allows large files to be separated from programs that use interfaces that are incompatible and supports the step-by-step introduction of large volumes and – at a second step – of large files.
Large objects are supported by both SF and SM pubsets. To achieve this, the following two attributes have been introduced:
LARGE_OBJECTS | Attribute for supporting volumes >= 32 GB |
LARGE_FILES_ALLOWED | Attribut steuert, ob auf dem Pubset auch große Dateien (Dateien >= 32 GB) unterstützt werden |
As for pubsets, an attribute has been introduced for volumes: LARGE_VOLUME. The LARGE_VOLUME attribute characterizes a logical volume and indicates that the volume has a gross capacity >= 32 GB. Just as the LARGE_OBJECTS characteristic is for pubsets, the LARGE_VOLUME characteristic is permanent and is stored in the SVL.
3-byte and 4-byte fields
The introduction of 4-byte fields for the following data stored in the catalog entry is a key aspect in the lifting of the 32-GB limit for volume and data sizes:
FILE-SIZE – the storage space allocated for the file
HIGHEST-USED-PAGE – the amount of storage that currently contains data
LHP (Logical halfpage number) and PHP (physical halfpage number) of the individual extents in the extent list, that allocate “physical” halfpages of volumes to the logical halfpages
Block numbers and block counters are visible at various BS2000 user interfaces. Although 4-byte fields have been used exclusively in all new versions of these interfaces, some old interface versions may still use 3-byte fields. If files >= 32 GB exist, you may experience compatibility problems, as is also the case when "only" volumes >= 32 GB exist.
The introduction of 4-byte LHPs and 4-byte PHPs means that a new (additional) format also exists for the extent list. In order to achieve as much downward compatibility as possible, the current BS2000 versions support both formats of the extent list:
In general, the “old” format with 3-byte block numbers will be used.
Only in the case of large files or of files that can only be addressed using PHPs larger than X'FFFFFF' will the new format with 4-byte block numbers be used.
Extent lists therefore contain either extents with 3-byte block numbers or extents with 4-byte block numbers.
Restrictions for large files
The following restrictions for large files must be taken into account when working with nonprivileged applications:
Support for files >= 32 GB is only possible on pubsets with LARGE_OBJECTS=TRUE, LARGE_FILES_ALLOWED=TRUE.
Support for files >= 32 GB is only possible for non-PAMKEY files: Files with BLKCTRL=PAMKEY are not supported since the logical page number is stored as a 3-byte field in the system section of the PAMKEY.
EAM files >= 32 GB are not possible, because the SYSEAM container file cannot be a large file.
Summary of the DMS pubset management interfaces affected by 32 GB objects
Interface | Change |
|---|---|
Nonprivileged commands | |
ADD-FILE-LINK | New operand EXCEED-32GB for specifying the permitted file size (larger or smaller than 32 GB) |
SHOW-FILE-LINK | Outputs the “large file” file attribute |
SHOW-FILE-ATTRIBUTES | Extended output for various output fields |
Macros | |
FCB | New operand LARGE_FILE for large files |
FILE | New operand EXC32GB for large files |
FSTAT | Effort involved in check and conversion if VERSION=0/1 |
OPEN | Take account of semantic problems |
RDTFT | Outputs the “large file” file attribute |
DIV | New operand LARGE_FILE for large files; Extended range of values for the BLOCK and SPAN operands |
FPAMACC | Extended range of values for BLOCK operand |
FPAMSRV | New operand LARGE_FILE for large files |
Executability of programs in configurations with files larger than 32 GB
It cannot be assumed that all programs have been prepared for accessing large objects, i.e. that they can address 4-byte block numbers and block counters, and it must be borne in mind that the interfaces available to user applications only relate to accessing and processing files and their metadata. The following description is therefore confined to large files. Program behavior can thus be classified as follows:
Class A: | A program is able to process large files without restrictions. This behavior is defined as LARGE_FILES-capable. |
Class B: | A program has not been prepared for processing large files and/or their metadata. It is, however, able to perform defined rejections of corresponding access attempts that it regards as illegal or, alternatively, there is no access to files or their metadata within the program. This behavior is defined as LARGE_FILES-compatible. |
Class C: | A program has not been prepared for processing large files and cannot perform a defined rejection of corresponding access attempts. This behavior is defined as LARGE_FILES-incompatible. |
In configurations that contain large files, programs that are compatible with or capable of LARGE_FILES are required. LARGE_FILES compatibility is to be regarded as the norm. Growth over 32 GB will initially be limited to a relatively small number of files; Only the programs that access these require LARGE_FILES capability.
For a presentation of all the considerations associated with the use of large volumes and files, refer to the “Files and Volumes larger than 32 GB” manual [18 (Related publications)].