The section below is based on measurements and data from current Fujitsu ETERNUS DX systems. Other systems can display different behavior with respect to their cache management and algorithms. However, most statements are of a fundamental nature and can also be applied to other systems.
An area in the cache of an ETERNUS DX system is reserved for write jobs. The remaining cache is available for caching read requests. The size of these two areas is managed dynamically and can be between 0% and 100% of the total cache, depending on the situation.
While write IOs can always be handled as cache hits (except in the case of an overload), it is more critical to achieve a high bit rate when reading. The behavior and efficiency of the cache and disk utilization resulting from this for various loads are described in terms of quality.
Sequential Read
When an ETERNUS DX system recognizes a Sequential Read, the read-ahead mechanism comes into effect. The following data which has not yet been requested by the client is read and stored in the cache. Provided no parallel loads cause a significant disturbance, this achieves a 100-percent hit rate. This also applies when a number of read operations are executed in parallel. It is consequently generally no problem to reduce the time slot for backup runs by parallelizing the operations (provided the RAID groups can supply the total throughput and a heavy productive load is not already imposed on the cache).
When a suitable RAID configuration is provided, the highest throughputs can be achieved with this load. In terms of cache (and also disk) utilization, Sequential Read is the optimal application.
Sequential Write
In the case of Sequential Write the data is practically written sequentially immediately from the cache to the disk drives. Here, too, a number of operations can run in parallel without any problem - without having a negative influence on cache utilization. Provided the performance of the RAID groups concerned is sufficient, the same throughput is achieved as with a Sequential Read. However, depending on the RAID level, the load on the disk drives is greater than when reading, which means that bottlenecks can occur earlier in the event of heavy loads. Details of this are provided in section RAID levels and their performance.
If the disk drives can no longer cope with the load, the write area of the cache will become full, and Delayed Fast Writes will occur (see section Components of the storage systems). This can be recognized from a write hit rate of below 100%. As this will probably occur with Random Write, the effects are described in the associated section.
Random Read
In the case of Random Read with large volumes of data, the performance is influenced negatively by cache misses. Here the read hit rate is determined by the characteristics of the load, size of the cache, and utilization of the cache by competing loads.
In ETERNUS DX systems, read and write hit rates can be determined separately at RAID group level. If only the total hit rate is available, take into account the relationship between read and write IOs when you evaluate it as normally cache misses are restricted to read IOs.
Parallel random loads compete more strongly for the cache (in the case of parallel loads in the same RAID group also for disk accesses) than sequential loads and potentially have a greater influence on each other when the throughput is the same. When the disk drives are approaching high utilization, the negative effects on the IO times are considerably higher than with sequential access. Observing the disk utilization enables the threat of bottlenecks to be recognized before they become apparent in appreciable increases in the IO times.
Random Write
In the case of Random Write (in contract to Sequential Write), the data is not written immediately to the disk drives, but initially only stored in the cache. When the load is low, the data is retained as long as possible in the cache to prevent duplicated writing if the data is modified again. Above all in the case of small IOs it makes sense to wait until a number of consecutive blocks have been modified in order to write them jointly to the disks and thus relieve the load on the disk drives. However, too much cache may not be reserved for the read IOs. The actual disk utilization is therefore largely dependent on the cache size and cache utilization. Management of the write and read percentages in the cache and the decision regarding whether and when data is written from the cache to the disk drives is the responsibility of the storage system's algorithms and as a rule cannot be influenced.
When data is finally written from the cache to the disks, depending on the circumstances disk utilization of 100% can occur. This is uncritical when:
the performance of the RAID group is sufficient to write the data from the cache quicker than new requests are received by the cache. The disks are heavily utilized in the background in order to process the accumulated write operations promptly. Provided this status only exists for a short time, this is not a certain indication of a bottleneck.
parallel read requests, also to other volumes of the RAID group, are not disturbed. This is ensured as far as possible by the storage system. In the case of high disk utilization it is recommendable to consider the read IO times to determine whether these actually impair the performance.
Only when the incoming write requests exceed the capability of the RAID groups concerned is continuously more write cache required. If a cache bottleneck then occurs, the performance is impaired in various respects:
Delayed Fast Writes occur, t.e. cache contents must first be saved before it is possible to store new data in the cache. This results in increased write IO times. The write hit rate decreases. A write hit rate below 100% is always a sign for an acute overload.
A high requirement for write cache automatically leads to a smaller available read cache. The read hit rate decreases, and increased IO times also occur there - also in RAID groups which were initially not affected.
In such cases, writing the data onto the relevant volumes has highest priority in order to relieve the cache situation. Competing read IOs (also to different volumes of the same RAID group) are significantly impaired by this and, in extreme cases, can require times which are increased by a factor of 10-100.
In extreme cases the effects can also continue for a few minutes beyond the high load situation until the write jobs accumulated in the cache have been processed.
You are certainly recommended to avoid cache bottlenecks and to provide sufficient write performance for any volume on which a heavy load is imposed by Random Writes. Relevant details and recommendations are provided in section "RAID levels and their performance".