Content verification service

When an object is created, HCP uses cryptographic hash algorithms to calculate various hash values for it. These values, which are generated based on the object data, system metadata, and custom metadata are stored with the primary metadata for the object.

One of the hash values that’s generated only from the object data is also stored with the secondary metadata for the object. The cryptographic hash algorithm HCP uses to calculate this hash value is namespace dependent. It is set when the namespace is created. Once set, it cannot be changed.

Users and applications can see, but not modify, hash values generated from object data and annotations. They cannot see any other hash values. For information on viewing hash values for objects, see Using a Namespace, Using the HCP HS3 API, or Using the Default Namespace.

The content verification service ensures the integrity of each object by:

Checking that the object data, system metadata, and custom metadata still match the stored cryptographic hash values

Note: The content verification service does not do a data check for objects:

That are stored through namespaces that have economy storage devices as their ingest tier. For information on ingest tiers, see Choose the ingest tier.

That are stored on extended storage. For information on extended storage, see Storage for HCP systems.

Ensuring that certain secondary metadata other than the hash value matches the primary metadata for the object

The content verification service runs according to the active service schedule. For information on service schedules, see Scheduling services.

During HCP content verification, HCP attempts to repair any files that HCP S Series Nodes report as being irreparable.

Cryptographic hash algorithms

HCP supports these cryptographic hash algorithms for selection at the namespace level:

MD5
SHA-1
SHA-256
SHA-384
SHA-512
RIPEMD-160

Note: The more complex the hash algorithm, the greater the impact on performance when objects are stored or when services run.

ETags and the content verification service

When an object is stored, HCP generates an ETag for it. An ETag is an identifier for the content of an object.

ETags were introduced in release 6.0 of HCP, so objects stored while the system was at an earlier release do not initially have ETags. When the content verification service runs, it generates ETags for objects that do not have them.

In response to an HS3 request to retrieve an object that does not yet have an ETag, HCP generates the ETag before returning the object. This can be time consuming for large objects, with the result that read performance is slow for those objects.

If tenant administrators will be enabling the HS3 API on namespaces that were created while the HCP system was at a release earlier than 6.0, consider scheduling more run time for the content verification service and/or increasing the performance level at which the service runs.

Trademarks and Legal Disclaimer

© 2017 Hitachi Data Systems Corporation. All rights reserved.