WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.

Architecture - HSM

From Obsolete Lustre Wiki
Jump to navigationJump to search

Note: The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.

References

  • Feature Description. Lustre will incorporate Hierarchical Storage Management (HSM) features. Typically, HSM enables a file system to:
  • Migrate less frequently used files to tertiary storage and free space consumed by such files.
  • Restore files into the file system upon access when they reside on tertiary storage.
Lustre HSM is designed to allow interoperability with multiple HSM backend solutions, including HPSS.

Definitions

Primary storage
The Lustre file system nearest to the client
Backend storage
The storage system used for archival of files from the primary system
Migration
Movement of file data and attributes between the primary and backend storage
Copy-in
Migration to the primary storage system
Copy-out
Migration to the backend storage system

Quality Attribute Scenario Summaries

Overall System

Id Quality Summary
cache-presence Feature When a file is accessed a determination can be made if the file is in the primary storage or in the backend storage system
cache-copyin Feature If a file is accessed from the primary storage system, but resides in the backend storage system it can be relocated in the primary storage system and made available
lru-generate Feature The system can generate a list of files eligible for migration, based on recent use, file size and other criteria
copy-out Feature The system can process a list of file eligible for migration to the backend storage and perform file migrations
punch Feature The system can process a list of files moved to the backend storage and purge the data in the primary storage system
access-during-migr Feature When a file is migrating and the file is accessed the access can be delayed transparently, aborted or the migration can be aborted, as the situation requires

Migration API

Id Quality Summary
HSM-migr-in Feature When a file is accessed and is determined to not be in the primary storage the file data is migrated to the primary storage
HSM-migr-out Feature When a file is selected for migration to the secondary storage system