WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.
Architecture - HSM
From Obsolete Lustre Wiki
Jump to navigationJump to search
Note: The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.
References
- Feature Description. Lustre will incorporate Hierarchical Storage Management (HSM) features. Typically, HSM enables a file system to:
- Migrate less frequently used files to tertiary storage and free space consumed by such files.
- Restore files into the file system upon access when they reside on tertiary storage.
- Lustre HSM is designed to allow interoperability with multiple HSM backend solutions, including HPSS.
- Architecture. See Architecture - HSM and Cache
Definitions
- Primary storage
- The Lustre file system nearest to the client
- Backend storage
- The storage system used for archival of files from the primary system
- Migration
- Movement of file data and attributes between the primary and backend storage
- Copy-in
- Migration to the primary storage system
- Copy-out
- Migration to the backend storage system
Quality Attribute Scenario Summaries
Overall System
| Id | Quality | Summary |
|---|---|---|
| cache-presence | Feature | When a file is accessed a determination can be made if the file is in the primary storage or in the backend storage system |
| cache-copyin | Feature | If a file is accessed from the primary storage system, but resides in the backend storage system it can be relocated in the primary storage system and made available |
| lru-generate | Feature | The system can generate a list of files eligible for migration, based on recent use, file size and other criteria |
| copy-out | Feature | The system can process a list of file eligible for migration to the backend storage and perform file migrations |
| punch | Feature | The system can process a list of files moved to the backend storage and purge the data in the primary storage system |
| access-during-migr | Feature | When a file is migrating and the file is accessed the access can be delayed transparently, aborted or the migration can be aborted, as the situation requires |
Migration API
| Id | Quality | Summary |
|---|---|---|
| HSM-migr-in | Feature | When a file is accessed and is determined to not be in the primary storage the file data is migrated to the primary storage |
| HSM-migr-out | Feature | When a file is selected for migration to the secondary storage system |
