WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.
Architecture Descriptions: Difference between revisions
No edit summary |
No edit summary |
||
| Line 31: | Line 31: | ||
[[Architecture - End-to-end Checksumming|End-to-end Checksumming ]] (Lustre network checksumming) | [[Architecture - End-to-end Checksumming|End-to-end Checksumming ]] (Lustre network checksumming) | ||
[[Architecture - Epochs|Epochs ]] ( | [[Architecture - Epochs|Epochs ]] (Used to merge distributed data and meta-data updates in a redundant cluster configuration) | ||
[[Architecture - External File Locking|External File Locking ]] (File range lock and whole-file lock capabilities) | [[Architecture - External File Locking|External File Locking ]] (File range lock and whole-file lock capabilities) | ||
| Line 59: | Line 59: | ||
[[Architecture - Libcfs|Libcfs ]] (Portable runtime environment for process management and debugging support) | [[Architecture - Libcfs|Libcfs ]] (Portable runtime environment for process management and debugging support) | ||
[[Architecture - Llog over OSD|Llog over OSD ]] | [[Architecture - Llog over OSD|Llog over OSD ]] (Re-implement llog API to use OSD device as backend device) | ||
[[Architecture - LRE Images|LRE Images ]] | [[Architecture - LRE Images|LRE Images ]] (Provide development and training Lustre software environments based on supported environments for Lustre) | ||
[[Architecture - Lustre Logging API|Lustre Logging API ]] | [[Architecture - Lustre Logging API|Lustre Logging API ]] (Requirements and detailed description) | ||
[[Architecture - MDS striping format|MDS striping format ]] | [[Architecture - MDS striping format|MDS striping format ]] (Striping extended attributes, striping formats, striping APIs) | ||
[[Architecture - MDS-on-DMU|MDS-on-DMU ]] | [[Architecture - MDS-on-DMU|MDS-on-DMU ]] (Metadata server on the ZFS Data Management Unit - use cases, features and functional behavior) | ||
[[Architecture - Metadata API|Metadata API ]] | [[Architecture - Metadata API|Metadata API ]] (A set of methods used by the Lustre file system driver to access and manipulate metadata) | ||
[[Architecture - Migration (1)|Migration (1) ]] | [[Architecture - Migration (1)|Migration (1) ]] (Overview of development path for migration capabilities) | ||
[[Architecture - Migration (2)|Migration (2) ]] | [[Architecture - Migration (2)|Migration (2) ]] (Use cases, quality attribute scenarios, and implementation details) | ||
[[Architecture - MPI IO and NetCDF|MPI IO and NetCDF ]] | [[Architecture - MPI IO and NetCDF|MPI IO and NetCDF ]] (Message Passing Interface I/O and network Common Data Form libraries - Lustre ADIO driver improvements and internal optimization) | ||
[[Architecture - MPI LND|MPI LND ]] | [[Architecture - MPI LND|MPI LND ]] (Link to paper ''Lustre Networking over MPI'') | ||
[[Architecture - Multiple Interfaces For LNET|Multiple Interfaces For LNET ]] | [[Architecture - Multiple Interfaces For LNET|Multiple Interfaces For LNET ]] (Use cases and configuration management for Lustre networking) | ||
[[Architecture - Network Request Scheduler|Network Request Scheduler ]] | [[Architecture - Network Request Scheduler|Network Request Scheduler ]] (Requirements for network request scheduler to manage incoming RPC requests on a server) | ||
[[Architecture - New Metadata API|New Metadata API ]] | [[Architecture - New Metadata API|New Metadata API ]] (Proposal and use cases) | ||
[[Architecture - Open by fid|Open by fid ]] | [[Architecture - Open by fid|Open by fid ]] (Returns a file descriptor based on a file ID - implementation choices, design description, use cases) | ||
[[Architecture - OSS-on-DMU|OSS-on-DMU ]] | [[Architecture - OSS-on-DMU|OSS-on-DMU ]] | ||
Revision as of 10:45, 22 January 2010
The architecture descriptions listed below provide information about Lustre architecture and design and are intended to help users better understand the conceptual framework of the Lustre file system.
Note: These documents reflect the state of design of a Lustre feature at a particular point in time. They many contain information that is incomplete or obsolete and may not reflect the current architecture, features and functionality of Lustre.
Backup (File system backup)
Caching OSS (Caching on object storage servers)
Changelogs (Per-server logs of data or metadata changes)
Changelogs 1.6 (Used to facilitate efficient replication of large Lustre 1.6 filesystems)
Client Cleanup (Use cases, business drivers, models to consider, implementation contraints)
Clustered Metadata (Clustered metadata server capability)
Commit on Share (Better recover-ability in an environment where clients miss reconnect window)
CROW (CReate On Write optimizes create performance by deferring OSS object creation)
CTDB with Lustre (Cluster implementation of the TDB database with Lustre provides a solution for windows pCIFS)
Cuts (Technique for recovering file system metadata stored on file server clusters)
DMU OSD (An implementation of the Object Storage Device API for a Data Management Unit)
End-to-end Checksumming (Lustre network checksumming)
Epochs (Used to merge distributed data and meta-data updates in a redundant cluster configuration)
External File Locking (File range lock and whole-file lock capabilities)
FIDs on OST (File identifiers used to identify objects on an object storage target)
Fileset (An efficient representation of a group of file identifiers (FIDs))
Flash Cache (Very fast read-only flash storage)
Free Space Management (Managing free space for stripe allocation)
GNS (Global namespace for a distributed file system)
HSM (Hierarchical storage management)
HSM and Cache (Reuse of components by Lustre features that involve migration of file system objects)
HSM Migration (Use cases and high-level architecture for migrating files between Lustre and a HSM system)
Interoperability fids zfs (Client, server, network, storage interoperability during migration to clusters based on file identifiers and the ZFS file system)
Interoperability 1.6 1.8 2.0 (interoperability definitions and QAS summary)
IO system (Client IO and server I/O request handling)
Libcfs (Portable runtime environment for process management and debugging support)
Llog over OSD (Re-implement llog API to use OSD device as backend device)
LRE Images (Provide development and training Lustre software environments based on supported environments for Lustre)
Lustre Logging API (Requirements and detailed description)
MDS striping format (Striping extended attributes, striping formats, striping APIs)
MDS-on-DMU (Metadata server on the ZFS Data Management Unit - use cases, features and functional behavior)
Metadata API (A set of methods used by the Lustre file system driver to access and manipulate metadata)
Migration (1) (Overview of development path for migration capabilities)
Migration (2) (Use cases, quality attribute scenarios, and implementation details)
MPI IO and NetCDF (Message Passing Interface I/O and network Common Data Form libraries - Lustre ADIO driver improvements and internal optimization)
MPI LND (Link to paper Lustre Networking over MPI)
Multiple Interfaces For LNET (Use cases and configuration management for Lustre networking)
Network Request Scheduler (Requirements for network request scheduler to manage incoming RPC requests on a server)
New Metadata API (Proposal and use cases)
Open by fid (Returns a file descriptor based on a file ID - implementation choices, design description, use cases)
Punch and Extent Migration Requirements
