WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.
Architecture Descriptions: Difference between revisions
No edit summary |
No edit summary |
||
| Line 87: | Line 87: | ||
[[Architecture - Open by fid|Open by fid ]] (Returns a file descriptor based on a file ID - implementation choices, design description, use cases) | [[Architecture - Open by fid|Open by fid ]] (Returns a file descriptor based on a file ID - implementation choices, design description, use cases) | ||
[[Architecture - OSS-on-DMU|OSS-on-DMU ]] | [[Architecture - OSS-on-DMU|OSS-on-DMU ]](Object storage server on the ZFS Data Management Unit - requirements, design constraints, and components) | ||
[[Architecture - PAG|PAG ]] | [[Architecture - PAG|PAG ]] (Process Authentication Groups - Use of Linux keyring, setuid in Lustre, Kerberos credential, use cases) | ||
[[Architecture - Pools of targets|Pools of targets ]] | [[Architecture - Pools of targets|Pools of targets ]] (Use cases, command line definitions of pools of OSTs, implementation constraints) | ||
[[Architecture - Profiling Tools for IO|Profiling Tools for IO ]] | [[Architecture - Profiling Tools for IO|Profiling Tools for IO ]] (Profiling system based on Ganglia - use cases and implementation constraints) | ||
[[Architecture - Proxy Cache|Proxy Cache ]] | [[Architecture - Proxy Cache|Proxy Cache ]] (Caching and aggregation used to reduce load on backend server and provide better throughput and latency to clients - requirements and use cases) | ||
[[Architecture - Punch and Extent Migration|Punch and Extent Migration ]] | [[Architecture - Punch and Extent Migration|Punch and Extent Migration ]] (Prototypes for ''punch'' and ''migrate'' functionality) | ||
[[Architecture - Punch and Extent Migration Requirements|Punch and Extent Migration Requirements]] | [[Architecture - Punch and Extent Migration Requirements|Punch and Extent Migration Requirements]] (Punch functionality use cases) | ||
[[Architecture - Recovery Failures|Recovery Failures ]] | [[Architecture - Recovery Failures|Recovery Failures ]] (Recovery terminology, architectures, and use cases) | ||
[[Architecture - Request Redirection|Request Redirection ]] | [[Architecture - Request Redirection|Request Redirection ]] (Allows target OST to redirect client requests to other servers - requirements, use cases, questions) | ||
[[Architecture - Scalable Pinger|Scalable Pinger ]] | [[Architecture - Scalable Pinger|Scalable Pinger ]] (Provides peer health information to Lustre clients and servers - requirements) | ||
[[Architecture - Security|Security ]] | [[Architecture - Security|Security ]] (Detailed description of the security architecture for Lustre) | ||
[[Architecture - Server Network Striping|Server Network Striping ]] | [[Architecture - Server Network Striping|Server Network Striping ]] (Description of Lustre-level striping of file data over multiple object servers with redundancy) | ||
[[Architecture - Simple Space Balance Migration|Simple Space Balance Migration ]] | [[Architecture - Simple Space Balance Migration|Simple Space Balance Migration ]] | ||
[[Architecture - Simplified Interoperation|Simplified Interoperation ]] | [[Architecture - Simplified Interoperation|Simplified Interoperation ]] | ||
Revision as of 11:07, 22 January 2010
The architecture descriptions listed below provide information about Lustre architecture and design and are intended to help users better understand the conceptual framework of the Lustre file system.
Note: These documents reflect the state of design of a Lustre feature at a particular point in time. They many contain information that is incomplete or obsolete and may not reflect the current architecture, features and functionality of Lustre.
Backup (File system backup)
Caching OSS (Caching on object storage servers)
Changelogs (Per-server logs of data or metadata changes)
Changelogs 1.6 (Used to facilitate efficient replication of large Lustre 1.6 filesystems)
Client Cleanup (Use cases, business drivers, models to consider, implementation contraints)
Clustered Metadata (Clustered metadata server capability)
Commit on Share (Better recover-ability in an environment where clients miss reconnect window)
CROW (CReate On Write optimizes create performance by deferring OSS object creation)
CTDB with Lustre (Cluster implementation of the TDB database with Lustre provides a solution for windows pCIFS)
Cuts (Technique for recovering file system metadata stored on file server clusters)
DMU OSD (An implementation of the Object Storage Device API for a Data Management Unit)
End-to-end Checksumming (Lustre network checksumming)
Epochs (Used to merge distributed data and meta-data updates in a redundant cluster configuration)
External File Locking (File range lock and whole-file lock capabilities)
FIDs on OST (File identifiers used to identify objects on an object storage target)
Fileset (An efficient representation of a group of file identifiers (FIDs))
Flash Cache (Very fast read-only flash storage)
Free Space Management (Managing free space for stripe allocation)
GNS (Global namespace for a distributed file system)
HSM (Hierarchical storage management)
HSM and Cache (Reuse of components by Lustre features that involve migration of file system objects)
HSM Migration (Use cases and high-level architecture for migrating files between Lustre and a HSM system)
Interoperability fids zfs (Client, server, network, storage interoperability during migration to clusters based on file identifiers and the ZFS file system)
Interoperability 1.6 1.8 2.0 (interoperability definitions and QAS summary)
IO system (Client IO and server I/O request handling)
Libcfs (Portable runtime environment for process management and debugging support)
Llog over OSD (Re-implement llog API to use OSD device as backend device)
LRE Images (Provide development and training Lustre software environments based on supported environments for Lustre)
Lustre Logging API (Requirements and detailed description)
MDS striping format (Striping extended attributes, striping formats, striping APIs)
MDS-on-DMU (Metadata server on the ZFS Data Management Unit - use cases, features and functional behavior)
Metadata API (A set of methods used by the Lustre file system driver to access and manipulate metadata)
Migration (1) (Overview of development path for migration capabilities)
Migration (2) (Use cases, quality attribute scenarios, and implementation details)
MPI IO and NetCDF (Message Passing Interface I/O and network Common Data Form libraries - Lustre ADIO driver improvements and internal optimization)
MPI LND (Link to paper Lustre Networking over MPI)
Multiple Interfaces For LNET (Use cases and configuration management for Lustre networking)
Network Request Scheduler (Requirements for network request scheduler to manage incoming RPC requests on a server)
New Metadata API (Proposal and use cases)
Open by fid (Returns a file descriptor based on a file ID - implementation choices, design description, use cases)
OSS-on-DMU (Object storage server on the ZFS Data Management Unit - requirements, design constraints, and components)
PAG (Process Authentication Groups - Use of Linux keyring, setuid in Lustre, Kerberos credential, use cases)
Pools of targets (Use cases, command line definitions of pools of OSTs, implementation constraints)
Profiling Tools for IO (Profiling system based on Ganglia - use cases and implementation constraints)
Proxy Cache (Caching and aggregation used to reduce load on backend server and provide better throughput and latency to clients - requirements and use cases)
Punch and Extent Migration (Prototypes for punch and migrate functionality)
Punch and Extent Migration Requirements (Punch functionality use cases)
Recovery Failures (Recovery terminology, architectures, and use cases)
Request Redirection (Allows target OST to redirect client requests to other servers - requirements, use cases, questions)
Scalable Pinger (Provides peer health information to Lustre clients and servers - requirements)
Security (Detailed description of the security architecture for Lustre)
Server Network Striping (Description of Lustre-level striping of file data over multiple object servers with redundancy)
