WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.

Difference between revisions of "Architecture Descriptions"

From Obsolete Lustre Wiki
Jump to navigationJump to search
Line 5: Line 5:
 
[[Architecture - Adaptive Timeouts - Use Cases|Adaptive Timeouts - Use Cases ]]
 
[[Architecture - Adaptive Timeouts - Use Cases|Adaptive Timeouts - Use Cases ]]
  
[[Architecture - Backup|Backup ]] (file system backup)
+
[[Architecture - Backup|Backup ]] (File system backup)
  
[[Architecture - Caching OSS|Caching OSS ]] (caching on object storage servers)
+
[[Architecture - Caching OSS|Caching OSS ]] (Caching on object storage servers)
  
[[Architecture - Changelogs|Changelogs ]] (per-server logs of data or metadata changes)
+
[[Architecture - Changelogs|Changelogs ]] (Per-server logs of data or metadata changes)
  
[[Architecture - Changelogs 1.6|Changelogs 1.6 ]] (used to facilitate efficient replication of large Lustre 1.6 filesystems)
+
[[Architecture - Changelogs 1.6|Changelogs 1.6 ]] (Used to facilitate efficient replication of large Lustre 1.6 filesystems)
  
[[Architecture - Client Cleanup|Client Cleanup ]] (use cases, business drivers, models to consider, implementation contraints)
+
[[Architecture - Client Cleanup|Client Cleanup ]] (Use cases, business drivers, models to consider, implementation contraints)
  
[[Architecture - Clustered Metadata|Clustered Metadata ]] (clustered metadata server capability)
+
[[Architecture - Clustered Metadata|Clustered Metadata ]] (Clustered metadata server capability)
  
[[Architecture - Commit on Share|Commit on Share ]] (better recover-ability in an environment where clients miss reconnect window)
+
[[Architecture - Commit on Share|Commit on Share ]] (Better recover-ability in an environment where clients miss reconnect window)
  
 
[[Architecture - CROW|CROW ]]  (CReate On Write optimizes create performance by deferring OSS object creation)  
 
[[Architecture - CROW|CROW ]]  (CReate On Write optimizes create performance by deferring OSS object creation)  
  
[[Architecture - CTDB with Lustre|CTDB with Lustre ]] (cluster implementation of the TDB database with Lustre provides a solution for windows pCIFS)
+
[[Architecture - CTDB with Lustre|CTDB with Lustre ]] (Cluster implementation of the TDB database with Lustre provides a solution for windows pCIFS)
  
[[Architecture - Cuts|Cuts ]] (technique for recovering file system metadata stored on file server clusters)
+
[[Architecture - Cuts|Cuts ]] (Technique for recovering file system metadata stored on file server clusters)
  
[[Architecture - DMU OSD|DMU OSD ]] (an implementation of the Object Storage Device API for a Data Management Unit)
+
[[Architecture - DMU OSD|DMU OSD ]] (An implementation of the Object Storage Device API for a Data Management Unit)
  
 
[[Architecture - DMU Zerocopy|DMU Zerocopy ]]
 
[[Architecture - DMU Zerocopy|DMU Zerocopy ]]
Line 33: Line 33:
 
[[Architecture - Epochs|Epochs ]] (used to merge distributed data and meta-data updates in a redundant cluster configuration)
 
[[Architecture - Epochs|Epochs ]] (used to merge distributed data and meta-data updates in a redundant cluster configuration)
  
[[Architecture - External File Locking|External File Locking ]] (file range lock and whole-file lock capabilities)
+
[[Architecture - External File Locking|External File Locking ]] (File range lock and whole-file lock capabilities)
  
[[Architecture - FIDs on OST|FIDs on OST ]] (file identifiers used to identify objects on an object storage target)
+
[[Architecture - FIDs on OST|FIDs on OST ]] (File identifiers used to identify objects on an object storage target)
  
[[Architecture - Fileset|Fileset ]] (an efficient representation of a group of file identifiers (FIDs))
+
[[Architecture - Fileset|Fileset ]] (An efficient representation of a group of file identifiers (FIDs))
  
[[Architecture - Flash Cache|Flash Cache ]] (very fast read-only flash storage)
+
[[Architecture - Flash Cache|Flash Cache ]] (Very fast read-only flash storage)
  
[[Architecture - Free Space Management|Free Space Management ]] (managing free space for stripe allocation)
+
[[Architecture - Free Space Management|Free Space Management ]] (Managing free space for stripe allocation)
  
[[Architecture - GNS|GNS ]] (global namespace for a distributed file system)
+
[[Architecture - GNS|GNS ]] (Global namespace for a distributed file system)
  
[[Architecture - HSM|HSM ]] (hierarchical storage management)
+
[[Architecture - HSM|HSM ]] (Hierarchical storage management)
  
[[Architecture - HSM and Cache|HSM and Cache ]] (reuse of components by Lustre features that involve migration of file system objects)
+
[[Architecture - HSM and Cache|HSM and Cache ]] (Reuse of components by Lustre features that involve migration of file system objects)
  
[[Architecture - HSM Migration|HSM Migration ]] (use cases and high-level architecture for migrating files between Lustre and a HSM system)
+
[[Architecture - HSM Migration|HSM Migration ]] (Use cases and high-level architecture for migrating files between Lustre and a HSM system)
  
[[Architecture - Interoperability fids zfs|Interoperability fids zfs ]] (client, server, network, storage interoperability during migration to clusters based on file identifiers and the ZFS file system)
+
[[Architecture - Interoperability fids zfs|Interoperability fids zfs ]] (Client, server, network, storage interoperability during migration to clusters based on file identifiers and the ZFS file system)
  
 
[[Architecture - Interoperability 1.6 1.8 2.0|Interoperability 1.6 1.8 2.0 ]] (interoperability definitions and QAS summary)
 
[[Architecture - Interoperability 1.6 1.8 2.0|Interoperability 1.6 1.8 2.0 ]] (interoperability definitions and QAS summary)
  
[[Architecture - IO system|IO system ]]
+
[[Architecture - IO system|IO system ]] (Client IO and server I/O request handling)
  
[[Architecture - Libcfs|Libcfs ]]
+
[[Architecture - Libcfs|Libcfs ]] (Portable runtime environment for process management and debugging support)
  
 
[[Architecture - Llog over OSD|Llog over OSD ]]
 
[[Architecture - Llog over OSD|Llog over OSD ]]

Revision as of 10:47, 22 January 2010

The architecture descriptions listed below provide information about Lustre architecture and design and are intended to help users better understand the conceptual framework of the Lustre file system.

Note: These documents reflect the state of design of a Lustre feature at a particular point in time. They many contain information that is incomplete or obsolete and may not reflect the current architecture, features and functionality of Lustre.

Adaptive Timeouts - Use Cases

Backup (File system backup)

Caching OSS (Caching on object storage servers)

Changelogs (Per-server logs of data or metadata changes)

Changelogs 1.6 (Used to facilitate efficient replication of large Lustre 1.6 filesystems)

Client Cleanup (Use cases, business drivers, models to consider, implementation contraints)

Clustered Metadata (Clustered metadata server capability)

Commit on Share (Better recover-ability in an environment where clients miss reconnect window)

CROW (CReate On Write optimizes create performance by deferring OSS object creation)

CTDB with Lustre (Cluster implementation of the TDB database with Lustre provides a solution for windows pCIFS)

Cuts (Technique for recovering file system metadata stored on file server clusters)

DMU OSD (An implementation of the Object Storage Device API for a Data Management Unit)

DMU Zerocopy

End-to-end Checksumming (Lustre network checksumming)

Epochs (used to merge distributed data and meta-data updates in a redundant cluster configuration)

External File Locking (File range lock and whole-file lock capabilities)

FIDs on OST (File identifiers used to identify objects on an object storage target)

Fileset (An efficient representation of a group of file identifiers (FIDs))

Flash Cache (Very fast read-only flash storage)

Free Space Management (Managing free space for stripe allocation)

GNS (Global namespace for a distributed file system)

HSM (Hierarchical storage management)

HSM and Cache (Reuse of components by Lustre features that involve migration of file system objects)

HSM Migration (Use cases and high-level architecture for migrating files between Lustre and a HSM system)

Interoperability fids zfs (Client, server, network, storage interoperability during migration to clusters based on file identifiers and the ZFS file system)

Interoperability 1.6 1.8 2.0 (interoperability definitions and QAS summary)

IO system (Client IO and server I/O request handling)

Libcfs (Portable runtime environment for process management and debugging support)

Llog over OSD

LRE Images

Lustre Logging API

MDS striping format

MDS-on-DMU

Metadata API

Migration (1)

Migration (2)

MPI IO and NetCDF

MPI LND

Multiple Interfaces For LNET

Network Request Scheduler

New Metadata API

Open by fid

OSS-on-DMU

PAG

Pools of targets

Profiling Tools for IO

Proxy Cache

Punch and Extent Migration

Punch and Extent Migration Requirements

Recovery Failures

Request Redirection

Scalable Pinger

Security

Server Network Striping

Simple Space Balance Migration

Simplified Interoperation

Space Manager

Sub Tree Locks

User Level Access

User Level OSS

Userspace Servers

Version Based Recovery

Wide Striping

Wire Level Protocol

Write Back Cache

ZFS for Lustre

ZFS large dnodes

ZFS TinyZAP