WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.

Architecture - Lustre DLDs

From Obsolete Lustre Wiki
Jump to navigationJump to search

You can access archived Lustre detail-level design (DLD) documents in PDF format from this page. You can also download the DLDs in LyX format from the Lustre CVS repository.

Note: The documents accessed from this page reflect the state of design of a Lustre feature at a particular point in time and may contain outdated information.

Checking out DLDs from CVS

Note: The HLD documents have not been migrated to Git. They can still be accessed in the CVS repository.

Assuming you have CVS installed, check out the Lustre DLDs as follows:

1. Set CVSROOT in your environment to :pserver:anonymous@cvs.lustre.org:/lustre.
   (Alternately, you can add the option -d :pserver:anonymous@cvs.lustre.org:/lustre when you run cvs login.)

2. Run cvs login. No password is needed. Press Enter to continue.

3. Check out the DLD module (doc/DLD)

Example:

% cvs -d :pserver:anonymous@cvs.lustre.org:/lustre co -P doc/DLD
cvs checkout: Updating doc/DLD
U doc/DLD/CRYPTO_API.lyx
U doc/DLD/CrayUnicosMPPort-dld.lyx
U doc/DLD/ELC-CMD-DLD.lyx
U doc/DLD/FilterFS_DLD.lyx
U doc/DLD/LAID5_DLD.lyx
...

DLDs Available as PDFs

For each DLD, a link to the PDF is provided, along with the title, author, date (original and update, if applicable), and synopsis.

2008

Quotas Support on HEAD
Date August 2008
Author Fan Yong
Synopsis This document describes quota support for HEAD. Quotas are currently supported in the Lustre 1.6.x release. Quotas functionality in Lustre 2.0 will be based on HEAD, which is originally from the CMD project without quota support. For compatibility between 1.6.x and 2.0, quotas support will be added to HEAD, consisting of two parts:
  • Port quota-related code from b1_6 to HEAD with the new MD stack on the MDS.
  • Add security support for quotas on HEAD, including MDS/OSS capability, remote user, etc.


Sptlrpc Interoperability
Date 22 July 2008
Author Eric Mei
Synopsis Interoperability FIDs ZFS]].


Client I/O Layering Cleanup
Date 20 April 2008 (updated) / 25 January 2008 (original)
Author Nikita Danilov
Synopsis This document describes re-organization of the Lustre client I/O path (i.e., data access: read, write, truncate, etc., as opposed to the metadata path), targeting more regular and extensible structure of the client stack.


Version-based Recovery
Date 15 April 2008 (updated) / 6 December 2007 (original)
Author Mike Pershin
Synopsis This document outlines the design for version-based recovery.


OST Pools
Date 27 February 2008
Author Jacques-Charles Lafoucrière
Synopsis This document outlines the implementation of OST pools.


uMDS Transaction API
Date 5 February 2008 (updated) / 24 January 2008 (original)
Author Amit Sharma
Synopsis This document outlines the detailed design for the uMDS transaction API task, which aims to replace the old transaction API with the new transaction API.


Interoperable Client Recovery
Date 5 February 2008 (updated) / 28 December 2007 (original)
Author Amit Sharma
Synopsis This document outlines the detailed design for the interoperable client recovery work.

2007

Size-on-MDS Recovery
Date 4 January 2008 (updated) / 13 July 2007 (original)
Author Vitaly Fertman
Synopsis This document describes the recovery of size-on-MDS attributes on the MDS after a node failure in a cluster. It allows the MDS to re-enable size-on-MDS caching for inodes with the proper attributes.


Lustre Log over OSD
Date 21 December 2007 (updated) / 10 December 2007 (original)
Author Yury Umanets
Synopsis This document describes Lustre Log (LLog) over OSD work. The objective is to get rid of LLog code (which is tied to lvfs and fsfilt interfaces) and replace it with new MDS stack components.


Object handling in uOSS
Date 6 December 2007
Author Mike Pershin
Synopsis This document outlines new uOSS obdfilter code to implement object management such as pre-creation, create-on-write and destroy.


MDT base
Date 27 November 2007 (updated) / 8 May 2007 (original)
Author Hua Huang
Synopsis This document describes changes to the MDT base to adapt to the new layering stack, data structures and interfaces.


Remote ACL v2
Date 18 October 2007 (updated) / 3 September 2007 (original)
Author Yong Fan
Synopsis TBD.


Size-on-MDS replay and re-send
Date 30 August 2007 (updated) / 6 August 2007 (original)
Author Vitaly Fertman
Synopsis This document describes the changes in the replay and re-send procedures made for the SOM-related RPC handling.


FIDs
Date 21 August 2007 (original) / 15 April 2006 (original)
Author Yury Umanets
Synopsis This document describes the details to implement FIDs.


LRU resize
Date 21 August 2007 (updated) / 10 April 2007 (original)
Author Yury Umanets, Vitaly Fertman
Synopsis This document describes the implementation of locks on the server side to provide accounting and shrinking policies for all locks in a cluster.


Metadata stat-ahead
Date 14 August 2007 (updated) / 26 March 2007 (original)
Author Siyao Lai
Synopsis This document describes metadata stat-ahead, which is part of metadata improvements. The client will perform metadata stat-ahead when it detects readdir and sequential stat of directory entries therein.


o2iblnd to userland
Date 14 August 2007 (updated) / 20 July 2007 (original)
Author Unknown
Synopsis This document describes code changes to run o2iblnd to userland.


size-on-MDS Lustre Logging
Date 10 July 2007 (updated) / 4 May 2007 (original)
Author Vitaly Fertman
Synopsis This document describes Lustre Logging (LLOg'ing) size-on-MDS attribute changes on the OST, which guarantees that OST object attribute changes will not be lost somewhere on the way to the MDS (to do a node failure) and allows the MDS to re-enable size-on-MDS caching for the inode with the proper attributes.


CMD & 1.6.x client request changes
Date 21 June 2007 (updated) / 23 April 2007 (original)
Author Siyao Lai
Synopsis This document describes how a 1.6.x client chooses different request formats to interoperate with both 1.6 and 1.8 servers.


Lustre client GSS with Linux keyrings
Date 18 May 2007 (updated) / 26 April 2007 (original)
Author Eric Mei
Synopsis This document describes implementation of a GSS policy instance and a key type under the Linux keyring framework.


LLog CTXT REFCOUNT
Date 13 May 2007
Author WangDi
Synopsis This document describes the Lustre Log (LLog) ctxt refcount (indicating how many users are using this ctxt), to avoid the race between LLog cleanup and other LLog user threads.


CMD3 stability fixing
Date 10 May 2007 (updated) / 2 April 2007 (original)
Author Hua Huang
Synopsis This document describes post-CMD3 stability fixes to make CMD lustre stable, add new features, and improve several existing features.


Design and implementation of XNU port of Lustre client file system
Date 24 April 2007 (Updated) / 1 February 2007 (Original)
Author Nikita Danilov
Synopsis This document describes the structure of the Lustre client file system for XNU (Darwin kernel) In particular, changes necessary in the core XNU kernel to enable unique Lustre requirements (i.e., intents) are discussed.


Porting Early Lock Cancellation to CMD
Date 5 April 2007 (updated) / 30 March 2007 (original)
Author Vitaly Fertman
Synopsis This document describes changes in the Early Lock Cancellation mechanism to have it work with CMD.


Shrink qunit size when remaining quota too low (bug 10600)
Date 25 March 2007
Author ZhiYong Tian
Synopsis This document describes a code change to shrink the qunit size when the remaining quota on the MDS is too low.


Parallel lock callback
Date 16 February 2007 (updated) / 6 December 2006 (original)
Author YaWei Niu
Synopsis This document describes modifications to several functions related to callbacks.


Tdinal v4.0
Date 22 January 2007 (updated) / 18 April 2005 (original)
Author Unknown
Synopsis This document outlines implementation of a new tdinal module to fully use TDI features and make a compact implementation upon TDI and Lustre NAL without any extra layers.


Patchless zero copy socklnd
Date 22 January 2007 (updated) / 27 November 2006 (original)
Author Zhen Liang
Synopsis This document describes the details of patchless zero-copy (ZC) socklnd.


Client-OSS connection
Date 22 January 2007 (updated) / 7 March 2005 (original)
Author Qiang Zhao
Synopsis This document describes the details of the client-OSS connection.


MDD Lustre Log
Date 22 January 2007 (updated) / 15 August 2006 (original)
Author WangDi
Synopsis This document describes how to use the Lustre Log (LLog) APIs (unlink and setattr) in the MDD layer.


MDD LOV
Date 22 January 2007 (updated) / 29 June 2006 (original)
Author WangDi
Synopsis This document describes how to make the previous LOV implementation work in the current metadata stack and implement it with fewer modifications. MDD will use several APIs to connect with the LOV.


Enable quota limits greater than 4 GB
Date 22 January 2007 (updated) / 9 August 2006 (original)
Author ZhiYong Tian
Synopsis This document describes changes to enable quota limits greater than 4 GB (Bug 10707).


Split directory
Date 22 January 2007 (updated) / 8 September 2006 (original)
Author WangDi
Synopsis This document describes the handling of split. CMM will control the split process, check the split possibility and split the directory. MDC will transfer the split directory entries to the slave MDS. The MDT will receive the split entries and write them to the slave objects.


Implementation of hash table
Date 22 January 2007
Author ZhangYong Yu
Synopsis This document outlines the implementation of a hash table for client UUID/NID export lookups (Bug 11013).


Inodebits lock performance improvement
Date 21 January 2007 (updated) / 19 December 2006 (original)
Author Bobi Jam
Synopsis This document outlines a performance improvement in Lustre DLM plain and inodebits lock management to handle large-scale usage).

2006

CMD rename locking
Date 18 September 2006 (updated) / 13 August 2006 (original)
Author Yury Umanets
Synopsis This document describes how the rename locking mechanism should be done in CMD3, by introducing CMD rename sanity checks and semantically proper rename locking as described in Linux documentation.


Open Recovery in CMD
Date 11 September 2006 (updated) / 4 September 2006 (original)
Author Hua Huang
Synopsis This document describes the details of open recovery handling in CMD.


CMD orphans handling
Date 10 September 2006 (updated) / 6 February 2006 (original)
Author Mike Pershin
Synopsis This document describes updated functionality so a file is not removed while open and by bottom fs orphan cleanup while the MDS re-initializes after a failure.


CMD MDS recovery
Date 22 August 2006 (updated) / 6 February 2006 (original)
Author Mike Pershin
Synopsis This document describes recovery changes in CMD.


OSD transaction API in CMD
Date 21 August 2006 (updated) / 25 July 2006 (original)
Author Hua Huang
Synopsis This document describes details of the OSD transaction API design, implementation and usage in CMD.


Object Index
Date 18 July 2006 (updated) / 13 July 2006 (original)
Author Nikita Danilov
Synopsis This document describes object index (OI), a server-side component of CMD3 Lustre architecture that maps FID into a "storage cookie".


Open handling in CMD
Date 15 July 2006 (updated) / 5 July 2006 (original)
Author Hua Huang
Synopsis This document describes details of open handling in CMD.


FIDs location database (FLD)
Date 4 July 2006 (updated) / 22 June 2006 (original)
Author Yury Umanets
Synopsis This document describes the FIDs location database (FLD), which is used in Lustre to provide location information for the node that a specified FID lives on.


Sequence Management
Date 29 June 2006 (updated) / 25 May 2006 (original)
Author Yury Umanets
Synopsis This document describes details of FIDs sequence management. The main focus is that sequences are cluster-wide and unique, which requires that they be specially managed.


Index API module
Date 20 June 2006 (updated) / 15 June 2006 (original)
Author Nikita Danilov
Synopsis This document describes implementation of Index API Module (IAM) functionality. IAM provides an abstraction of the transactional persistent container, where user-supplied records (identified by keys), are stored and retrieved.


CMM
Date 11 June 2006 (updated) / 8 May 2006 (original)
Author Mike Pershin
Synopsis This document introduces the Clustered Metadata Manager (CMM), a new layer in the MDS which tracks all clustered metadata issues and relationships. CMM acts as a layer between the MDT and MDD, provides MDS-MDS interaction, queries and updates the FLD, does local and remote operations (if needed), and does rollback (epoch control and undo logging).


MDD over OSD
Date 9 June 2006 (updated) / 4 June 2006 (original)
Author WangDi
Synopsis This document describes how the new metadata stack layer, MDD, will run on the DT device.


Metadata API
Date 25 May 2006 (updated) / 9 February 2005 (original)
Author Yury Umanets
Synopsis This document describes a set of related structures that are needed to deliver metadata API functionality. These structures consist of three types: information presenting actions that should be performed against a Lustre object (file, directory, etc.), information presenting an object that an action should be performed against, and other structures needed to connect a metadata stack to another Lustre sub-system.


MD API
Date 13 May 2006 (updated) / 28 April 2006 (original)
Author Nikita Danilov
Synopsis This document describes the overall structure of server-side and, potentially, client-side code layering. Specifically, it introduces mechanisms that are used to separate various layers from one another while providing communication flexibility without losing type-safety. This cleanup of code interfaces related to layering is necessary because of changes to MD server code layering in CMD3 development.


Adding MDT with MountConf
Date 9 April 2006 (updated) / 31 March 2006 (original)
Author Huang Hua
Synopsis This document describes updates to MountConf based on Lustre changes due to CMD. These updates include the addition of MDT with MountConf in CMD and consideration of a new layering model.


Lustre Parallel CIFS driver
Date 2 April 2006 (updated) / 16 February 2006 (original)
Author Matt Wu
Synopsis This document introduces the new Lustre Parallel CIFS (pCIFS) driver.


Lustre Log API
Date 20 March 2006 (updated) / 11 November 2005 (original)
Author Unknown
Synopsis This document describes the Lustre Log (LLog) API.


Security API and null policy
Date 15 March 2006
Author Eric Mei
Synopsis This document describes the security API, which will provide security service to PTLRPC (sptlrpc) and a policy module to implement the current "no security" scenario (i.e. the null policy).


Lustre capabilities (OSS and MDT)
Date 25 February 2006 (updated) / 7 June 2005 (original)
Author Lai Siyao
Synopsis This document describes OSS and MDT capabilities. OSS capability (generated by the MDS) is sent to a client when the client opens/truncates a file, and then included in each request from a client to the OSS to authorize an action. MDS capability (which could be called FID capability), proves that a client has access to an FID.


Unique client connections (Bug 9636)
Date 9 February 2006 (updated) / 26 January 2006 (original)
Author Andreas Dilger
Synopsis This document outlines functionality to ensure that only a single instance of a client establishes a Lustre connection to a server, even though the client may be re-sending one or more connect requests after server unresponsiveness (Bug 9636).


lustre_msg v2
Date 5 February 2006 (updated) / 17 January 2006 (original)
Author Lai Siyao
Synopsis This document describes a new version (v2) of lustre_msg that will support lustre_msg embedding, hide RPC-specific data, support security and keep protocol compatible.


LAID5 / LAID3
Date 20 January 2006
Author Unknown
Synopsis This document describes enhancement of the LOV layer to implement the data layout pattern (LAID5/LAID3).