WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.
Architecture - Lustre HLDs
From Obsolete Lustre Wiki
Jump to navigationJump to search
You can access archived Lustre high-level design (HLD) documents in PDF format from this page. You can also download the HLDs in LyX format from the Lustre CVS repository.
Note: The documents accessed from this page reflect the state of design of a Lustre feature at a particular point in time and may contain outdated information.
Checking out HLDs from CVS
Note: The HLD documents have not been migrated to Git. They can still be accessed in the CVS repository.
Assuming you have CVS installed, check out the Lustre HLDs as follows:
1. Set CVSROOT in your environment to :pserver:anonymous@cvs.lustre.org:/lustre.
(Alternately, you can add the option -d :pserver:anonymous@cvs.lustre.org:/lustre when you run cvs login.)
2. Run cvs login. No password is needed. Press Enter to continue.
3. Check out the HLD module (doc/HLD)
Example:
% cvs -d :pserver:anonymous@cvs.lustre.org:/lustre co -P doc/HLD cvs checkout: Updating doc/HLD U doc/HLD/0-conf-HLD.lyx U doc/HLD/Channel Bonding_06_15_09.odt U doc/HLD/Channel_Bonding_06_15_09.pdf U doc/HLD/CollectorHLD.lyx U doc/HLD/End-to-End-Integrity-2009-06-15.pdf U doc/HLD/End-to-End-Integrity-2009-06-15_Changes_Accepted.odt U doc/HLD/FSCK_Design-2009-06-15-09.odt U doc/HLD/FSCK_Design-2009-06-15-09.pdf U doc/HLD/FilterFS_HLD.lyx U doc/HLD/GUI-HLD.lyx U doc/HLD/HPCS_CMD_06_15_09.odt U doc/HLD/HPCS_CMD_06_15_09.pdf U doc/HLD/IO-API-HLD.lyx U doc/HLD/IntervalTree_hld.lyx U doc/HLD/JKD_Wiki_V1_2009_08_07_Lustre_HPCS_Overview.pdf U doc/HLD/LAID.lyx U doc/HLD/LAID5_DLD.lyx ...
HLDs Available as PDFs
For each HLD, a link to the PDF is provided, along with the title, author, date (original and update, if applicable), and synopsis.
2008
| Platform-independent Proc Interface | |
| Date | 18 September 2008 (updated) / 7 May 2008 (original) |
| Author | WangDi, Komal Nawandar |
| Synopsis | This document describes how to implement a platform-independent proc interface for Lustre. The basic concept is that the platform-independent proc tree will be maintained in the kernel base, which is similar to the Linux proc tree (called the parameters tree in this HLD). The tree will only be accessed by the lctl params command with ioctl. |
| Commit-on-Sharing | |
| Date | 8 August 2008 (updated) / 25 April 2008 (original) |
| Author | Mike Pershin, Alexander 'Zam' Zarochentsev |
| Synopsis | This document outlines the commit-on-sharing feature, which helps recover more clients using version-based recovery. This feature does a 'commit' if any resource has uncommitted changes from more than one node. Therefore, all replays on clients no longer depend on other clients, and clients can recover cleanly even if several clients are missed. |
| LNET SMP Performance Improvement | |
| Date | 5 August 2008 (updated) / 21 June 2008 (original) |
| Author | Liang Zhen |
| Synopsis | This document describes LNET scalability improvements, specifically, scalable MD matching and grained locking for LNET. |
| Interoperability at the Server-Side | |
| Date | 5 August 2008 (updated) / 28 April 2008 (original) |
| Author | Rahul Deshmukh |
| Synopsis | In older Lustre releases, the metadata server (MDS) and its storage were FID-less. In newer releases, the MDS and its storage are FID-enabled. This document outlines network protocol-related interoperability changes that enable clients to use FID-less and FID-enabled protocols. This document also covers disk interoperability, i.e., a FID-enabled MDS running on a FID-less disk format. |
| Lustre client GSS with Linux keyring | |
| Date | 22 July 2008 (updated) / 26 March 2007 (original) |
| Author | Eric Mei |
| Synopsis | This document describes how Lustre will use keyring to populate and manage the client GSS context, to replace the current mechanism which uses pipefs. |
| Lustre Capabilities Security Level | |
| Date | 20 June 2008 |
| Author | Fan Yong |
| Synopsis | This document introduces MDS/OSS capabilities to enhance Lustre system security. It defines Lustre capabilities security levels to match different security requirements. This document also describes how Lustre runs under different capabilities security levels. |
| Version-based recovery | |
| Date | 12 June 2008 (updated) / 11 May 2007 (original) |
| Author | Peter Braam, Mike Pershin |
| Synopsis | This document describes updates to Lustre version-based recovery. This feature allows clients to replay if the objects that the client is using have exactly the same version as during the original execution. After replay, recovery can complete successfully if the data that clients obtained from servers before recovery is present and has not rolled back. |
| Sptlrpc Interoperability | |
| Date | 2 June 2008 (updated) / April 2008 (original) |
| Author | Eric Mei |
| Synopsis | This document outlines the sptlrpc aspect of interoperability during migration from 1.6-based, FID-less, Lustre clients using ldiskfs as a back-end file system, to clusters based on FIDs and the ZFS filesystem. The complete architecture and design for Lustre interoperability is available at Interoperability FIDS ZFS. |
| LLog on a New Device Stack | |
| Date | 28 May 2008 |
| Author | Yury Umanets |
| Synopsis | This document describes the design of LLog on a new metadata stack (MDD, OSD and others), which is required for many features in uMDS and uOSS components. This functionality is also needed for better portability, a key requirement for the upcoming Solaris port. |
| Metadata write-back cache | |
| Date | 12 May 2008 (updated) / 18 December 2007 (original) |
| Author | Nikita Danilov |
| Synopsis | This document introduces the metadata write-back cache design. Write-back cache is a client and server mechanism for caching metadata operations in a manner similar to the existing cache for data operations. |
| Userland proc Interface | |
| Date | 7 May 2008 |
| Author | WangDi |
| Synopsis | This document describes how to implement proc and ioctl mechanisms on the user-base server. The basic concept is that the user-base server maintains a platform-independent tree, and lctl retrieves parameters from the server instead of procfs. Communication between the lctl process and user-base server process will be implemented by socket (implemented in another task). |
| Lustre Windows client | |
| Date | 8 April 2008 |
| Author | Tony Mason |
| Synopsis | This document introduces a basic working model for developing a native Lustre client for Windows. It describes several key issues in this implementation. |
| CTDB Node and IP Management | |
| Date | 26 March 2008 |
| Author | Matt Wu |
| Synopsis | This document describes CTDB analysis and an improved design on CTDB flexible node/IP management. As part of the Lustre pCIFS project, CTDB improvement focuses on failover support to all pCIFS clients. |
| Epochs | |
| Date | 19 March 2008 (updated) / 14 February 2008 (original) |
| Author | Alexander 'Zam' Zarochentsev |
| Synopsis | This document introduces epochs, a collection of file system modifications that is used to merge distributed state updates (both data and metadata) in a redundant cluster configuration. This HLD is based on the Epochs architecture page by Nikita Danilov. |
| Subtree locks | |
| Date | 13 March 2008 (updated) / 1 February 2008 (original) |
| Author | Alexander 'Zam' Zarochentsev, Vladimir Savaliev |
| Synopsis | This document covers the design for subtree lock, a lock on a directory which protects the entire namespace (for its part) rooted at that directory. A subtree lock should be optimal for workloads where clients work in isolated directories and not make things worse in highly-contended workloads by resorting to the current client-server locking protocol. |
| Fragmentation reporting tool (Bug 10555) | |
| Date | 11 March 2008 (updated), 18 June 2007 (original) |
| Author | Andreas Dilger, Kalpak Shah |
| Synopsis | This document introduces the user interface and internal implementation of an efficient fragmentation reporting tool for ext3/4. The main objective is to easily and efficiently allow inspection of the disk layout of one or more files without requiring user access to each of the underlying OST devices. There are three principal components to this design: ldiskfs interface, Lustre RPC transport for the extent mapping information, and llite interface. |
| Client I/O stack layering cleanup | |
| Date | 6 March 2008 (updated), 21 May 2007 (original) |
| Author | Nikita Danilov |
| Synopsis | This document describes updates to the Lustre client I/O stack code, to reduce the number of bugs in the data paths. The main objective is to introduce more systematic layering on the client. |
| Bug 14115 - Extended Attributes in dnode | |
| Date | 7 February 2008 (updated) / 29 December 2007 (original) |
| Author | Girish Shilamkar |
| Synopsis | This document discusses how Extended Attributes (EAs) in dnode can meet the requirement of users who would like a mechanism to store EAs much nearer to a file's metadata. |
| [[Media:UOSS-filter-objects.pdf|Object handling in uOSS | |
| Date | 6 February 2008 (updated) / 6 December 2007 (original) |
| Author | Mike Pershin |
| Synopsis | This document outlines new uOSS obdfilter code to implement object management such as pre-creation, create-on-write and destroy. |
| uMDS Transaction API | |
| Date | 5 February 2008 (updated) / 17 January 2008 (original) |
| Author | Amit Sharma |
| Synopsis | This document outlines the high-level design for the uMDS transaction API task, which aims to replace the old transaction API with the new transaction API. |
| Interoperability - client recovery | |
| Date | 5 February 2008 (updated) / 22 November 2007 (original) |
| Author | Amit Sharma |
| Synopsis | This document outlines the high-level design of changes to the 1.6.x client to enable it to recover after any fault when it is running in interoperability mode. |
| Changelogs and feeds | |
| Date | 30 January 2008 |
| Author | Nathan Rutman |
| Synopsis | This document describes the design to implement changelogs as a low-level service for rollback logs, replication logs and audit logs. |
| Samba oplock/share mode conflicts | |
| Date | 16 January 2008 |
| Author | Unknown |
| Synopsis | This document describes the design for pCIFS multiple open instances support. In default new open request will cause share mode conflicts and extra oplock broken notification, then it prevents pCIFS to function correctly. |
| Commit callback for DMU transactions | |
| Date | 3 January 2008 (updated) / 24 December 2007 (original) |
| Author | Andreas Dilger |
| Synopsis | This document outlines implementation of DMU Commit Callback, a mechanism to allow upper layers of the software stack to get asynchronous notification of low-level events, in order to enable a coherent distributed transaction mechanism for Lustre, but without having to know the internal details of transaction mechanism. |
| Journal checksums | |
| Date | 3 January 2008 (updated) / 23 May 2007 (original0 |
| Author | Girish Shilamkar |
| Synopsis | This document outlines the design to implement journal checksums to ensure the correctness of the journal befor it is written to disk. This design includes changes to e2fsprogs to do checksum-based recovery and ensure that older e2fsprogs or kernels do not recover the journal when they had been checksummed. This design will also take advantage of checksumming by asynchronously writing the commit record. |
2007
| LLog and MountConf via OSD | |
| Date | 12 December 2007 (updated) / 8 November 2007 (original) |
| Author | Yury Umanets |
| Synopsis | This document outlines the design for how the Lustre Log (LLog) will work with OSD. This is required for a number of features in uMDS and uOSS components. The objective is to get rid of the Fsfilt interface in the LLog backend and use OSD as the backing storage interface. Another objective is to re-work LLog to make it suitable for use in a new device stack (via lu_device, lu_object) with uMDS and uOSS. |
| Mount permission | |
| Date | 10 December 2007 |
| Author | Fan Yong |
| Synopsis | This document describes the design for new utility (lctl mount_perm) to configure system mount permissions. This utility will allow administrators to grant, revoke, modify and query mount permission configurations for a specified client or list all mount permission configurations in a specified Lustre system. This is a Lustre 1.8 feature. |
| MDT base in CMD | |
| Date | 21 August 2007 (updated) / 13 April 2006 (original) |
| Author | Nikita Danilov |
| Synopsis | This document describes changes to the MDT base (for CMD3), to handle network requests, deal with locks, etc. |
| FIDs Location Database (FLDs) | |
| Date | 21 August 2007 (updated) / 3 February 2006 (original) |
| Author | Yury Umanets, Nikita Danilov |
| Synopsis | This document outlines the new FIDs Location Database (FLDs), to enable clients and metadata servers to find out what server object with a given FID is stored. |
| FIDs | |
| Date | 21 August 2007 (updated) / 20 January 2006 (original) |
| Author | Yury Umanets |
| Synopsis | This document introduces FIDs to make cleanups of the object identification protocol possible for regular Lustre activities, and for write-back caches and proxy nodes. |
| Stat-ahead | |
| Date | 14 August 2007 (updated) / 16 March 2007 (original) |
| Author | Lai Siyao |
| Synopsis | This document describes metadata stat-ahead, which is part of metadata improvements. The client will perform metadata stat-ahead when it detects readdir and sequential stat of directory entries therein. |
| Remote ACL v2 | |
| Date | 8 August 2007 (updated) / 18 October 2007 (original) |
| Author | Fan Yong |
| Synopsis | This document describes updates to improve defects in the Remote ACL v1 design.
Note: For the original Remote ACL design, see Remote ACL |
| Size-on-MDS | |
| Date | 21 July 2007 |
| Author | Vitaly Fertman |
| Synopsis | This document describes the size-on-MDS improvement, which includes the caching of the inode size, blocks, ctime and mtime on the MDS. Attribute caching enables clients to avoid making RPCs to the OSTs to find the attributes encoded in the file objects kept on those OSTs, resulting in significantly-improved performance of listing directories. |
| Multiple Mount Protection | |
| Date | 25 June 2007 (updated), 3 April 2007 (original) |
| Author | Kalpak Shah |
| Synopsis | This document introduces the Multiple Mount Protection (MMP) feature, intended to protect the filesystem from being mounted more than one time simultaneously. MMP will also protect changes by e2fsprogs to the filesystem, if the filesystem is mounted. |
| e2fsck heuristics for detecting corrupted nodes | |
| Date | 21 June 2007 (updated), 6 April 2007 (original) |
| Author | Girish Shilamkar |
| Synopsis | This document describes new e2fsck functionality, a badness counter, to indicate the extent of inode corruption. If the 'badness' is above a certain threshold, the complete inode is deemed to be corrupt and the inode is deleted. |
| MDS and OSS Capabilities | |
| Date | 24 May 2007 |
| Author | Unknown |
| Synopsis | This document describes capabilities design for the MDS and OSS. |
| CMD & 1.6.x client request changes | |
| Date | 20 May 2007 (updated) / 12 March 2007 (original) |
| Author | Lai Siyao |
| Synopsis | This document describes how a 1.6.x client chooses different request formats to interoperate with both 1.6 and 1.8 servers. |
| Lustre Log ctext reference count | |
| Date | 9 May 2007 (updated) / 14 March 2007 (original) |
| Author | WangDi |
| Synopsis | This document introduces the Lustre Log (LLog) ctxt reference count (refcount) to protect ctxt from being freed improperly during LLog cleanup (Bug 10800). |
| LRU resize | |
| Date | 2 May 2007 (updated) / 15 March 2007 (original) |
| Author | Yury Umanets, Vitaly Fertman |
| Synopsis | This document describes fixes to several LRU sizing issues, by introducing a number of solutions to control a client's LRU size (or number of cached locks according to a chosen policy) and to control the average number of locks in a cluster given to clients by servers and have automatic lock LRU size adjustments. |
| Ext3 inode versioning | |
| Date | 4 April 2007 (updated) / 13 December 2006 (original) |
| Author | Andreas Dilger |
| Synopsis | This document describes on-disk implementation of inode versions for the ext3 filesystem used by Lustre. Some parts of the design are mandatory, while others are included to make the implementation more useful generally and acceptable to the ext3 community at large. |
| FID to inode translation | |
| Date | 30 March 2007 (updated) / 12 March 2007 (original) |
| Author | Yury Umanets |
| Synopsis | This document describes improvements to a new generation policy created in the CMD3 project. The policy is based on FID, as the unique object ID. |
| Adaptive RPC timeouts | |
| Date | 28 March 2007 |
| Author | Eric Barton, Nathan Rutman |
| Synopsis | This document describes the new adaptive timeouts feature, which will enable the client's RPC timeout to scale with server congestion. |
| CMD3 stability fixing | |
| Date | 25 March 2007 (updated) / 15 March 2007 (original) |
| Author | Huang Hua |
| Synopsis | This document outlines stability fixes, specifically fixes for issues found during the CMD3 timeframe. These fixes should provide more stable baseline code. |
| Parallel lock callback | |
| Date | 16 February 2007 (updated) / 28 November 2006 (original) |
| Author | Niu YaWei |
| Synopsis | This document introduces a mechanism to perform LDLM lock callbacks in parallel. |
| Grants | |
| Date | 29 January 2007 (updated) / 22 January 2007 (original) |
| Author | Unknown |
| Synopsis | This document explains how space grants work and proposes several alternative strategies for support grants under an I/O scheduler. |
| Lustre mmap support | |
| Date | 28 January 2007 |
| Author | Unknown |
| Synopsis | This document describes new mmap support in Lustre, to enable access pages via mmap path to be protected by ldlm extent locks. |
| inodebits interoperability feature | |
| Date | 28 January 2007 |
| Author | Unknown |
| Synopsis | This document describes the addition of inodebits interoperability. The inodebits feature is a performance enhancement to parallel directory accesses. It distinguishes a LOOKUP lock (which protects the cached entry and owner/group/mode information) from an UPDATE lock (which protects the directory data itself. |
| Splitting directories | |
| Date | 22 January 2007 (updated) / 25 August 2006 (original) |
| Author | WangDi |
| Synopsis | This document introduces the splitting directory feature. In this design, when directory entries exceed a specified limit in CMD, the directory will be split and distributed over several MDSs for load balancing. NOTE: Splitting is just a feature for this cycle, it will be removed after this cycle. |
| Enable quota limits greater than 4 GB | |
| Date | 22 January 2007 (updated) / 18 July 2006 (original) |
| Author | WangDi |
| Synopsis | This document describes changes to enable quota limits greater than 4 GB (Bug 10707). |
| Interval tree for extent locks | |
| Date | 21 January 2007 (updated) / 13 December 2006 (original) |
| Author | Huang Wei |
| Synopsis | This document introduces a performance improvement in Lustre DLM extent lock management (Bug 11300). |
2006
| Patchless TCP Zero Copy Socklnd | |
| Date | 6 December 2006 (updated) / 20 November 2006 (original) |
| Author | Liang Zhen |
| Synopsis | This document describes an update so socklnd will send out an explicit ACK for paged bulk, so the zero-copy patch can be removed. |
| CMD3 security layering | |
| Date | 29 September 2006 |
| Author | Nikita Danilov |
| Synopsis | This document describes how the security infrastructure (capabilities, remote users, and remote ACLs) are integrated into the new MDS layering. |
| Open recovery | |
| Date | 28 August 2006 |
| Author | Huang Hua |
| Synopsis | This document describes the handling of open recovery in CMD. |
| CMD MDS recovery | |
| Date | 27 July 2006 |
| Author | Mike Pershin |
| Synopsis | This document describes basic recovery changes in CMD. |
| Open handling on layered MDS | |
| Date | 3 July 2006 |
| Author | Mike Pershin |
| Synopsis | This document describes the handling of open/close requests for files in the new layered MDS environment. |
| Group locks | |
| Date | 5 May 2006 |
| Author | Unknown |
| Synopsis | This document introduces functionality to lock files (inodes) for exclusive access by processes belonging to a logical group of processes. |
| Optimized stripe assignment (QOS) | |
| Date | 24 April 2006 |
| Author | Nathan Rutman |
| Synopsis | This document describes a new optimization (integrating OSS optimization with QOS). Combining this optimization with several other optimizations enables the allocation of stripes for new files. |
| Ping evictor | |
| Date | 24 April 2006 (updated) / 22 March 2005 (original) |
| Author | Phil Schwan |
| Synopsis | This document outlines updates to ping evictor functionality, which causes a client to be evicted if an MDT or OST detects it has not received traffic or an export from some period of time (a percentage of the timeout value). |
| CMD with MountConf | |
| Date | 2 April 2006 (updated) / 21 March 2005 (original) |
| Author | Huang Hua |
| Synopsis | This document describes updates to enable MountConf to work in the CMD3 version of Lustre. |
| Lustre Parallel CIFs Driver | |
| Date | 2 April 2006 (updated) / 11 January 2006 (original) |
| Author | Matt Wu |
| Synopsis | This document introduces the new Lustre Parallel CIFS (pCIFS) driver to use the osr filter kit to implement parallel I/O dispatching, instead of the original design with the ifskit sfilter. |
| GSS policy with Kerberos 5 mechanism | |
| Date | 29 March 2006 |
| Author | Eric Mei |
| Synopsis | This document describes implementation of the GSS policy for the secure PTLRPC framework and the Kerberos 5 mechanism for the GSS policy. |
| Create on Write (CROW) | |
| Date | 29 March 2006 (updated) / 10 February 2006 (original) |
| Author | Yury Umanets |
| Synopsis | This document describes the CReate on Write (CROW) feature, where a client sends an RPC to the MDT, and then the MDT creates an object on the OST server using a pre-creation pools approach. |
| MDC changes in CMD | |
| Date | 27 March 2006 (updated) / 24 March 2006 (original) |
| Author | Huang Hua |
| Synopsis | This document describes updates to MDC to adapt to new data structures and protocols because of the addition of FIDs. |
| OSD API | |
| Date | 20 March 2006 (updated) / 27 February 2006 (original) |
| Author | Huang Hua |
| Synopsis | This document describes updates to the OSD API, based on changes in layering on the MDS. |
| LAID locking | |
| Date | 23 March 2006 (updated) / 24 January 2006 (original) |
| Author | Qian YingJin |
| Synopsis | This document introduces a new locking model for mirror LAID and LAID5 to avoid locking across more than one OST. |
| Gaps handling simplification | |
| Date | 10 March 2006 (updated) / 17 February 2006 (original) |
| Author | Huang Hua |
| Synopsis | This document describes issues related to the simplification of gaps handling in Lustre. Gap is an issue in Lustre recovery time. Handling gaps correctly, with simplicity, is important. |
| CMD orphans handling | |
| Date | 9 March 2006 |
| Author | Mike Pershin |
| Synopsis | This document provides a review of the orphans handling scheme and adaptations to the CMD environment. |
| Cascading timeouts | |
| Date | 9 March 2006 |
| Author | Peter Braam, Alex Tomas |
| Synopsis | This document describes the issue of cascading timeouts (caused by nesting requests related to the creation of OST objects from MDS and clustered metadata), and changes to resolve the problem. |
| Patch-free Lustre client in Darwin 8.x (Tiger) operating system | |
| Date | 8 March 2006 (updated) / 7 February 2006 (original) |
| Author | Liang Zhen |
| Synopsis | This document describes how to construct a patch-free Lustre client in the Darwin 8.x (Tiger) operating system. |
| Network resource handling | |
| Date | 8 March 2006 (updated) / 1 March 2006 (original) |
| Author | WangDi |
| Synopsis | This document describes the move of network-level resource handling from OSD/MDD to MDT/OST, with support from the MDD and OSD layer. |
| readdir | |
| Date | 7 March 2006 (updated) / 25 January 2006 (original) |
| Author | WangDi |
| Synopsis | This document describes making readdir POSIX-compliant, especially for split directory nodes. |
| CMD server API | |
| Date | 6 March 2006 / 25 January 2006 (original) |
| Author | Huang Hua, Mike Pershin |
| Synopsis | This document introduces the design of the clustered metadata server (CMD) environment in Lustre. CMD will overcome the limitations of Lustre having only one active metadata server in a cluster (with all metadata managed by this server). |
| Security API and null policy | |
| Date | 6 March 2006 |
| Author | Eric Mei |
| Synopsis | This document outlines the general framework for the security API, which will provide security service to PTLRPC (sptlrpc) and a policy module to implement the current "no security" scenario (i.e. the null policy). |
| CMD rollback | |
| Date | 2 March 2006 |
| Author | Peter Braam, Mike Pershin |
| Synopsis | This document describes rollback, a recovery mechanism for a cluster of metadata servers (CMD). In rollback, nodes will engage in a distributed algorithm that restores the disk state to a snapshot based on an undo log.
NOTE: This HLD supercedes Rollback |
| Networking for MDT/OST | |
| Date | 27 February 2006 (updated) / 13 February 2006 (original) |
| Author | Niu YaWei |
| Synopsis | This document describes networking elements (exports and imports, network recovery and DLM locks) that will move into the MDT and OST layers with the new MDS. |
| MDS layering | |
| Date | 27 February 2006 |
| Author | Mike Pershin |
| Synopsis | This document describes server layers and their functionality based on requirements of the MDS design. |
| Client metadata API | |
| Date | 27 February 2006 (updated) / 23 February 2006 (original) |
| Author | Alex Tomas, Yury Umanets |
| Synopsis | This document outlines the design for a metadata API to build a stackable, portable client. |
| LAID5 | |
| Date | 23 February 2006 (updated) |
| Author | Unknown |
| Synopsis | This document describes enhancements to the LOV layer to implement the data layout pattern LAID5. |
| MDS on OSD | |
| Date | 27 February 2006 (updated) / 17 January 2006 (original) |
| Author | WangDi |
| Synopsis | This document describes how the MDD works based on OSD and what kind of OSD support is needed for the MDD. |
| CMD rename locking | |
| Date | 9 February 2006 (updated) / 24 January 2006 (original) |
| Author | Niu Yawei |
| Synopsis | This document describes the design for the CME rename locking feature. |
| lustre_msg v2 | |
| Date | 5 February 2006 (updated) / 11 January 2006 (original) |
| Author | Lai Siyao |
| Synopsis | This document describes a new version (v2) of lustre_msg that will support lustre_msg embedding, hide RPC-specific data, support security and keep protocol compatible. |
| Index API Module | |
| Date | 30 January 2006 (updated) / 20 January 2006 (original) |
| Author | Nikita Danilov |
| Synopsis | This document describes the design for the Index API Module (IAM), which is used by other Lustre components to persistently and transactionally store keyed data items. |
| Truncate without extent lock on the client | |
| Date | 27 January 2006 |
| Author | Unknown |
| Synopsis | This document introduces functionality enabling a Lustre client to perform file truncate without obtaining an extent lock. The lock is taken by the OST. |
| Storage Management File System | |
| Date | 27 January 2006 (updated) / 14 February 2005 (original) |
| Author | Peter Braam, Mike Pershin |
| Synopsis | This document outlines the design for the Storage Management File System (SMFS), offering these features: works as a transparent pseudo-filesystem on top of the real filesystem, have access to Lustre's API-like LLog and btree, be portable to filesystems other than ext3, and be able to call side-functions from other modules and plugins using its own API. |
| Rollback | |
| Date | 27 January 2006 |
| Author | Unknown |
| Synopsis | This document describes rollback, a recovery mechanism for a cluster of metadata servers (CMD). In rollback, nodes will engage in a distributed algorithm that restores the disk state to a snapshot based on an undo log.
Note: This HLD is superceded by CMD rollback |
| User revoke | |
| Date | 27 January 2006 (updated) / 31 January 2005 (original) |
| Author | Peter Braam, Eric Mei |
| Synopsis | This document introduces user revoke functionality, which will be added to the existing lctl tool. If a system administrator needs to remove a user from the Lustre filesystem (e.g., because the user is known to be malicious or an account is compromised), the SA can use this functionality on MDSs to prevent the user from accessing the Lustre filesystem right away. |
| Remote UID/GID handling | |
| Date | 27 January 2006 (updated) / 10 February 2005 (original) |
| Author | Peter Braam, Eric Mei |
| Synopsis | This document provides an architectural design to perform UID/GID translation between remote clients and a local user database, handle client program calling setuid/setgid/setgroups syscalls to get unusual privilege, handle supplementary groups membership and offer security policies in situations with/without strong authentication, like Kerberos v5. |
| Remote ACL | |
| Date | 27 January 2006 (updated) / 22 February 2005 (original) |
| Author | Peter Braam, Eric Mei |
| Synopsis | This document introduces the remote ACL feature that provides support, on a remote client, for an ACL-based permission check and setacl/getacl.
Note: For the follow-on version of Remote ACL (v2), see [[Media:Remote_acl_HLD_v2.pdf Remote ACL v2]. |
| Quota for Lustre | |
| Date | 27 January 2006 |
| Author | Unknown |
| Synopsis | This document describes the design for the quotas feature to meet these requirements: Lustre can operate and enforce disk block quota and file quota, support hard and soft quota, and use central management tools to set limits for users and initialize quota check operations. |
| OST I/O locks | |
| Date | 27 January 2006 |
| Author | Oleg Drokin, Nikita Danilov |
| Synopsis | This document describes functionality for OST I/O locks, enabling a client to request the OST perform LDLM locking for certain read/write requests and fixing the OST/LDLM so other extent locks are never granted to the liblustre client. |
| Client-OSS connection | |
| Date | 27 January 2006 (updated) / 2 March 2005 (original) |
| Author | Peter Braam, Eric Mei |
| Synopsis | This document introduces the design to establish GSS connections between clients and the OSS, and GSS connections between servers. |
| Old confobd | |
| Date | 27 January 2006 (updated) / 17 June 2005 (original) |
| Author | Nathan Rutman |
| Synopsis | This document describes confobd, which was implemented to make it easier to run Lustre as a rootfs. |
| Tdinal v3.0 | |
| Date | 27 January 2006 (updated) / 23 February 2005 (original) |
| Author | Unknown |
| Synopsis | This document outlines implementation of a new tdinal module to fully use TDI features and make a compact implementation upon TDI and Lustre NAL without any extra layers. |
| Lustre configuration | |
| Date | 27 January 2006 |
| Author | Unknown |
| Synopsis | This document introduces the design for a new configuration and management interface to make Lustre configuration dramatically simpler. |
| Metadata refinements | |
| Date | 27 January 2006 (updated) / 6 May 2005 (original) |
| Author | Peter Braam |
| Synopsis | This document describes a number of metadata handling improvements. |
| CIFS LOV EA | |
| Date | 27 January 2006 (updated) / 5 September 2005 (original) |
| Author | Matt Wu |
| Synopsis | This document describes LOV EA support, a sub-task of the Windows CIFS parallel I/O filter project. |
| LLog basics | |
| Date | 27 January 2006 (updated) / 5 August 2005 (original) |
| Author | Nathan Rutman |
| Synopsis | This document describes the basics of Lustre Log (LLog). |
| I/O API | |
| Date | 27 January 2006 (updated) / 5 October 2005 (original) |
| Author | Yury Umanets |
| Synopsis | This document introduces the design for an I/O API that is usable on the server and client. |
| Trap Monitor application | |
| Date | 27 January 2006 (updated) / March 2005 (original) |
| Author | Gustavo Rahal |
| Synopsis | This document describes the design for the Trap Monitor application, which will display traps from all accessible Lustre filesystems. The tool will include an interface to display all collected traps in an organized and structured manner. |
| GSS | |
| Date | 27 January 2006 (updated) / 27 February 2005 (original) |
| Author | Peter Braam, Eric Mei |
| Synopsis | This document describes implementation of GSS, which will meet these requirements: support the GSS API framework in Lustre, support Kerberos 5 (as a mechanism of GSS API), and support user authentication and integrity/privacy protection for ptlrpc messages between clients and MDSs. |
| Global NameSpace (GNS) | |
| Date | 27 January 2006 (updated) / 20 January 2005 (original) |
| Author | Peter Braam, Yury Umanets |
| Synopsis | This document introduces Global NameSpace (GNS) in llite. |
| CIFS parallel I/O filter | |
| Date | 27 January 2006 (updated) / 7 September 2005 (original) |
| Author | Matt Wu |
| Synopsis | This document proposes the CIFS parallel I/O filter, to simplify Lustre Windows porting. |
| Lustre trap collector | |
| Date | 27 January 2006 |
| Author | Johan Dahlin |
| Synopsis | This document describes the Lustre trap collector, a daemon that usually runs on an MDS in the Lustre filesystem. |
| Bug 5921 | |
| Date | 27 January 2006 |
| Author | Unknown |
| Synopsis | This document describes the requirements of Bug 5921, rebooted Lustre clients be efficiently removed from the lists of connected clients maintained by the OSTs and MDTs. |
| Auditing | |
| Date | 27 January 2006 (updated) / 3 May 2005 (original) |
| Author | Peter Braam |
| Synopsis | This document describes how to generate LLog entries for undo, replay and auditing. Additionally, it describes the infrastructure required for the auditing component in Lustre. |
