WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.

Search results

From Obsolete Lustre Wiki
Jump to navigationJump to search

Page title matches

  • Architecture - Backup
    The architecture described here enables: The architecture decomposes into several pieces:
    4 KB (710 words) - 13:58, 22 January 2010
  • Architecture - Writing Architecture Documents
    This page provides some guidelines for preparing architecture documents. * SEI books - Documenting Software Architectures, Software Architecture in Practice
    6 KB (832 words) - 16:52, 20 January 2010
  • Architecture - HSM
    * ''Architecture.'' See [[Architecture - HSM and Cache]] ;Copy-in
    2 KB (369 words) - 14:12, 22 January 2010
  • Architecture - Security
    This chapter outlines the security architecture for Lustre. Special administrative accounts with un-usual
    52 KB (8,446 words) - 14:23, 22 January 2010
  • Architecture - IO system
    ...quite different because data can be modified on the client before it is re-played, while for metadata we replay operations, not state.
    6 KB (964 words) - 14:14, 22 January 2010
  • Architecture - GNS
    of such trees is that they are self-describing, i.e. they require little or no configuration information on the clients. We describe how global namespaces require mount-point objects stored
    29 KB (4,464 words) - 14:11, 22 January 2010
  • Architecture - CROW
    == CROW Architecture == ...on of just destroyed objects are properly recognized and objects are not re-created
    6 KB (925 words) - 14:01, 22 January 2010
  • Architecture - Changelogs
    ...olation, etc.) Will typically be converted into a '''feed''' for userspace-level usage. ...mer''' : an entity receiving the changelog or feed entries; may be a Lustre-internal consumer (replication) or an external consumer (database feed).
    16 KB (2,263 words) - 13:58, 22 January 2010
  • Architecture - Libcfs
    definitions which allow conditional compilation and build-time runtimes (e.g. catamount) elide all locking primitives. Multi-threaded
    4 KB (647 words) - 14:14, 22 January 2010
  • Architecture - Fileset
    ...eristics will be done a database that mirrors the Lustre file tree via a [[Architecture - Changelogs|ChangeLog]]. The files matching these criteria will be added |rowspan="6" writing-mode="vertical"|'''details'''
    12 KB (1,742 words) - 14:10, 22 January 2010
  • Architecture - PAG
    authentication, it can always succeed because it have access to on-disk 2) non-root access<br>
    4 KB (808 words) - 14:20, 22 January 2010
  • Architecture - Cuts
    355 bytes (53 words) - 14:02, 22 January 2010
  • Architecture - Epochs
    ...below, that is used to merge distributed state updates (both data and meta-data) in a redundant cluster configuration. Typical (in fact, motivating) re * [[Architecture - Write_Back_Cache|WBC]], where redundancy is between objects on servers, a
    11 KB (1,784 words) - 14:07, 22 January 2010
  • Architecture Descriptions
    The architecture descriptions listed below provide information about Lustre architecture and design and are intended to help users better understand the conceptual ...information that is incomplete or obsolete and may not reflect the current architecture, features and functionality of Lustre.''
    9 KB (1,200 words) - 12:09, 20 January 2011
  • Architecture - Lustre HLDs
    You can access archived Lustre high-level design (HLD) documents in PDF format from this page. You can also down U doc/HLD/0-conf-HLD.lyx
    47 KB (6,185 words) - 17:15, 26 February 2010
  • Architecture - Lustre DLDs
    You can access archived Lustre detail-level design (DLD) documents in PDF format from this page. You can also down U doc/DLD/CrayUnicosMPPort-dld.lyx
    24 KB (3,151 words) - 17:14, 26 February 2010
  • Architecture - HSM and Cache
    [[Image:HSM-deploy.jpg]] The architecture under consideration will target re-use of components.
    1 KB (210 words) - 14:12, 22 January 2010
  • Architecture - LRE Images
    5 KB (723 words) - 14:15, 22 January 2010
  • Architecture - Open by fid
    The architecture of open by fid decomposes into several pieces: ...c mechanisms with the MDS to get that file descriptor (based on existing re-open by fid code, used during recovery)
    2 KB (394 words) - 14:19, 22 January 2010
  • Architecture - Pools of targets
    ## [[Architecture - Wide Striping|Wide Striping]] can use this to stripe over a large number ## doing [[Architecture - Simple Space Balance Migration|Object Migration]] from a set OSTs to anot
    3 KB (470 words) - 19:48, 2 February 2010
  • Architecture - ZFS for Lustre
    The purpose of this page is to document architecture and requirements related to Lustre servers using the DMU. There are two p ...is an obstacle then two things can be done: one is use a secondary name-to-fid ZAP in a fork of the directory, the second is to add the fid to a dirent
    7 KB (1,243 words) - 14:26, 22 January 2010
  • Architecture - Wide Striping
    == Definitions (see fid-hld) == ; A pool: defines an un-ordered sets of OSTs and will be used to describe the striping in a manageab
    2 KB (338 words) - 14:26, 22 January 2010
  • Architecture - Migration (1)
    ...r, involving re-striping. This page contains considerations for migration architecture. ...uring migration this will cause the migrate to be aborted and the partially-migrated objects to be destroyed.
    2 KB (349 words) - 14:16, 22 January 2010
  • Architecture - HSM Migration
    This page describes use cases and high-level architecture for migrating files between Lustre and a HSM system. ; Copy tool : HSM-specific component of the '''mover'''. (May be the entire mover.)
    25 KB (3,874 words) - 14:12, 22 January 2010
  • Architecture - Scalable Pinger
    #. servers evict not-heard-from clients, in case the clients died. #. Disentangle pinger from recovery, and turn it into a simple rain-or-shine periodic RPC.
    3 KB (428 words) - 14:22, 22 January 2010
  • Architecture - MDS-on-DMU
    ...stack (as per CMD3 server architecture). osd implements transactions, data-objects, indices, local locking, object attributes, and extended attributes. ...erations (link, unlink, readdir), permission checks (i_mode and acls), fine-grained pdirops locking of directories, lov attributes.
    6 KB (758 words) - 14:16, 22 January 2010
  • Architecture - MPI LND
    240 bytes (40 words) - 14:17, 22 January 2010
  • Architecture - Clustered Metadata
    ...ffers clustered metadata servers. This section will give an outline of the architecture. ...uted file system design is to do such clustering by allowing clients to pre-compute the location of the correct services.
    20 KB (3,407 words) - 14:00, 22 January 2010
  • Architecture - CTDB with Lustre
    |'''CTDB''' || [http://ctdb.samba.org CTDB] is a cluster-based implementation of TDB. Samba servers with CTDB then could exchange the ...reading or writing || it's not secure with all data over CIFS left in plain-text.
    16 KB (2,220 words) - 13:14, 2 February 2010
  • Architecture - Profiling Tools for IO
    |rowspan="7" writing-mode="vertical"|'''details''' ...detail information. (percentage of !M rpc)/proc/fs/lustre/obdfilter/lustre-OST0001/brw_stats
    4 KB (663 words) - 14:20, 22 January 2010
  • Architecture - MPI IO and NetCDF
    ...Hierarchical Data Format (HDF5) is a set of software libraries and machine-independent standard for storing scientific data(metadata and array data) in ...endent data formats that support the creation, access, and sharing of array-oriented scientific data. Similar as HDF5, but do not support parallel I/O.
    6 KB (861 words) - 14:17, 22 January 2010
  • Architecture - DMU Zerocopy
    |rowspan="6" writing-mode="vertical"|'''details'''
    965 bytes (113 words) - 14:02, 22 January 2010
  • Architecture - ZFS TinyZAP
    ...ea || performance || TinyZAP must be flexible to allow storage in arbitrary-sized containers such as large dnode or ancillary dbuf, and not waste too mu |compatible-zfs || usability || integration should be done at the DMU level so that any
    6 KB (965 words) - 14:27, 22 January 2010
  • Architecture - Request Redirection
    ...t data cached by clients and dedicated servers, maintains persistent and in-code redirect information. ...istributed over clients or dedicated cache servers. Target OSTs maintain in-core information about clients and cache servers and data they are caching a
    11 KB (1,647 words) - 14:22, 22 January 2010
  • Architecture - Proxy Cache
    ...volatile caching proxy - i.e. a proxy that extends its cache onto local non-volatile memory (e.g. disk, flash). Both volatile (in-store) and non-volatile (flash, disk) cache may be
    2 KB (308 words) - 14:21, 22 January 2010
  • Architecture - Sub Tree Locks
    ...e things worse in highly contended workloads by resorting to current client-server locking protocol. ;'''Usability''': usability to other components ([[Architecture - Write_Back_Cache|WBC]], disconnected operations).
    20 KB (3,036 words) - 14:24, 22 January 2010
  • Architecture - OSS-on-DMU
    FID - cluster-wide ID of any Lustre object, including objects on MDS'es and OSS'es | dio || performance || reads/writes || fsfilt || reads/writes should be zero-copy to allow high throughput
    4 KB (623 words) - 14:20, 22 January 2010
  • Architecture - Flash Cache
    ...usability||clients see consistent file data in the face of concurrent read-write accesses to the master and proxy servers |rowspan="6" writing-mode="vertical"|'''details'''
    4 KB (705 words) - 14:10, 22 January 2010
  • Architecture - Space Manager
    ...esponsible for copying-out candidate files to the HSM when possible (see [[Architecture - HSM Migration|HSM Migration]]). === pre-migration ===
    5 KB (722 words) - 14:24, 22 January 2010
  • Architecture - Changelogs 1.6
    Intended as a short-term feature to facilitate efficient replication of large Lustre 1.6 filesys
    704 bytes (98 words) - 13:59, 22 January 2010
  • Architecture - Userspace Servers
    # put all platform dependent code into few components with well-defined API (in order to improve portability) ...recovery model (atomic updates, executed-once semantics, clients retain non-committed requests)
    8 KB (1,034 words) - 16:47, 3 February 2010
  • Architecture - DMU OSD
    [[Architecture - MDS-on-DMU|MDS-on-DMU]] [[Architecture - OSS-on-DMU|OSS-on-DMU]]
    860 bytes (119 words) - 14:02, 22 January 2010
  • Architecture - Caching OSS
    # follow same recovery model (atomic updates, executed-once semantics, replay changes from clients) | retain pages referred by non-committed write request
    13 KB (1,825 words) - 13:58, 22 January 2010
  • Architecture - Commit on Share
    Commit-on-Share is intended to allow better recoverability in enviroment where clients * per-object -- all changes to same object are considered dependent,
    9 KB (1,118 words) - 14:00, 22 January 2010
  • Architecture - FIDs on OST
    ...old objects are mapped into the reserved IDIF namespace as discussed in [[Architecture - Interoperability fids zfs|Interoperability fids zfs]], using the ost_idx ; '''fid''': cluster-wide unique identificator
    3 KB (382 words) - 14:09, 22 January 2010
  • Architecture - Metadata API
    mdc_intent_lock() is that lock() in pseudo-code. ; '''page''': where to store result ('''LINUX-SPECIFIC YET''')
    11 KB (1,588 words) - 14:16, 22 January 2010
  • Architecture - New Metadata API
    # support for few llite-like layer at same time (lustre, pnfs, etc) # all network-related things are extracted to MDC-MDT (intents, recovery)
    6 KB (919 words) - 14:18, 22 January 2010
  • Architecture - Simplified Interoperation
    ...recovered on reconnection. This completely eliminate some classes of inter-version recovery interactions and simplifies interoperation on server upgrad
    4 KB (563 words) - 14:24, 22 January 2010
  • Architecture - Llog over OSD
    # Re-implement llog API to use OSD device as backend device instead of fsfilt; ...ifferent parts of Lustre. This API should be suitable for the following use-cases:
    3 KB (507 words) - 14:14, 22 January 2010
  • Architecture - Client Cleanup
    |io:read-ahead||have to be encapsulated into llite layer. Current semantics is to be |io:read||how layers handle typical read call, including read-ahead.
    4 KB (654 words) - 14:00, 22 January 2010
  • Architecture - Recovery Failures
    ...r indicates it does not have the cached data anymore and possibly cannot re-create it. |rowspan="6" writing-mode="vertical"|'''details'''
    5 KB (785 words) - 14:22, 22 January 2010
  • Architecture - Migration (2)
    ...a moving within one cluster (one namespace) as well as to/from external non-Lustre storage servers. HSM, | conflicting requests abort in-progress migration
    8 KB (1,123 words) - 14:17, 22 January 2010
  • Architecture - Feature FS Replication
    # The solution should have a suitable architecture to synchronize flash caches or object replicas # The architecture of this solution will be suitable for (i) ldiskfs (ii) Lustre 1.8 (ldiskfs
    28 KB (4,289 words) - 11:40, 21 January 2010
  • Architecture - Punch and Extent Migration
    [[Architecture - Punch and Extent Migration Requirements|Punch and Extent Migration Requir
    444 bytes (60 words) - 14:21, 22 January 2010
  • Architecture - Multiple Interfaces For LNET
    ...rewall to connect to a server behind a firewall where the server has non-reachable nids (like 192.168.1.*) which might have a different meaning ne ...used, they shall be big enough to contains both linkset modifiers and IP-v6 addresses
    4 KB (668 words) - 14:18, 22 January 2010
  • Architecture - User Level Access
    ...This PID may be shared by several O/S processes which may in turn be multi-threaded. Processes sharing the same LNET PID may share ME, MD and EQ handl ; Thread safety : All exported LNET API calls are thread-safe.
    3 KB (423 words) - 14:25, 22 January 2010
  • Architecture - External File Locking
    ; '''flock lock''' : whole-file lock obtained through BSD [http://man.cx/flock(2) flock] interface. ; '''group lock''' : whole-file lock obtained through Lustre-specific '''LL_IOC_GROUP_LOCK''' ioctl().
    5 KB (683 words) - 14:09, 22 January 2010
  • Architecture - Wire Level Protocol
    ...s the Lustre nid of the receiver. The nid can be an IP address or Elan node-id. || This field is set to the final destination Lustre work id: IP addr or ...e packet when the packet is routed. || This field is set to the Lustre node-id from the packet originates.
    36 KB (5,757 words) - 14:26, 22 January 2010
  • Architecture - Lustre Logging API
    ...In this discussion, we speak of log objects and sometimes of llogs (lustre-logs). Through an object identifier or llog-log identifier -A directory of llogs which can lookup a name to get an id ca
    24 KB (3,889 words) - 14:15, 22 January 2010
  • Architecture - ZFS large dnodes
    ; '''znode''' : ZPL-format dnode, uses bonus buffer to store POSIX data for ZFS |rowspan="6" writing-mode="vertical"|'''details'''
    2 KB (382 words) - 14:27, 22 January 2010
  • Architecture - Network Request Scheduler
    ; '''RPC Concurrency''' : The number of RPC requests in-flight RPC between a given client and a server. ; '''Active fan-out''' : The number of clients with in-flight requests to a given server at a given time.
    5 KB (837 words) - 14:18, 22 January 2010
  • Architecture - Write Back Cache
    The meta-data write-back cache (WBC or MDWBC, where a possibility of misunderstanding exists) allows client meta-data operations to be
    24 KB (3,464 words) - 14:39, 27 July 2010
  • Architecture - Server Network Striping
    Server Network Striping (SNS) is Lustre-level striping of file data over multiple object servers with redundancy in '''Full Stripe''' A collection of stripe units in chunk objects that maps 1-1 to an extent in striped object. (In RAIDframe, ''stripe'' is short for ''f
    26 KB (3,768 words) - 14:23, 22 January 2010
  • Architecture - Free Space Management
    ...h utilization. The weighting factor between these two optimizations is user-adjustable. ...n 20%. Until then, a faster '''round-robin allocater''' is used. The round-robin order maximizes network balancing.
    3 KB (420 words) - 14:11, 22 January 2010
  • Architecture - MDS striping format
    This architecture page is available as a .pdf file at this link: [[Media:ManagingLustreDataSt ...emory (kernel mode inside Lustre), or in-application (striping EA in a user-level application). Each status corresponds to a different format.
    31 KB (4,814 words) - 14:15, 22 January 2010
  • Architecture - Interoperability fids zfs
    ...migration from 1.6-based, fidless Lustre clusters, using ldiskfs as a back-end file system, to clusters based on fids and zfs file system. ...ate release whose purpose is to effect transition from ldiskfs-based to DMU-based clusters.
    34 KB (5,038 words) - 00:19, 4 October 2012
  • Architecture - Version Based Recovery
    ; '''pre-version''': a version of object before change ; '''post-version''': a version of object after change
    17 KB (2,305 words) - 14:25, 22 January 2010
  • Architecture - Adaptive Timeouts - Use Cases
    ...t immediately by the server informing the client of a predicted slower-than-expected final response. |'''Service estimate'''||The expected worst-case time a request on a given portal to a given service will take. This va
    12 KB (1,843 words) - 13:57, 22 January 2010
  • Architecture - User Level OSS
    1,002 bytes (174 words) - 14:25, 22 January 2010
  • Architecture - Simple Space Balance Migration
    ...iety of reasons, such as moving objects from full OSTs to less-full or less-busy OSTs, or evacuating OSTs for removal/replacement. Simple space balance == Simple Space Balance Migration Architecture ==
    9 KB (1,467 words) - 14:23, 22 January 2010
  • Architecture - Punch and Extent Migration Requirements
    [[Architecture - Punch and Extent Migration|Punch and Extent Migration]]
    1 KB (205 words) - 14:21, 22 January 2010
  • Architecture - Interoperability 1.6 1.8 2.0
    | 1.6-fid ...mited mode is able to run on top of 1.6 storage, and interoperates with 1.6-fid clients.
    2 KB (315 words) - 14:13, 22 January 2010
  • Architecture - End-to-end Checksumming
    * Integrate the lustre network checksumming with the DMU on-disk checksum mechanism. The plan is to use a Tiger Tree Hash algorithm ([ht # probably more space to store a larger checksum value (only 32-bits today) - mechanism for reads to force a buffer out of cache if client d
    2 KB (397 words) - 14:03, 22 January 2010

Page text matches

  • Architecture - Backup
    The architecture described here enables: The architecture decomposes into several pieces:
    4 KB (710 words) - 13:58, 22 January 2010
  • Architecture Descriptions
    The architecture descriptions listed below provide information about Lustre architecture and design and are intended to help users better understand the conceptual ...information that is incomplete or obsolete and may not reflect the current architecture, features and functionality of Lustre.''
    9 KB (1,200 words) - 12:09, 20 January 2011
  • Lustre Design Document Archive
    * [[Architecture Descriptions]]. Offer Lustre feature and architectural descriptions along w * [[Architecture_-_Lustre_HLDs|Lustre High-Level Designs (HLDs)]]. Describe the overall design of Lustre features and p
    882 bytes (115 words) - 11:21, 22 March 2010
  • Lustre Internals
    == Lustre Architecture == The Lustre Architecture presentation includes the following topics:
    1 KB (157 words) - 11:22, 22 February 2010
  • Architecture - Fileset
    ...eristics will be done a database that mirrors the Lustre file tree via a [[Architecture - Changelogs|ChangeLog]]. The files matching these criteria will be added |rowspan="6" writing-mode="vertical"|'''details'''
    12 KB (1,742 words) - 14:10, 22 January 2010
  • Lustre User Group 2008
    [[Media:Lug-2008-Braam.pdf|Sun Storage Perspective & Lustre Architecture Slides]] and [http://video.google.com/videoplay?docid=-3180589066635872586& [[Media:Lustre-RoadmapUpdate-20080428.pdf|Lustre Roadmap Update Slides]] and [http://video.google.com/vid
    4 KB (571 words) - 10:44, 20 January 2011
  • Architecture - Migration (1)
    ...r, involving re-striping. This page contains considerations for migration architecture. ...uring migration this will cause the migrate to be aborted and the partially-migrated objects to be destroyed.
    2 KB (349 words) - 14:16, 22 January 2010
  • Architecture - Writing Architecture Documents
    This page provides some guidelines for preparing architecture documents. * SEI books - Documenting Software Architectures, Software Architecture in Practice
    6 KB (832 words) - 16:52, 20 January 2010
  • Architecture - DMU OSD
    [[Architecture - MDS-on-DMU|MDS-on-DMU]] [[Architecture - OSS-on-DMU|OSS-on-DMU]]
    860 bytes (119 words) - 14:02, 22 January 2010
  • Architecture - Punch and Extent Migration
    [[Architecture - Punch and Extent Migration Requirements|Punch and Extent Migration Requir
    444 bytes (60 words) - 14:21, 22 January 2010
  • Architecture - HSM
    * ''Architecture.'' See [[Architecture - HSM and Cache]] ;Copy-in
    2 KB (369 words) - 14:12, 22 January 2010
  • Lustre 1.8
    ...d, the RPC timeout values decrease, allowing faster client detection of non-responsive servers and faster attempts to reconnect to a server's failover p * [[Architecture - Adaptive_Timeouts_-_Use_Cases|Architecture page - Adaptive timeouts (use cases)]]
    8 KB (1,286 words) - 11:24, 20 January 2011
  • Architecture - OSS-on-DMU
    FID - cluster-wide ID of any Lustre object, including objects on MDS'es and OSS'es | dio || performance || reads/writes || fsfilt || reads/writes should be zero-copy to allow high throughput
    4 KB (623 words) - 14:20, 22 January 2010
  • Architecture - Feature FS Replication
    # The solution should have a suitable architecture to synchronize flash caches or object replicas # The architecture of this solution will be suitable for (i) ldiskfs (ii) Lustre 1.8 (ldiskfs
    28 KB (4,289 words) - 11:40, 21 January 2010
  • Architecture - Pools of targets
    ## [[Architecture - Wide Striping|Wide Striping]] can use this to stripe over a large number ## doing [[Architecture - Simple Space Balance Migration|Object Migration]] from a set OSTs to anot
    3 KB (470 words) - 19:48, 2 February 2010
  • Architecture - Punch and Extent Migration Requirements
    [[Architecture - Punch and Extent Migration|Punch and Extent Migration]]
    1 KB (205 words) - 14:21, 22 January 2010
  • FAQ - Release Testing and Upgrading
    ...tecture and design:'' Depending on how severe or invasive an update to the architecture description is, it may be written and reviewed by senior architects. A det ...sign and code review to ensure that any 1.x release will work with any 1.(x-2) release.
    3 KB (500 words) - 08:32, 22 February 2010
  • Preparing to Install Lustre
    ...s) on the [http://www.oracle.com/technetwork/indexes/downloads/sun-az-index-095901.html#L Lustre download site]. ...ssues and workarounds, see the [http://wiki.lustre.org/images/6/60/821-2077-10.pdf ''Lustre 2.0 Release Notes''].
    9 KB (1,347 words) - 10:17, 20 January 2011
  • Architecture - CROW
    == CROW Architecture == ...on of just destroyed objects are properly recognized and objects are not re-created
    6 KB (925 words) - 14:01, 22 January 2010
  • Architecture - HSM and Cache
    [[Image:HSM-deploy.jpg]] The architecture under consideration will target re-use of components.
    1 KB (210 words) - 14:12, 22 January 2010
  • Architecture - MDS-on-DMU
    ...stack (as per CMD3 server architecture). osd implements transactions, data-objects, indices, local locking, object attributes, and extended attributes. ...erations (link, unlink, readdir), permission checks (i_mode and acls), fine-grained pdirops locking of directories, lov attributes.
    6 KB (758 words) - 14:16, 22 January 2010
  • Architecture - Userspace Servers
    # put all platform dependent code into few components with well-defined API (in order to improve portability) ...recovery model (atomic updates, executed-once semantics, clients retain non-committed requests)
    8 KB (1,034 words) - 16:47, 3 February 2010
  • Lustre User Group 2007
    ...was held in stylish South Beach District of Miami Beach, Florida, April 23-25, 2007, at the beautiful [http://www.thepalmshotel.com Palms Hotel]. .... Attendees will have access to experts and peers who will share their real-world experiences. You will also be able to meet with Lustre management and
    3 KB (500 words) - 10:44, 20 January 2011
  • Contribute:Contribute
    ...re and architectural descriptions, high-level design documents and detailed-level design documents. ...ute or about any of the procedures on this page, please [mailto:lustre-wiki-feedback@sun.com let us know].
    2 KB (353 words) - 16:36, 24 July 2013
  • Architecture - ZFS for Lustre
    The purpose of this page is to document architecture and requirements related to Lustre servers using the DMU. There are two p ...is an obstacle then two things can be done: one is use a secondary name-to-fid ZAP in a fork of the directory, the second is to add the fid to a dirent
    7 KB (1,243 words) - 14:26, 22 January 2010
  • Lustre Design Documents
    *[[Media:FSCK_Design-2009-06-15-09.pdf|''Filesystem Integrity Check Design'']] *[[Media:End-to-End-Integrity-2009-06-15.pdf|''End to End Data Integrity Design'']]
    2 KB (257 words) - 11:06, 20 January 2011
  • Lustre 2.0
    !Architecture ...ghtly different procedure to upgrade to 2.0. See [[Lustre_Interoperability_-_Upgrading_From_1.8_to_2.0|Upgrading from 1.8 to 2.0]].
    2 KB (288 words) - 10:38, 24 August 2010
  • Lustre Publications
    ...its applications, including videos and podcasts describing Lustre at a high-level and more detailed white papers, blue prints and engineering presentati |[http://www.rce-cast.com/index.php/Podcast/rce-14-lustre-cluster-filesystem.html '''RCE 14: Lustre Cluster File System''']|| Research Comput
    11 KB (1,510 words) - 17:57, 18 December 2009
  • Architecture - Open by fid
    The architecture of open by fid decomposes into several pieces: ...c mechanisms with the MDS to get that file descriptor (based on existing re-open by fid code, used during recovery)
    2 KB (394 words) - 14:19, 22 January 2010
  • Architecture - ZFS large dnodes
    ; '''znode''' : ZPL-format dnode, uses bonus buffer to store POSIX data for ZFS |rowspan="6" writing-mode="vertical"|'''details'''
    2 KB (382 words) - 14:27, 22 January 2010
  • Architecture - Simple Space Balance Migration
    ...iety of reasons, such as moving objects from full OSTs to less-full or less-busy OSTs, or evacuating OSTs for removal/replacement. Simple space balance == Simple Space Balance Migration Architecture ==
    9 KB (1,467 words) - 14:23, 22 January 2010
  • Architecture - FIDs on OST
    ...old objects are mapped into the reserved IDIF namespace as discussed in [[Architecture - Interoperability fids zfs|Interoperability fids zfs]], using the ost_idx ; '''fid''': cluster-wide unique identificator
    3 KB (382 words) - 14:09, 22 January 2010
  • Architecture - Epochs
    ...below, that is used to merge distributed state updates (both data and meta-data) in a redundant cluster configuration. Typical (in fact, motivating) re * [[Architecture - Write_Back_Cache|WBC]], where redundancy is between objects on servers, a
    11 KB (1,784 words) - 14:07, 22 January 2010
  • Lustre User Group 2006
    The fourth annual Lustre User Group Spring meeting was held April 18-20, 2006 on Hilton Head Island, SC. ...lease send a short proposal to lustre2006-submissions@clusterfs.com. This e-mail address is being protected from spam bots, you need JavaScript enabled
    6 KB (925 words) - 10:45, 20 January 2011
  • POSIX Compliance Testing
    ...stre.org/public/tools/benchmarks/posix/ download]. Each version is gcc and architecture specific. You will need to determine which version of gcc you are running l If a package isn't available for your particular combination of gcc+architecture, see [[POSIX Compliance Testing#Building and Running a POSIX Compliance Tes
    11 KB (1,795 words) - 14:35, 23 March 2010
  • Architecture - Adaptive Timeouts - Use Cases
    ...t immediately by the server informing the client of a predicted slower-than-expected final response. |'''Service estimate'''||The expected worst-case time a request on a given portal to a given service will take. This va
    12 KB (1,843 words) - 13:57, 22 January 2010
  • Configuring RAID for Disk Arrays
    organizes disks into a RAID array. All Lustre-supported kernels have software RAID ...u are using software RAID functionality, we recommend that you use a Lustre-patched RHEL 4 or 5 kernel to take advantage of these performance improvemen
    9 KB (1,461 words) - 06:31, 20 January 2011
  • Architecture - HSM Migration
    This page describes use cases and high-level architecture for migrating files between Lustre and a HSM system. ; Copy tool : HSM-specific component of the '''mover'''. (May be the entire mover.)
    25 KB (3,874 words) - 14:12, 22 January 2010
  • Architecture - CTDB with Lustre
    |'''CTDB''' || [http://ctdb.samba.org CTDB] is a cluster-based implementation of TDB. Samba servers with CTDB then could exchange the ...reading or writing || it's not secure with all data over CIFS left in plain-text.
    16 KB (2,220 words) - 13:14, 2 February 2010
  • Architecture - Profiling Tools for IO
    |rowspan="7" writing-mode="vertical"|'''details''' ...detail information. (percentage of !M rpc)/proc/fs/lustre/obdfilter/lustre-OST0001/brw_stats
    4 KB (663 words) - 14:20, 22 January 2010
  • Architecture - Space Manager
    ...esponsible for copying-out candidate files to the HSM when possible (see [[Architecture - HSM Migration|HSM Migration]]). === pre-migration ===
    5 KB (722 words) - 14:24, 22 January 2010
  • Architecture - MDS striping format
    This architecture page is available as a .pdf file at this link: [[Media:ManagingLustreDataSt ...emory (kernel mode inside Lustre), or in-application (striping EA in a user-level application). Each status corresponds to a different format.
    31 KB (4,814 words) - 14:15, 22 January 2010
  • Architecture - ZFS TinyZAP
    ...ea || performance || TinyZAP must be flexible to allow storage in arbitrary-sized containers such as large dnode or ancillary dbuf, and not waste too mu |compatible-zfs || usability || integration should be done at the DMU level so that any
    6 KB (965 words) - 14:27, 22 January 2010
  • FAQ - Sizing
    ''' What is the maximum single-file size? ''' On 32-bit clients, the page cache makes it quite difficult to read or write a sing
    6 KB (1,004 words) - 08:32, 22 February 2010
  • Architecture - Interoperability fids zfs
    ...migration from 1.6-based, fidless Lustre clusters, using ldiskfs as a back-end file system, to clusters based on fids and zfs file system. ...ate release whose purpose is to effect transition from ldiskfs-based to DMU-based clusters.
    34 KB (5,038 words) - 00:19, 4 October 2012
  • Architecture - Write Back Cache
    The meta-data write-back cache (WBC or MDWBC, where a possibility of misunderstanding exists) allows client meta-data operations to be
    24 KB (3,464 words) - 14:39, 27 July 2010
  • Architecture - Request Redirection
    ...t data cached by clients and dedicated servers, maintains persistent and in-code redirect information. ...istributed over clients or dedicated cache servers. Target OSTs maintain in-core information about clients and cache servers and data they are caching a
    11 KB (1,647 words) - 14:22, 22 January 2010
  • Architecture - Server Network Striping
    Server Network Striping (SNS) is Lustre-level striping of file data over multiple object servers with redundancy in '''Full Stripe''' A collection of stripe units in chunk objects that maps 1-1 to an extent in striped object. (In RAIDframe, ''stripe'' is short for ''f
    26 KB (3,768 words) - 14:23, 22 January 2010
  • Building and Installing Lustre from Source Code
    ...le at the [http://www.oracle.com/technetwork/indexes/downloads/sun-az-index-095901.html#L Lustre download site] have been extensively tested and verifie '''''Note:''''' When using third-party network hardware with Lustre, the third-party
    11 KB (1,697 words) - 11:59, 20 January 2011
  • Architecture - Clustered Metadata
    ...ffers clustered metadata servers. This section will give an outline of the architecture. ...uted file system design is to do such clustering by allowing clients to pre-compute the location of the correct services.
    20 KB (3,407 words) - 14:00, 22 January 2010
  • FAQ - Fundamentals
    ...tored on Lustre Object Storage Servers (OSSs). Note that unlike many block-based clustered filesystems where the MDS is still in charge of block alloca ...r or administrator to choose other striping methods, such as RAID-1 or RAID-5 redundancy.
    9 KB (1,512 words) - 08:25, 22 February 2010
  • Architecture - Metadata API
    mdc_intent_lock() is that lock() in pseudo-code. ; '''page''': where to store result ('''LINUX-SPECIFIC YET''')
    11 KB (1,588 words) - 14:16, 22 January 2010
  • Architecture - Security
    This chapter outlines the security architecture for Lustre. Special administrative accounts with un-usual
    52 KB (8,446 words) - 14:23, 22 January 2010
  • Documenting Code
    ...is expected to have a good overall grasp of the [[Lustre_Internals|Lustre architecture and internals]]. The code documentation provides reference information on t ...ts can be processed by [http://www.doxygen.org doxygen] into online, browse-able (HTML) documentation.
    13 KB (1,965 words) - 06:56, 22 February 2010
  • Architecture - Sub Tree Locks
    ...e things worse in highly contended workloads by resorting to current client-server locking protocol. ;'''Usability''': usability to other components ([[Architecture - Write_Back_Cache|WBC]], disconnected operations).
    20 KB (3,036 words) - 14:24, 22 January 2010
View (previous 100 | ) (20 | 50 | 100 | 250 | 500)
Retrieved from "http://wiki.old.lustre.org/index.php?title=Special:Search"