WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.
Preparing to Install Lustre
To successfully install and run Lustre, make sure the following installation prerequisites have been met:
- Supported Operating System, Platform and Interconnect
- Required Lustre Software
- Required Tools and Utilities
- (Optional) High-Availability Software
- Environmental Requirements
- Memory Requirements
Required Lustre Software
To install Lustre, the following are required:
- Linux kernel patched with Lustre-specific patches (the patched Linux kernel is required only on Lustre MDSs and OSSs)
- Lustre modules compiled for the Linux kernel
- Lustre utilities required for Lustre configuration
- (Optional) Network-specific kernel modules and libraries (for example, kernel modules and libraries required for an InfiniBand interconnect)
These packages can be downloaded from the Lustre download site.
Required Tools and Utilities
Several third-party utilities are required:
- e2fsprogs - Lustre requires a recent version of e2fsprogs that understands extents.Use e2fsprogs-1.41-6 or later, available with the Lustre file downloads.
- Note: Lustre-patched e2fsprogs utility only needs to be installed on machines that mount backend (ldiskfs) file systems, such as the OSS, MDS and MGS nodes. It does
not need to be loaded on clients.
- Perl - Various userspace utilities are written in Perl. Any recent version of Perl
will work with Lustre.
(Optional) High-Availability Software
If you plan to enable failover server functionality with Lustre (either on an OSS or an MDS), you must add high-availability (HA) software to your cluster software. For more information, see the Failover chapter in the Lustre Operations Manual.
Environmental Requirements
Make sure the following environmental requirements are met before installing Lustre:
- (Recommended) Provide remote shell access to clients. Although not strictly required to run Lustre, we recommend that all cluster nodes have remote shell client access, to facilitate the use of Lustre configuration and monitoring scripts. Parallel Distributed SHell (pdsh) is preferable, although Secure SHell (SSH) is acceptable. For more information, see pdsh.
- Ensure client clocks are synchronized. Lustre uses client clocks for timestamps. If clocks are out-of-sync between clients and servers, timeouts and client evictions will occur. Drifting clocks can also cause problems by, for example, making it difficult to debug multi-node issues or correlate logs, which depend on timestamps. We recommend that you use Network Time Protocol (NTP) to keep client and server clocks in sync with each other. For more information about NTP, see: [1].
- Maintain uniform file access permissions on all cluster nodes. Use the same user IDs (UID) and group IDs (GID) on all clients. If use of supplemental groups is required, verify that the group_upcall requirements have been met. See the User/Group Cache Upcall section in the Lustre Operations Manual.
- (Recommended) Disable Security-Enhanced Linux (SELinux) on servers and clients. Lustre does not support SELinux. Therefore, disable the SELinux system extension on all Lustre nodes and make sure other security extensions, like Novell AppArmorand network packet filtering tools (such as iptables) do not interfere with Lustre.
Memory Requirements
This section describes the memory requirements for Lustre.
MDS Memory Requirements
MDS memory requirements are determined by the following factors:
- Number of clients
- Size of the directories
- Extent of load
For example, for a single MDT on an MDS with 1,000 clients, 16 interactive nodes, and a 2 million file working set (of which 400,000 files are cached on the clients), memory requirements include:
- File system journal = 400 MB
- 1000 * 4-core clients * 100 files/core * 2kB = 800 MB
- 16 interactive clients * 10,000 files * 2kB = 320 MB
- 1,600,000 file extra working set * 1.5kB/file = 2400 MB
Thus, the minimum requirement for a system with this configuration is 4-GB RAM. However, additional memory may significantly improve performance. For information about determining MDS memory requirements for more complex systems, see Memory Requirements in the Lustre Operations Manual.
