Creating and Managing OST Pools

(Updated: Jan 2010)

The OST pool feature enables users to group OSTs together to make object placement more flexible. A "pool" is a name associated with an arbitrary subset of OSTs in a Lustre™ cluster.

OST pools follow these rules:

• An OST can be a member of multiple pools.
• No ordering of OSTs in a pool is defined or implied.
• Stripe allocation within a pool follows the same rules as the normal stripe allocator
• OST membership in a pool is flexible and can change over time.

When an OST pool is defined, it can be used to allocate files. When file or directory striping is set to a pool, only OSTs in the pool are candidates for striping. If a stripe_index is specified that refers to an OST that is not a member of the pool, an error is returned.

OST pools are used only at file creation. If the definition of a pool changes (an OST is added or removed or the pool is destroyed), already-created files are not affected.

Note: An error (EINVAL) results if you create a file using an empty pool.

Note: Files created in a pool are not accessible from clients or servers running Lustre 1.6.5 or earlier (an error will be reported to the client). We recommend one of the following options:

• Use Lustre 1.6.6 or later prior to upgrading (to have a downgrade path available)
• Use Lustre 1.8 without using OST pools until there is no longer concern about downgrading to 1.6.5 or earlier.

Working with OST Pools

OST pools are defined in the configuration log on the MGS. Use the lctl command to:

• Create/destroy a pool
• Add/remove OSTs in a pool
• List pools and OSTs in a specific pool

The lctl command MUST be run on the MGS. Another requirements for managing OST pools is to either have the MDT and MGS on the same node or have a Lustre client mounted on the MGS node if it is separate from the MDS. This is needed to validate the pool commands being run are correct.

Caution: Running the writeconf command on the MDS will erase all pool information (as well as any other parameters set using lctl conf_param). We recommend that the pool definitions (and conf_param settings) be executed using a script, so they can be reproduced easily after a writeconf is performed.

To create a new pool, enter:

lctl pool_new <fsname>.<poolname>

Note: The pool name is an ASCII string up to 16 characters.

To add the named OST to a pool, enter:

lctl pool_add <fsname>.<poolname> <ost_list>

Where:

• <ost_list> is <fsname->OST<index_range>[_UUID]
• <index_range> is <ost_index_start>-<ost_index_end>[,<index_range>] or <ost_index_start>-<ost_index_end>/<step>

If the leading <fsname> and/or ending _UUID are missing, they are automatically added.

For example, to add the even-numbered OSTs to pool1 on file system lustre, run a single command (add) to add many OSTs to the pool at one time:

lctl pool_add lustre.pool1 OST[0-10/2]

Note: Each time an OST is added to a pool, a new llog configuration record is created. For convenience, you can run a single command.

To remove a named OST from a pool, enter:

lctl pool_remove <fsname>.<poolname> <ost_list>

To destroy a pool, enter:

lctl pool_destroy <fsname>.<poolname>

Note: All OSTs must be removed from a pool before the pool can be destroyed.

To list pools in the named file system, enter:

lctl pool_list <fsname> | <pathname>

To list OSTs in a named pool, enter:

lctl pool_list <fsname>.<poolname>

Using the lfs Command with OST Pools

Several lfs commands can be run with OST pools. Use the lfs setstripe command to associate a directory with an OST pool. This causes all new regular files and directories in the directory to be created in the pool. The lfs command can be used to list pools in a file system and OSTs in a named pool.

To associate a directory with a pool, so all new files and directories will be created in the pool, enter:

lfs setstripe <filename|dirname> --pool|-p pool-name

To set striping patterns, enter:

lfs setstripe [--size|-s stripe_size] [--offset|-o start_ost]
[--count|-c stripe_count] [--pool|-p pool_name]
<dir|filename>

Note: If you specify striping with an invalid pool name, because the pool does not exist or the pool name was mistyped, lfs setstripe returns an error. Run lfs pool_list to make sure the pool exists and the pool name is entered correctly.

Note: The --pool option for lfs setstripe is compatible with other modifiers. For example, you can set striping on a directory to use an explicit starting index.

Tips for Using OST Pools

Here are several suggestions for using OST pools.

• A directory or file can be given an extended attribute (EA), that restricts striping to a pool.
• Pools can be used to group OSTs with the same technology or performance (slower or faster) or that are preferred for certain jobs. Examples are SATA OSTs versus SAS OSTs or remote OSTs versus local OSTs.
• A file created in an OST pool tracks the pool by keeping the pool name in the file LOV EA.