WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.

Configuring Lustre File Striping

From Obsolete Lustre Wiki
Revision as of 16:23, 7 October 2009 by Docadmin (talk | contribs)
Jump to navigationJump to search

Have asked Torben to suggest subtopics for wiki

Intro...Configure striping to optimize performance ...

http://wiki.lustre.org/index.php/Striping_Guidelines

Cl400_OH - Slides 133-114

Page 13 in Torben's paper

OM Chapter 25 - Striping and IO Options


Using stripes

One of the main reasons for Lustre file system's performance is the striping of data blocks over multiple OSTs. The stripe count can be set on a file system, directory or file level.

To see the current stripe size, use the command lfs getstripe [file, dir, fs]. On the current system this will produce the following output:

root@LustreClient01 lustre]# lfs getstripe /mnt/lustre
OBDS:
0: lustre-OST0000_UUID ACTIVE
1: lustre-OST0001_UUID ACTIVE
2: lustre-OST0002_UUID ACTIVE
3: lustre-OST0003_UUID ACTIVE
4: lustre-OST0004_UUID ACTIVE
5: lustre-OST0005_UUID ACTIVE
/mnt/lustre
(Default) stripe_count: 2 stripe_size: 4M stripe_offset: 0

As can be seen, the default stripe count is 2 (that is, striping over two OSTs), default stripe size is 4 MB (can be set in K, M or G), and all writes start from the first OST.

Note: –When setting the stripe, the offset is set before the stripe count.

Setting a new stripe pattern on the file system can look like this:

[root@LustreClient01 lustre]# lfs setstripe /mnt/lustre 4M 0 1

This example sets the stripe of /mnt/lustre to 4 MB blocks starting at OST0 and spanning over one OST. If a new file is created with these settings, the following results are seen:

[root@LustreClient01 lustre]# dd if=/dev/zero of=/mnt/lustre/test1 bs=10M count=100
root@LustreClient01 lustre]# lfs df -h
UUID bytes Used Available Use% Mounted on
lustre-MDT0000_UUID 4.4G 214.5M 3.9G 4% /mnt/lustre[MDT:0]
lustre-OST0000_UUID 2.0G 1.1G 830.1M 53% /mnt/lustre[OST:0]
lustre-OST0001_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:1]
lustre-OST0002_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:2]
lustre-OST0003_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:3]
lustre-OST0004_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:4]
lustre-OST0005_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:5]
filesystem summary: 11.8G 1.5G 9.7G 12% /mnt/lustre

As can be seen, the entire file was written to the first OST, and there is a very uneven distribution of data blocks.