WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.

Configuring Lustre File Striping: Difference between revisions

From Obsolete Lustre Wiki
Jump to navigationJump to search
No edit summary
No edit summary
Line 12: Line 12:
-----
-----


Using stripes
== Using stripes ==
One of the main reasons for Lustre file system's performance is the striping of data blocks over multiple OSTs. The stripe count can be set on a file system, directory or file level.
One of the main reasons for Lustre file system's performance is the striping of data blocks over multiple OSTs. The stripe count can be set on a file system, directory or file level.
To see the current stripe size, use the command lfs getstripe [file, dir, fs]. On the current system this will produce the following output:
 
To see the current stripe size, use the command ''lfs getstripe [file, dir, fs]''. On the current system this will produce the following output:
 
<pre>
root@LustreClient01 lustre]# lfs getstripe /mnt/lustre
OBDS:
0: lustre-OST0000_UUID ACTIVE
1: lustre-OST0001_UUID ACTIVE
2: lustre-OST0002_UUID ACTIVE
3: lustre-OST0003_UUID ACTIVE
4: lustre-OST0004_UUID ACTIVE
5: lustre-OST0005_UUID ACTIVE
/mnt/lustre
(Default) stripe_count: 2 stripe_size: 4M stripe_offset: 0
</pre>
 
As can be seen, the default stripe count is 2 (that is, striping over two OSTs), default stripe size is 4 MB (can be set in K, M or G), and all writes start from the first OST.
As can be seen, the default stripe count is 2 (that is, striping over two OSTs), default stripe size is 4 MB (can be set in K, M or G), and all writes start from the first OST.
Note – When setting the stripe, the offset is set before the stripe count.
 
'''''Note:''''' –When setting the stripe, the offset is set before the stripe count.
 
Setting a new stripe pattern on the file system can look like this:
Setting a new stripe pattern on the file system can look like this:
This example sets the stripe of /mnt/lustre to 4 MB blocks starting at OST0 and spanning over one OST. If a new file is created with these settings, the following results are seen:
 
[root@LustreClient01 lustre]# lfs setstripe /mnt/lustre 4M 0 1
 
This example sets the stripe of ''/mnt/lustre'' to 4 MB blocks starting at OST0 and spanning over one OST. If a new file is created with these settings, the following results are seen:
 
<pre>
[root@LustreClient01 lustre]# dd if=/dev/zero of=/mnt/lustre/test1 bs=10M count=100
root@LustreClient01 lustre]# lfs df -h
UUID bytes Used Available Use% Mounted on
lustre-MDT0000_UUID 4.4G 214.5M 3.9G 4% /mnt/lustre[MDT:0]
lustre-OST0000_UUID 2.0G 1.1G 830.1M 53% /mnt/lustre[OST:0]
lustre-OST0001_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:1]
lustre-OST0002_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:2]
lustre-OST0003_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:3]
lustre-OST0004_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:4]
lustre-OST0005_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:5]
filesystem summary: 11.8G 1.5G 9.7G 12% /mnt/lustre
</pre>
 
As can be seen, the entire file was written to the first OST, and there is a very uneven distribution of data blocks.
As can be seen, the entire file was written to the first OST, and there is a very uneven distribution of data blocks.

Revision as of 16:23, 7 October 2009

Have asked Torben to suggest subtopics for wiki

Intro...Configure striping to optimize performance ...

http://wiki.lustre.org/index.php/Striping_Guidelines

Cl400_OH - Slides 133-114

Page 13 in Torben's paper

OM Chapter 25 - Striping and IO Options

Using stripes

One of the main reasons for Lustre file system's performance is the striping of data blocks over multiple OSTs. The stripe count can be set on a file system, directory or file level.

To see the current stripe size, use the command lfs getstripe [file, dir, fs]. On the current system this will produce the following output: 

root@LustreClient01 lustre]# lfs getstripe /mnt/lustre
OBDS:
0: lustre-OST0000_UUID ACTIVE
1: lustre-OST0001_UUID ACTIVE
2: lustre-OST0002_UUID ACTIVE
3: lustre-OST0003_UUID ACTIVE
4: lustre-OST0004_UUID ACTIVE
5: lustre-OST0005_UUID ACTIVE
/mnt/lustre
(Default) stripe_count: 2 stripe_size: 4M stripe_offset: 0

As can be seen, the default stripe count is 2 (that is, striping over two OSTs), default stripe size is 4 MB (can be set in K, M or G), and all writes start from the first OST.

Note: –When setting the stripe, the offset is set before the stripe count.

Setting a new stripe pattern on the file system can look like this: 

[root@LustreClient01 lustre]# lfs setstripe /mnt/lustre 4M 0 1

This example sets the stripe of /mnt/lustre to 4 MB blocks starting at OST0 and spanning over one OST. If a new file is created with these settings, the following results are seen: 

[root@LustreClient01 lustre]# dd if=/dev/zero of=/mnt/lustre/test1 bs=10M count=100
root@LustreClient01 lustre]# lfs df -h
UUID bytes Used Available Use% Mounted on
lustre-MDT0000_UUID 4.4G 214.5M 3.9G 4% /mnt/lustre[MDT:0]
lustre-OST0000_UUID 2.0G 1.1G 830.1M 53% /mnt/lustre[OST:0]
lustre-OST0001_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:1]
lustre-OST0002_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:2]
lustre-OST0003_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:3]
lustre-OST0004_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:4]
lustre-OST0005_UUID 2.0G 83.3M 1.8G 4% /mnt/lustre[OST:5]
filesystem summary: 11.8G 1.5G 9.7G 12% /mnt/lustre

As can be seen, the entire file was written to the first OST, and there is a very uneven distribution of data blocks.

Retrieved from "http://wiki.old.lustre.org/index.php?title=Configuring_Lustre_File_Striping&oldid=7934"