WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.
Load Balancing with Infiniband: Difference between revisions
(Created page with 'Load Balancing with InfiniBand There is one OSS with two InfiniBand HCAs. Lustre clients have only one InfiniBand HCA using native Lustre drivers of o2ibind. Load balancing is do...') |
No edit summary |
||
| Line 1: | Line 1: | ||
There is one OSS with two InfiniBand HCAs. Lustre clients have only one InfiniBand | There is one OSS with two InfiniBand HCAs. Lustre clients have only one InfiniBand | ||
HCA using native Lustre drivers of o2ibind. Load balancing is done on both HCAs | HCA using native Lustre drivers of o2ibind. Load balancing is done on both HCAs | ||
on the OSS with the help of LNET. | on the OSS with the help of LNET. | ||
Options under modprobe.conf are used to specify the networks available to a | |||
node. You have the choice of two different options – the networks option, which | |||
explicitly lists the networks available and the ip2nets option, which provides a listmatching | |||
lookup. Only one option can be used at any one time. The order of LNET | |||
lines in modprobe.conf is important when configuring multi-homed servers. If a | |||
server node can be reached using more than one network, the first network specified | |||
in modprobe.conf will be used. | |||
7.3.1 Modprobe.conf | 7.3.1 Modprobe.conf | ||
Lustre users have options available on following networks. | Lustre users have options available on following networks. | ||
| Line 11: | Line 20: | ||
■ Client with the even IP address | ■ Client with the even IP address | ||
options lnet ip2nets=o2ib1(ib0) 192.168.10.[102-254/2] | options lnet ip2nets=o2ib1(ib0) 192.168.10.[102-254/2] | ||
7.3.2 Start servers | 7.3.2 Start servers | ||
To start the MGS and MDT server, run: | To start the MGS and MDT server, run: | ||
modprobe lnet | modprobe lnet | ||
| Line 19: | Line 30: | ||
$ mount -t lustre /dev/sda /mnt/test/mdt | $ mount -t lustre /dev/sda /mnt/test/mdt | ||
$ mount -t lustre mgs@o2ib0:/lustre /mnt/mdt | $ mount -t lustre mgs@o2ib0:/lustre /mnt/mdt | ||
To start the OSS, run: | To start the OSS, run: | ||
$ mkfs.lustre --fsname lustre --ost --mgsnode=mds@o2ib0 /dev/sda | $ mkfs.lustre --fsname lustre --ost --mgsnode=mds@o2ib0 /dev/sda | ||
| Line 24: | Line 36: | ||
$ mount -t lustre /dev/sda /mnt/test/ost | $ mount -t lustre /dev/sda /mnt/test/ost | ||
$ mount -t lustre mgs@o2ib0:/lustre /mnt/ost | $ mount -t lustre mgs@o2ib0:/lustre /mnt/ost | ||
7.3.3 Start clients | 7.3.3 Start clients | ||
For the IB client, run: | For the IB client, run: | ||
mount -t lustre | mount -t lustre | ||
192.168.10.101@o2ib0,192.168.10.102@o2ib1:/mds/client /mnt/lustre | 192.168.10.101@o2ib0,192.168.10.102@o2ib1:/mds/client /mnt/lustre | ||
For more information about othe LNET options, see 31.2.1 LNET OPtions in the Lustre Operations Manual | |||
For information about "How to build and configure Infiniband support for Lustre" the topic by thi =s nathe Lustre Knowledge Base in Appendix B in the Lustre Operations Manual. | |||
Revision as of 15:57, 30 September 2009
There is one OSS with two InfiniBand HCAs. Lustre clients have only one InfiniBand HCA using native Lustre drivers of o2ibind. Load balancing is done on both HCAs on the OSS with the help of LNET.
Options under modprobe.conf are used to specify the networks available to a
node. You have the choice of two different options – the networks option, which
explicitly lists the networks available and the ip2nets option, which provides a listmatching
lookup. Only one option can be used at any one time. The order of LNET
lines in modprobe.conf is important when configuring multi-homed servers. If a
server node can be reached using more than one network, the first network specified
in modprobe.conf will be used.
7.3.1 Modprobe.conf Lustre users have options available on following networks. ■ Dual HCA OSS server options lnet ip2nets= "o2ib0(ib0),o2ib1(ib1) 192.168.10.1.[101-102] ■ Client with the odd IP address options lnet ip2nets=o2ib0(ib0) 192.168.10.[103-253/2] ■ Client with the even IP address options lnet ip2nets=o2ib1(ib0) 192.168.10.[102-254/2]
7.3.2 Start servers
To start the MGS and MDT server, run: modprobe lnet To start MGS and MDT, run: $ mkfs.lustre --fsname lustre --mdt --mgs /dev/sda $ mkdir -p /mnt/test/mdt $ mount -t lustre /dev/sda /mnt/test/mdt $ mount -t lustre mgs@o2ib0:/lustre /mnt/mdt
To start the OSS, run: $ mkfs.lustre --fsname lustre --ost --mgsnode=mds@o2ib0 /dev/sda $ mkdir -p /mnt/test/mdt $ mount -t lustre /dev/sda /mnt/test/ost $ mount -t lustre mgs@o2ib0:/lustre /mnt/ost
7.3.3 Start clients
For the IB client, run: mount -t lustre 192.168.10.101@o2ib0,192.168.10.102@o2ib1:/mds/client /mnt/lustre
For more information about othe LNET options, see 31.2.1 LNET OPtions in the Lustre Operations Manual
For information about "How to build and configure Infiniband support for Lustre" the topic by thi =s nathe Lustre Knowledge Base in Appendix B in the Lustre Operations Manual.
