WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.
User talk:Chuntera
We want to reconfigure our lustre for multi-homed network. However the mgs only appears to respond to clients using the original network (o2ib).
Here is the kerenel module (modprobe.conf) line > options lnet 'networks="o2ib(ib0),tcp(eth0)"'
When mounting lustre on the new (tcp) network, we see the following syslog messages: > kernel: LustreError: 10899:0:(events.c:454:ptlrpc_uuid_to_peer()) No NID found for 10.2.1.30@o2ib > kernel: LustreError: 10899:0:(client.c:58:ptlrpc_uuid_to_connection()) cannot find peer 10.2.1.30@o2 ib! > kernel: LustreError: 10899:0:(ldlm_lib.c:321:client_obd_setup()) can't add initial connection > kernel: LustreError: 8819:0:(connection.c:144:ptlrpc_put_connection()) NULL connection > kernel: LustreError: 10899:0:(obd_config.c:332:class_setup()) setup lustre0-MDT0000-mdc-ffff81022ea1 cc00 failed (-2) > kernel: LustreError: 10899:0:(obd_config.c:1071:class_config_llog_handler()) Err -2 on cfg command: > kernel: Lustre: cmd=cf003 0:lustre0-MDT0000-mdc 1:lustre0-MDT0000_UUID 2:10.2.1.30@o2ib > kernel: LustreError: 15c-8: MGC10.1.1.30@tcp: The configuration from log 'lustre0-client' failed (-2).
After viewing syslogs and searching mailing lists, it appears the mdt needs to be reformatted to include info about the new network (tcp) routing to the mgs.
We have a combined mgs+mdt. My best guess, the appropriate upgrade command on the mds is: tunefs.lustre --mgsnode="IP1@o2ib0,IP2@tcp0" dev_mdt
The original command to make the mdt did not include the mgsnode flag: tunefs --dryrun -l dev_mdt >Reading CONFIGS/mountdata > > Read previous values: >Target: lustre0-MDT0000 >Index: 0 >Lustre FS: lustre0 >Mount type: ldiskfs >Flags: 0x5 > (MDT MGS ) >Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr >Parameters: mdt.group_upcall=/usr/sbin/l_getgroups
Q. Is the tunefs.lustre command non-destructive ? ie. tunefs.lustre --mgsnode="IP1@o2ib0,IP2@tcp0" dev_mdt
Q. Do I need to update the config on the OSTs for our new multi-homed network ?
