WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.
Upgrading to a New Version of Lustre: Difference between revisions
Line 126: | Line 126: | ||
There are two options: | There are two options: | ||
* Rolling upgrade keeps a copy of the original configuration log, allowing | * Rolling upgrade keeps a copy of the original configuration log, allowing immediate reintegration into a live file system, but prevents OSC parameter and failover NID changes. (The writeconf procedure can be performed later to eliminate these restrictions. For details, see [[Running the Writeconf Command]].) | ||
immediate reintegration into a live file system, but prevents OSC parameter | |||
and failover NID changes. (The writeconf procedure can be performed later to | |||
eliminate these restrictions. For details, see Running the Writeconf Command.) | |||
mdt1# tunefs.lustre --mgs --mdt --fsname=testfs /dev/sda1 | mdt1# tunefs.lustre --mgs --mdt --fsname=testfs /dev/sda1 | ||
* i.--writeconf begins a new configuration log, allowing permanent modification | * ''i.--writeconf'' begins a new configuration log, allowing permanent modification of all parameters (see Changing Parameters), but requiring all other servers and clients to be stopped at this point. No clients can be started until all OSTs are upgraded. | ||
of all parameters (see Changing Parameters), but requiring all other servers | |||
and clients to be stopped at this point. No clients can be started until all OSTs | |||
are upgraded. | |||
<pre> | <pre> | ||
Line 180: | Line 174: | ||
</pre> | </pre> | ||
4. Upgrade and start the OSTs for the file system in a similar manner, except they | 4. Upgrade and start the OSTs for the file system in a similar manner, except they need the address of the MGS. Old installations may also need to specify the OST index (for instance, ''--index=5''). | ||
need the address of the MGS. Old installations may also need to specify the | |||
OST index (for instance, --index=5). | |||
<pre> | <pre> | ||
Line 220: | Line 212: | ||
/dev/sda1 101086 14964 80903 16% /boot | /dev/sda1 101086 14964 80903 16% /boot | ||
none 501000 0 501000 0% /dev/shm | none 501000 0 501000 0% /dev/shm | ||
/dev/sda4 101492248 471672 95781780 1% /mnt/test/ | /dev/sda4 101492248 471672 95781780 1% /mnt/test/ost | ||
</pre> | </pre> | ||
Revision as of 16:41, 29 September 2009
This section provides information about supported upgrades, upgrade paths, and interoperability of nodes with different versions of Lustre installed. It also describes procedures for upgrading your Lustre file system to a new version of Lustre. For information about available releases, see Lustre Release Information.
Supported Upgrades
For Lustre 1.8.x, the following upgrades are supported:
- Lustre 1.6.x to Lustre 1.8.0.
- One minor version to the next (for example, 1.8.0 > 1.8.x). [[Should this say 1.8.0 > 1.8.1?]] Or is it OK to skip over a minor release?
For Lustre 1.8.0, if you have upgraded from an earlier version, you can downgrade to that version or any version in between?. For example:
- If you upgrade from Lustre 1.6.x > 1.8.0, you can downgrade to version 1.6.x. Can 1.6.x be any version of 1.6?
- If you upgrade from one minor version to the next (for example Lustre 1.8.0 > 1.8.x), you can downgrade to an earlier minor version (e.g., version 1.8.x). Should this say ...? If you upgrade from one minor version to the next (for example, Lustre 1.8.0.1 > 1.8.1), you can downgrade to an earlier minor version (e.g., version 1.8.0.1 or 1.8.0).
To downgrade from 1.8.x to 1.6.x, see More Information.
Caution: A new installation of Lustre 1.8.x is not guaranteed to be downgradeable to an earlier Lustre version Does this mean you may not be able to downgrade to 1.8.0 or does it mean you may not be able to downgrade to a 1.6.x version?).
Note: If you are upgrading from Lustre 1.4.6 or later to Lustre 1.6, see Upgrading from 1.4.6 and later to 1.6
Supported Upgrade Paths
Two Lustre upgrade paths are supported. Either can be applied to the entire file system or a combination of the two can be used to upgrade individual servers and clients.
- Rolling upgrade - Individual servers (or their failover partners) and clients are upgraded one at a time and restarted, so that the file system never goes down. This type of upgrade limits the ability to change certain parameters.
- Entire file system upgrade - All servers and clients are shut down and upgraded at the same time.
Interoperability between Nodes with Different Versions of Lustre Installed
Interoperability between clients, OSTs, and MDTs with different versions of Lustre installed is described below.
Does "old" mean a node with any older version of Lustre installed or just 1.6.x?
Does "new" mean specifically 1.8.x??
Clients
- Old live clients can continue to communicate with old/new/mixed servers.
- Old clients can start up using old/new/mixed servers.
- New clients can start up using old/new/mixed servers (use old mount format for
old MDT).
OSTs
- New clients/MDTs can continue to communicate with old OSTs.
- New OSTs can only be started after the MGS has been started (typically this
means "after the MDT has been upgraded.")
MDTs
- New clients can communicate with old MDTs.
- New co-located MGS/MDTs can be started at any point.
- New non-MGS MDTs can be started after the MGS starts.
Interoperability limitations:
- Old clients cannot mount a file system which was created by a new MDT.
- If your file system was originally created using Lustre 1.8.x, you will not be able to mount the (a?) file system created using an earlier Lustre version. However, if your system has been upgraded from 1.6.x to 1.8.x, you can mount Lustre clients on both Lustre versions. Lustre 1.8.x clients are interoperable with 1.6.x servers.
Find a place for this note
Note: In Lustre version 1.6 and later, the file system name (--fsname parameter) is limited to 8 characters so that it fits on the disk label.
Prerequisites to Upgrading Lustre
This content was in the OM section "Upgrading from Lustre 1.6.x to Lustre 1.8.0"
What are these procedures? Do they need to be completed before the upgrade is started?
Remember the following points before upgrading Lustre.
The MDT must be upgraded before the OSTs are upgraded. summarize steps in a sentence
1. Shut down lconf failover.
2. Install the new modules.
3. Run tunefs.lustre.
4. Mount startup.
A Lustre upgrade can be done across a failover pair. - In a failover pair, do the backup server first
1. On the backup server, install the new modules.
2. Shut down lconf failover.
3. On the new server, run tunefs.lustre.
4. On the new server, mount startup.
5. On the primary server, install the new modules.
Upgrade procedures
Upgrading Clients
- Starting Clients - if upgraded client only would need to know how to start a new client with an old MDT - says use old command - relevant depending on which machines you start with - if you start with clients you need to know this information. Need instructions on upgrading a clientsee initial client install ch 3 - intermingled - discuss with Sheila - something missing
You can start a new client with an old MDT by using the old format of the client mount command:
client# mount -t lustre <mdtnid>:/<mdtname>/client <mountpoint>
You can start a new client with an upgraded MDT by using the new format and pointing it at the MGS, not the MDT (for co-located MDT/MGS, this is the same):
client# mount -t lustre <mgsnid>:/<fsname> <mountpoint>
Old clients always use the old format of the mount command, regardless of whether the MDT has been upgraded or not.
Upgrading a Single File System
tunefs.lustre will find the old client log on an 1.4.x MDT that is being upgraded to 1.6. (If the name of the client log is not "client", use the lustre_up14.sh script, described in Step 2 and Step 3.)
1. Shut down the MDT.
mdt1# lconf --failover --cleanup config.xml
2. Install the new Lustre version and run tunefs.lustre to upgrade the configuration.
There are two options:
- Rolling upgrade keeps a copy of the original configuration log, allowing immediate reintegration into a live file system, but prevents OSC parameter and failover NID changes. (The writeconf procedure can be performed later to eliminate these restrictions. For details, see Running the Writeconf Command.)
mdt1# tunefs.lustre --mgs --mdt --fsname=testfs /dev/sda1
- i.--writeconf begins a new configuration log, allowing permanent modification of all parameters (see Changing Parameters), but requiring all other servers and clients to be stopped at this point. No clients can be started until all OSTs are upgraded.
[root@mds1]# tunefs.lustre --mgs --writeconf --mgs --mdt -- fsname=ldiskfs /dev/hda4 checking for existing Lustre data: found CONFIGS/mountdata Reading CONFIGS/mountdata Read previous values: Target: testfs-MDT0000 Index: 0 UUID: mds-1_UUID Lustre FS: testfs Mount type: ldiskfs Flags: 0x205 (MDT MGS upgrade1.4 ) Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr Parameters: Permanent disk data: Target: ldiskfs-MDT0000 Index: 0 UUID: mds-1_UUID Lustre FS: ldiskfs Mount type: ldiskfs Flags: 0x305 (MDT MGS writeconf upgrade1.4 ) Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr Parameters: Writing CONFIGS/mountdata Copying old logs
3. Start the upgraded MDT.
mdt1# mkdir -p /mnt/test/mdt mdt1# mount -t lustre /dev/hda4 /mnt/test/mdt mdt1 # df Filesystem 1K-blocks Used Available Use% Mounted on /dev/hda2 10080520 4600820 4967632 49% / /dev/hda1 101086 14787 81080 16% /boot none 501000 0 501000 0% /dev/shm /dev/hda4 23339176 455236 21550144 3% /mnt/test/mdt
4. Upgrade and start the OSTs for the file system in a similar manner, except they need the address of the MGS. Old installations may also need to specify the OST index (for instance, --index=5).
ost1# tunefs.lustre --ost --fsname=lustre --mgsnode=mds /dev/sda4 checking for existing Lustre data: found last_rcvd tunefs.lustre: Unable to read /tmp/dirQi2cwV/mountdata (No such file or directory.) Trying last_rcvd Reading last_rcvd Feature compat=2, incompat=0 Read previous values: Target: Index: 0 UUID: ost1_UUID Lustre FS: lustre Mount type: ldiskfs Flags: 0x202 (OST upgrade1.4 ) Persistent mount opts: Parameters: Permanent disk data: Target: lustre-OST0000 Index: 0 UUID: ost1_UUID Lustre FS: lustre Mount type:ldiskfs Flags: 0x202 (OST upgrade1.4 ) Persistent mount opts: errors=remount-ro,extents,mballoc Parameters: mgsnode=192.168.10.34@tcp Writing CONFIGS/mountdata 11.1.5 Upgrading Multiple File Systems with a Shared MGS Ost-1# mount -t lustre /dev/sda4 /mnt/test/ost/ Ost1# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 10080520 3852036 5716416 41% / /dev/sda1 101086 14964 80903 16% /boot none 501000 0 501000 0% /dev/shm /dev/sda4 101492248 471672 95781780 1% /mnt/test/ost
Upgrading to the Next Minor Version
Upgrading Lustre 1.8.0 to the Next Minor Version X
More Information
See the Lustre Operations Manual for information about:
- Upgrading Multiple File Systems with a Shared MGS
- Downgrading from Lustre 1.8.0 to Lustre 1.6.x