Applying Lustre Patches to a Kernel

This page describes how to apply Lustre kernel patches to a tree, how to set up a package supplied with the Lustre code called Quilt to manage changes to patches, and how to modify an existing kernel patch or contribute a new kernel patch.

Overview of Lustre Patches
To support Lustre development and functionality, some changes must be made to the core Linux kernel. These changes are organized in a set of kernel patches kept in the Lustre CVS repository in the directory lustre/kernel_patches/patches/.

The patches to be applied depend on the kernel that is to be used. A series file is created in lustre/kernel_patches/series/ for each supported kernel to define and control the patches to be used for that kernel.

For example, the file lustre/kernel_patches/series/rh-2.4.20 lists all the patches that must be applied to a Red Hat 2.4.20 kernel to build a Lustre compatible kernel. An excerpt from the current rh-2.4.20 series is shown below:

mcore-2.4.20-8.patch dsp.patch dev_read_only_2.4.20-rh.patch exports_2.4.20-rh-hp.patch kmem_cache_validate_2.4.20-rh.patch lustre_version.patch vfs_intent-2.4.20-rh.patch invalidate_show-2.4.20-rh.patch iod-rmap-exports-2.4.20.patch export-truncate.patch .... .... ....

Introduction to the Quilt package
The Quilt package can be used to manage many patches on a single source tree. You will need Quilt to apply and manage Lustre kernel patches. A general overview of how this works is as follows:


 * A series file lists an ordered collection of patches.
 * The patches in the series form a stack.
 * Quilt can be used to push and pop the patches.
 * When the stack is managed with Quilt, patches can be edited and refreshed (updated).
 * Inadvertent changes can be reverted and patches forked or cloned. Diffs allow before and after change comparisons.

Quilt is included in most Linux distributions, or can be installed afterward using yum, apt-get, etc. (OK to replace with something like this?) It can also be downloaded from the Quilt Project Site.

Applying Lustre Kernel Patches to a Tree
After you have checked out Lustre source code (see Accessing Lustre Code) and run the autogen script (see Building Lustre Code), follow these steps to apply the appropriate Lustre kernel patches to your tree.

Preparing to Apply Patches
1. Select a series file. The file lustre/kernel_patches/which_patch contains information about which series file corresponds to which kernel source (for each series file only one kernel source is supported). The architecture of what? is also described.

2. Choose a .config file from the directory lustre/kernel_patches/kernel_configs .Each .config file corresponds to a supported kernel.

Note: The name of each config file begins with the series name. For example, the config file 2.6-rhel5.config is a UML config file for the vanilla 2.4.20 series. Are the numbers correct in this sentence?

3. Unpack a kernel source tree (kernel sources can be found at http://downloads.lustre.org/public/kernels/). For example, enter: tar -xf linux-2.6.18-128.1.1-el5.tar.bz2 The resulting source tree, referred to as the "destination tree" may be located in, for example, /tmp/kernels/linux-2.6.18-128.1.1.

Applying the patches
You will need Quilt to setup the series to use for your kernel. Complete the steps below:

1. Add two symbolic links? (symlinks) to your linux source tree. In this example, you will add:


 * symlink series -> ../lustre/kernel_patches/series/2.6-rhel5.series
 * symlink patches -> ../lustre/kernel_patches/patches

To add the symlinks, enter:
 * 1) cd /usr/src/linux-2.6.18-128.1.1
 * 2) ln -s ../lustre/kernel_patches/series/2.6-rhel5.series series
 * 3) ln -s ../lustre/kernel_patches/patches patches


 * 2. Apply all the patches listed in the series file that quilt added a symlink for in the kernel source tree: Can this step be replaced by something like the following?

Quilt adds a symlink for each patch listed in the series field.

2. Apply the patches to the kernel source tree.

The destination Linux tree should now be able to act as a base Linux source tree for Lustre.
 * 1) cd /usr/src/linux-2.6.18-128.1.1
 * 2) quilt push -av

Lustre Kernel Patch Development
If you are going to be modifying existing kernel patches or contributing new kernel patches, follow the procedures in this section.

Note: When you modify or add a kernel patch, you may? will? introduce a change into the series for the kernel. It is important that you follow the procedures below so that developers on the QA or release engineering team know where to make any changes needed to the other kernels. Is this note OK?

General Guidelines
As a general guideline, limit the scope of changes in a patch file to a group of related changes.

CVS Directory Layout
Patches are stored in CVS as follows:


 * patches/ - Contains all the patch files. Each patch should correspond to a single functional change.
 * series/ - The text files that patch-utils use to define the order that patches are applied to a tree. We have a series file for each kernel tree variant that requires wildly different patches (architecture differences, stock vs. redhat, etc). OK to replace with this? A series file exists for each distinctive variant of a kernel tree (as determined by architecture, source (kernel.org vs. Red Hat), etc.).

Changing Patches
Can this section be deleted? The link is no longer live. It may be necessary to study Andrew's documentation also. Hopefully the instructions below are complete.


 * http://www.zip.com.au/~akpm/linux/patches/patch-scripts-0.8/docco.txt

Naming Completed Patches
When naming patches, follow these guidelines :
 * Use the format .patch (for example, vfs_intent-2.4.20-rh.patch). Patches are stored in lustre/kernel_patches/ in the subdirectory patches. Is this sentence correct?
 * Include the following in class_obd.c:
 * - LUSTRE_MIN_VERSION == Smallest acceptable Lustre-version.
 * - LUSTRE_MAX_VERSION == Highest acceptable Lustre-version.
 * This allows a script to list obsolete complete patches and clean them up.


 * * which_patch should have one entry per line so that the lbuild script can plough through it and build kernels.

OK to change this bullet to something like the following?
 * In which_patch, include one entry per line -- not sure how this fits: kernel-memnonic series --, so that the lbuild script can process the file line-by-line when it is building the kernel.

When updating patches, follow these guidelines:
 * Keep new versions of the patches as similar to the old ones as possible, so that cvs diff will clearly show the changes that have been made to the patch.
 * Keep the different versions of each patch as close as possible, so diff -u foo-rhel4.patch foo-sles10.patch will show as small a diff as possible and we can verify that each of the patches contains the fixes that have been applied to the others.

To make this easier, options can be passed to quilt via $HOME/.quiltrc:

export QUILT_DIFF_OPTS="-upa" export QUILT_NO_DIFF_TIMESTAMPS=1

Maintaining Series Files
The following conventions apply to series files:


 * A series file lists patches that are part of the series.
 * By convention, a series file supports one kernel, not two or more kernels.
 * The number of patches used by all the different series by more than one series? needs to be kept to a minimum.

Fixing a bug involving a kernel patch
The following example illustrates how to fix a bug that involves a kernel patch.

Example: A solution to a bug requires a change to a patch that affects fs/ext3/iopen.c. Working on a private branch does this apply to external developers?, the developer completes these steps:

1. Check the series file to find the name of the patch, in this case, iopen-2.6-rhel5.

2. Pop to the patch by entering: quilt pop iopen-2.6-rhel5

3. Make the fix.

4. Update the patch in the lustre source by entering (where -o starts a new session): quilt refresh -o

5. Show the changes to fix the bug that were made in this session by entering: quilt gendiff > developers-fix.diff

6. Put the other patches back by entering: quilt push -a

7. Test the code.

8. OK to add step with link to Submitting Patches here?

The Lustre release engineering team will then apply developers-fix.diff to the iopen.c patches in each of the series with that patch. The QA team will then test each of the affected the series.

Upgrading a Kernel
To upgrade a kernel, following this procedure.

Example: Assume the new kernel is 2.6.25 and you have patches for 2.6.24.

Can this procedure...
 * * Update which_patch
 * * Start pushing patches, suppose one foo-2.6.20.patch fails (it had been around since 2.6.20 probably):
 * ** If the patch is shared with another series fork it:
 * ** quilt fork foo-2.6.25.patch
 * ** quilt push -f 
 * *** This forces the patch in
 * *** fix conflicts
 * ** quilt refresh 


 * You can back out a forced patch with quilt pop -R -f.

...be replaced with something like this?

1. Update which_patch.

2. Start pushing patches.

3. If a patch fails (for example, foo-2.6.20.patch) and it is used in another series, fork it by entering:
 * quilt fork foo-2.6.25.patch'''
 * quilt fork foo-2.6.25.patch'''

4. Force it in by entering:
 * quilt push -f
 * quilt push -f

5. Fix conflicts.

6. Update the patch in the source by entering:
 * quilt refresh
 * quilt refresh

Are more details needed in any of these steps?

To back out a forced patch, enter: quilt pop -R -f

Making Changes to a Patch
To make a change to a patch:

1. Push the patches up to the one that needs to be changed.

2. Edit the source file that needs changing.

3. Call quilt refresh.

4. Verify that the rest of the series applies by calling quilt push.

Adding a New File to the Kernel
To add a new file to the kernel:

1. To make sure the top patch is the patch you want, use quilt push {patch_name} or quilt pop {patch_name}.

2. Edit the new file.

3. To add the file to one of your patches, call quilt add [-p {patch_name}] {file_name}. If the top patch is the patch to which you want to add your new file, you can omit the  -p {patch_name}  option.

Making Changes in Another Source File

 * Make sure the top patch is the patch you want. (quilt push {patch_name} or quilt pop {patch_name})
 * Call quilt add {that_source_name} to add that source file to the top patch
 * Make changes in the source file
 * Call quilt refresh to refresh the patch.

Adding a Patch that Someone Else Gave You
To Add a Patch into a Series:
 * Ideally a patch can be added to the end of the series. This is most easily done with quilt add. After the patch is imported it still needs to be applied and refreshed with quilt push and quilt refresh. Remember to cvs_add the patch with -ko so that tags in the context of the diff aren't changed by CVS, rendering the patch unusable.
 * If introducing a new patch use quilt new, editing, and quilt refresh.