View source for Applying Lustre Patches to a Kernel

This page describes how to apply Lustre kernel patches to a tree, how to use ''Quilt'' (a package provided with most Linux distributions) to manage changes to patches, and how to modify an existing kernel patch or contribute a new kernel patch. 

== Overview of Lustre Patches ==
To support Lustre development and functionality, some changes must be made to the core Linux kernel. These changes are organized in a set of kernel patches kept in the Lustre CVS repository in the directory ''lustre/kernel_patches/patches/''. 

The patches to be applied depend on the kernel that is to be used. A series file is created in ''lustre/kernel_patches/series/'' for each supported kernel to define and control the patches to be used for that kernel.

For example, the file ''lustre/kernel_patches/series/rh-2.4.20'' lists all the patches that must be applied to a Red Hat 2.6.18 kernel to build a Lustre compatible kernel. An excerpt from the current ''rh-2.6.18'' series is shown below:
<pre>
lustre/kernel_patches/series/2.6-rhel5.series:
lustre_version.patch
vfs_races-2.6-rhel5.patch
i_filter_data.patch
jbd-jcberr-2.6.18-vanilla.patch
export_symbols-2.6.18-vanilla.patch
...
quota-large-limits-rhel5.patch
</pre>

== Introduction to the Quilt package ==

The ''Quilt'' package can be used to manage many patches on a single source tree. You will need ''Quilt'' to apply and manage Lustre kernel patches. A general overview of how this works is as follows:

* A series file lists an ordered collection of patches.
* The patches in the series form a stack.
* Quilt can be used to push and pop the patches.
* When the stack is managed with Quilt, patches can be edited and refreshed (updated).
* Inadvertent changes can be reverted and patches forked or cloned. Diffs allow before and after change comparisons.

''Quilt'' is included in most Linux distributions and can be installed using a package management utility such as yum or apt-get. It can also be downloaded from the [http://savannah.nongnu.org/projects/quilt Quilt Project Site].

== Applying Lustre Kernel Patches to a Tree ==

After you have checked out Lustre source code (see [[Accessing Lustre Code]]) and run the ''autogen'' script (see [[A Building Lustre Code|Building Lustre Code]]), follow these steps to apply the appropriate Lustre kernel patches to your tree.

==== Preparing to apply patches ====

1. ''Select a series file.'' The file ''lustre/kernel_patches/which_patch'' contains information about which series file corresponds to which kernel source (for each series file only one kernel source is supported). The architecture is also described.

2. ''Unpack a kernel source tree'' (supported kernel sources can be found at http://downloads.lustre.org/public/kernels/). For example, enter:
<pre>
tar -xf linux-2.6.18-128.1.1-el5.tar.bz2
</pre>
The resulting source tree, referred to as the "destination tree" may be located in, for example, ''/tmp/kernels/linux-2.6.18-128.1.1''.

3. ''Choose a'' .config file ''from the directory'' lustre/kernel_patches/kernel_configs ''.'' Each .config file corresponds to a supported kernel and contains the supported kernel build configuration for that kernel.  

4. ''Copy the selected kernel config to the root of the kernel source tree'', ensuring that the final file name is ''.config'':
<pre>
cp
lustre/kernel_patches/kernel_configs/kernel-2.6.18-2.6-rhel5-x86_64-smp.config /tmp/kernels/linux-2.6.18-128.1.1/.config
</pre>

==== Applying the patches ====

You will need ''Quilt'' to setup the series to use for your kernel. Complete the steps below:

1. ''Add two symbolic links (symlinks) to your linux source tree.'' In this example, you will add:

:* symlink  series -> ../lustre/kernel_patches/series/2.6-rhel5.series
:* symlink  patches -> ../lustre/kernel_patches/patches

To add the symlinks, enter:
<pre>
# cd /usr/src/linux-2.6.18-128.1.1
# ln -s ../lustre/kernel_patches/series/2.6-rhel5.series series
# ln -s ../lustre/kernel_patches/patches patches
</pre>

2. ''Apply the patches to the kernel source tree.'' 

<pre>
# cd /usr/src/linux-2.6.18-128.1.1
# quilt push -av
</pre>
The patched Linux source tree is now suitable for use during the Lustre server build process.

== Lustre Kernel Patch Development ==

If you are going to be modifying existing kernel patches or contributing new kernel patches, follow the procedures in this section. 

'''''Note:''''' If you plan to submit your modifications to Lustre Engineering for
possible inclusion in future product releases, please be sure to follow
the procedures described below.

=== General Guidelines ===

As a general guideline, limit the scope of changes in a patch file to a group of related changes.

==== CVS Directory Layout ====

Patches are stored in CVS as follows:

* ''patches/'' - Contains all the patch files. Each patch should correspond to a single functional change.
* ''series/'' - The text files that ''patch-utils'' use to define the order that patches are applied to a tree. A series file exists for each distinctive variant of a kernel tree (corresponding to the source, such as kernel.org or Red Hat).

=== Naming completed patches ===
When naming patches, follow these guidelines :
* Use the format <patchname><kernel-version>''.patch'' (for example, ''vfs_intent-2.4.20-rh.patch''). Patches are stored in ''lustre/kernel_patches/patches''.
* Include the following in ''class_obd.c'':
:: LUSTRE_MIN_VERSION  == Smallest acceptable Lustre-version.
:: LUSTRE_MAX_VERSION  == Highest acceptable Lustre-version.
:This allows a script to list obsolete patches and clean them up.

When updating patches, follow these guidelines:
* Keep new versions of the patches as similar to the old ones as possible, so that ''cvs diff'' will clearly show the changes that have been made to the patch.  
* Keep the different versions of each patch as close as possible so that  ''diff -u foo-rhel4.patch foo-sles10.patch'' will show as small a diff as possible and we can verify that each of the patches contains the fixes that have been applied to the others.

To make this easier, options can be passed to ''quilt'' via ''$HOME/.quiltrc'':

 export QUILT_DIFF_OPTS="-upa"
 export QUILT_NO_DIFF_TIMESTAMPS=1

=== Maintaining series files ===
The following conventions apply to series files:

* A series file lists patches that are part of the series.
* By convention, a series file supports ''one'' kernel, not two or more kernels.
* '''The number of patches used by all the different series needs to be kept to a minimum.''' [[OK to replace with this?]] When possible, patches should be applicable to multiple kernels to minimize the total number of series-specific patches.

=== Fixing a bug involving a kernel patch ===

The following example illustrates how to fix a bug that involves a kernel patch. In this example, the solution to the bug requires a change to a patch that affects ''fs/ext3/iopen.c''. 

Complete these steps:  

1. Check the series file to find the name of the patch, in this case, ''iopen-2.6-rhel5''.

2. Pop to the patch by entering:
<pre>
quilt pop iopen-2.6-rhel5
</pre>

3. Make the fix.

4. Update the patch in the lustre source by entering (where -o starts a new session):
<pre>
quilt refresh -o
</pre>

5. Show the changes to fix the bug that were made in this session by entering:
<pre>
quilt gendiff > developers-fix.diff
</pre>

6. Put the other patches back by entering:
<pre>
quilt push -a
</pre>

7. Test the code.

8. When you have built and tested your changes by running the ''acceptance-small'' test
suite,submit your patch for review and possible inclusion in an upcoming Lustre release. See [[Submitting Patches]] for details.

The Lustre release engineering team will then apply ''developers-fix.diff'' to the ''iopen.c'' patches in each of the series with that patch. The QA team will then test  each of the affected the series.

=== Upgrading a kernel ===

To upgrade a kernel, following this procedure.

''Example:'' Assume the new kernel is 2.6.25 and you have patches for 2.6.24.

'''[[Can this procedure...]]'''
::'''* Update which_patch'''
::'''* Start pushing patches, suppose one foo-2.6.20.patch fails (it had been around since 2.6.20 probably):'''
::'''** If the patch is shared with another series fork it:'''
::'''** quilt fork foo-2.6.25.patch'''
::'''** '''quilt push -f ''''''
::'''*** This forces the patch in'''
::'''*** fix conflicts'''
::'''** '''quilt refresh ''''''

::'''You can back out a forced patch with ''quilt pop -R -f''.'''

'''[[...be replaced with something like this?]]'''

1. Update ''which_patch''.

2. Start pushing patches.

3. If a patch fails (for example, ''foo-2.6.20.patch'') and it is used in another series, fork it by entering:
:<pre>
:quilt fork foo-2.6.25.patch'''
:</pre>

4. Force it in by entering:
:<pre>
:quilt push -f
:</pre>

5. Fix conflicts.

6. Update the patch in the source by entering:
:<pre>
:quilt refresh
:</pre>

[[Are more details needed in any of the above steps?]]

To back out a forced patch, enter:
<pre>
quilt pop -R -f
</pre>

=== Making changes to a patch ===
To make a change to a patch:

1. Push the patches up until you arrive at the one that needs to be changed.

2.  Edit the source file that needs changing.

3.  Call ''quilt refresh''.

4.  Verify that the rest of the series applies by calling ''quilt push''.

=== Adding a new file to the kernel ===
To add a new file to the kernel:

1.  To make sure the top patch is the patch you want, use ''quilt push {patch_name}'' or ''quilt pop {patch_name}''.

2. Edit the new file.

3. To add the file to one of your patches, call ''quilt add [-p {patch_name}] {file_name}''. (If the top patch is the patch to which you want to add your new file, you can omit the '' -p {patch_name} '' option.)

=== Making changes in another source file ===
To make changes to another source file:

1. Make sure the top patch is the patch you want using ''quilt push {patch_name}'' or ''quilt pop {patch_name}''.

2. To add that source file to the top patch, call ''quilt add {that_source_name}''.

3. Make changes in the source file.

4. To refresh the patch, call ''quilt refresh''.

=== Adding a patch provided by someone else ===

To Add a Patch into a Series:
*Ideally a patch can be added to the end of the series.  This is most easily done with ''quilt add''. After the patch is imported it still needs to be applied and refreshed with ''quilt push'' and ''quilt refresh''. '''Remember to ''cvs_add'' the patch with ''-ko'' so that tags in the context of the diff aren't changed by CVS, rendering the patch unusable.'''
*If introducing a new patch use ''quilt new'', editing, and ''quilt refresh''.

'''[[OK to replace all the text above with something like this?]]'''

To add a patch into a series:

1. Add the patch to the end of the series using ''quilt add''. 

2. After the patch is imported, apply and refreshed it using ''quilt push'' and ''quilt refresh''. 

3. [[Check in the patch?]] using ''cvs_add'' with the ''-ko'' option so that tags in the context of the diff aren't changed by CVS, rendering the patch unusable.

'''''Note:''''' If you are introducing a new patch, use ''quilt new'', then edit the patch, and then use ''quilt refresh''.