WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.

Difference between revisions of "Applying Lustre Patches to a Kernel"

From Obsolete Lustre Wiki
Jump to navigationJump to search
 
(127 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This page describes how to apply Lustre kernel patches to a tree, how to set up a package supplied with the Lustre code called ''Quilt'' to use to manage changes to patches, and how to modify an existing kernel patch or contribute a new kernel patch.  
+
<small>''(Updated: Jan 2010)''</small>
 +
 
 +
'''''NOTICE:'''''  The transition from CVS to Git took place on Monday, December 14.  For more information about the transition, see the [[Git Transition Notice]]. For details about how to migrate to Git, see [[Migrating to Git]].
 +
 
 +
----
 +
__TOC__
 +
This page describes how to apply Lustre™ kernel patches to a tree, how to use ''Quilt'' (a package provided with most Linux distributions) to manage changes to patches, and how to modify an existing kernel patch or contribute a new kernel patch.  
  
 
== Overview of Lustre Patches ==
 
== Overview of Lustre Patches ==
To support Lustre development and functionality, some changes must be made to the core Linux kernel. These changes are organized in a set of kernel patches kept in the Lustre CVS repository in the directory ''lustre/kernel_patches/patches/''.  
+
To support Lustre development and functionality, some changes must be made to the core Linux kernel. These changes are organized in a set of kernel patches kept in the Lustre repository in the directory ''lustre/kernel_patches/patches/''.
 +
 
 +
For a given Linux distribution, such as RHEL5 or SLES10, the
 +
corresponding kernel karget file specifies details about the kernel for
 +
which Lustre is being built.  These kernel target definitions are
 +
updated by the Lustre Engineering team whenever the supported kernel
 +
version changes.
 +
 
 +
For example, the file ''lustre/kernel_patches/targets/2.6-rhel5.target.in'' contains
 +
the following information:
 +
* Currently-supported kernel version (e.g., 2.6.18-128.7.1.el5)
 +
* Supported build architectures (e.g., i686, x86_64, ia64)
 +
* Name of the correct kernel patch series (e.g., 2.6-rhel5)
 +
* Version of OFED that will be used to build Infiniband drivers
 +
 
 +
The ''vanilla'' target is special in that it does not correspond to any
 +
specific Linux distribution.  The target describes the latest unmodified
 +
kernel.org kernel which has been tested to work with this version of
 +
Lustre.
  
 
The patches to be applied depend on the kernel that is to be used. A series file is created in ''lustre/kernel_patches/series/'' for each supported kernel to define and control the patches to be used for that kernel.
 
The patches to be applied depend on the kernel that is to be used. A series file is created in ''lustre/kernel_patches/series/'' for each supported kernel to define and control the patches to be used for that kernel.
  
For example, the file ''lustre/kernel_patches/series/rh-2.4.20'' lists all the patches that must be applied to a Red Hat 2.4.20 kernel to build a Lustre compatible kernel. An excerpt from the current rh-2.4.20 series is shown below:
+
For example, the file ''lustre/kernel_patches/series/2.6-rhel5.series'' lists all the patches that must be applied to a Red Hat 2.6.18 kernel to build a Lustre compatible kernel. An excerpt from the current ''2.6-rhel5.series'' is shown below:
 +
<pre>
 +
lustre_version.patch
 +
vfs_races-2.6-rhel5.patch
 +
i_filter_data.patch
 +
jbd-jcberr-2.6.18-vanilla.patch
 +
export_symbols-2.6.18-vanilla.patch
 +
...
 +
quota-large-limits-rhel5.patch
 +
</pre>
  
mcore-2.4.20-8.patch
+
'''''Note:''''' For more information about the set of patches developed to address issues with RAID-5 that are included in the 2.6-rhel5.series file, see [[RAID5 Patches]].
dsp.patch
 
dev_read_only_2.4.20-rh.patch
 
exports_2.4.20-rh-hp.patch
 
kmem_cache_validate_2.4.20-rh.patch
 
lustre_version.patch
 
vfs_intent-2.4.20-rh.patch
 
invalidate_show-2.4.20-rh.patch
 
iod-rmap-exports-2.4.20.patch
 
export-truncate.patch
 
....
 
....
 
....
 
  
 +
== Introduction to the Quilt package ==
  
 +
The ''Quilt'' package can be used to manage many patches on a single source tree. You will need ''Quilt'' to apply and manage Lustre kernel patches. A general overview of how this works is as follows:
  
 +
* A series file lists an ordered collection of patches.
 +
* The patches in the series form a stack.
 +
* Quilt can be used to push and pop the patches.
 +
* When the stack is managed with Quilt, patches can be edited and refreshed (updated).
 +
* Inadvertent changes can be reverted and patches forked or cloned. Diffs allow before and after change comparisons.
 +
 +
''Quilt'' is included in most Linux distributions and can be installed using a package management utility such as yum or apt-get. It can also be downloaded from the [http://savannah.nongnu.org/projects/quilt Quilt Project Site].
  
 
== Applying Lustre Kernel Patches to a Tree ==
 
== Applying Lustre Kernel Patches to a Tree ==
  
After you have checked out Lustre source code (see [[Accessing Lustre Code]]), follow these steps to apply the appropriate Lustre kernel patches to your tree:
+
After you have checked out Lustre source code (see [[Accessing Lustre Code]]) and run the ''autogen'' script (see [[Building Lustre Code|Building Lustre Code]]), follow these steps to apply the appropriate Lustre kernel patches to your tree.
 
 
1. ''Select a series file.'' The file ''lustre/kernel_patches/which_patch'' contains information about which series file corresponds to which kernel source (for each series file only one kernel source is supported). The architecture [[of what?]] is also described.
 
  
2. ''Choose a .config file.'' Each .config file corresponds to a supported kernels.  in the directory ''lustre/kernel_patches/kernel_configs''.
+
==== Preparing to apply patches ====
  
'''''Note:''''' The name of each config file begins with the series name. For example, the config file ''2.6-rhel5.config'' is a UML config file for the vanilla 2.4.20 series. [[Are the numbers correct in this sentence?]]
+
1. ''Select a series file.'' First, choose the correct kernel target (see [[#Overview of Lustre Patches|Overview of Lustre Patches]]) for your distribution and then determine the corresponding series file.
  
3. ''Unpack a kernel source tree'' (kernel sources can be found at http://downloads.lustre.org/public/kernels/). For example, enter:
+
2. ''Unpack a kernel source tree'' (supported kernel sources can be found at http://downloads.lustre.org/public/kernels/). For example, enter:
 
<pre>
 
<pre>
 
tar -xf linux-2.6.18-128.1.1-el5.tar.bz2
 
tar -xf linux-2.6.18-128.1.1-el5.tar.bz2
Line 41: Line 69:
 
The resulting source tree, referred to as the "destination tree" may be located in, for example, ''/tmp/kernels/linux-2.6.18-128.1.1''.
 
The resulting source tree, referred to as the "destination tree" may be located in, for example, ''/tmp/kernels/linux-2.6.18-128.1.1''.
  
== Introduction to the Quilt package ==
+
3. ''Choose a'' .config file ''from the directory'' lustre/kernel_patches/kernel_configs''.'' Each ''.config'' file corresponds to a supported kernel and contains the supported kernel build configuration for that kernel. 
 +
 
 +
4. ''Copy the selected kernel config to the root of the kernel source tree'', ensuring that the final file name is ''.config'':
 +
<pre>
 +
cp
 +
lustre/kernel_patches/kernel_configs/kernel-2.6.18-2.6-rhel5-x86_64-smp.config /tmp/kernels/linux-2.6.18-128.1.1/.config
 +
</pre>
 +
 
 +
==== Applying the patches ====
 +
 
 +
You will need ''Quilt'' to setup the series to use for your kernel. Complete the steps below:
 +
 
 +
1. ''Add two symbolic links (symlinks) to your linux source tree.'' In this example, you will add:
 +
 
 +
:* symlink  series -> ../lustre/kernel_patches/series/2.6-rhel5.series
 +
:* symlink  patches -> ../lustre/kernel_patches/patches
 +
 
 +
To add the symlinks, enter:
 +
<pre>
 +
# cd /usr/src/linux-2.6.18-128.1.1
 +
# ln -s ../lustre/kernel_patches/series/2.6-rhel5.series series
 +
# ln -s ../lustre/kernel_patches/patches patches
 +
</pre>
 +
 
 +
2. ''Apply the patches to the kernel source tree.''
 +
 
 +
<pre>
 +
# cd /usr/src/linux-2.6.18-128.1.1
 +
# quilt push -av
 +
</pre>
 +
The patched Linux source tree is now suitable for use during the Lustre server build process.
  
A package called '''Quilt''' package has been developed by Andreas Gruenbacher to facilitate managing many patches on a single source tree. A general overview of how this works is as follows:
+
==== Building and installing a patched kernel ====
  
* A series file lists an ordered collection of patches.
+
After successfully applying the Lustre kernel patches, you will need to build a new kernel in order to proceed with the Lustre build process. The new kernel must be installed and running before any Lustre server components can be used.
* The patches in the series form a stack.
 
* Quilt can be used to push and pop the patches.
 
* When the stack is managed with Quilt, patches can be edited and refreshed (updated).
 
* Inadvertent changes can be reverted and patches forked or cloned. Diffs allow before and after change comparisons.
 
  
''Quilt'' is included in most Linux distributions, [[or can be installed afterward using yum, apt-get, etc.]] [[(OK to replace with something like this?)]] It can also be downloaded from the [http://savannah.nongnu.org/projects/quilt Quilt Project Site].=== Applying the patches ===
+
'''''Note:''''' The kernel patch, build, and install process does not need to be
 +
repeated unless the Lustre kernel patch set changes.
  
1. Now you need use ''quilt'' to setup the series to use for your kernel. This means adding two symlinks to your linux source tree:
+
1. ''To build the new kernel, enter'':
  
a. symlink  series -> ../lustre/kernel_patches/series/2.6-rhel5.series
+
<pre>
 +
# cd /usr/src/linux-2.6.18-128.1.1
 +
# make oldconfig
 +
# make bzImage
 +
# make modules
 +
</pre>
  
b. symlink patches -> ../lustre/kernel_patches/patches
+
Completing these steps will result in a new Linux kernel (''vmlinuz'') and its
 +
associated modules.   
  
# cd /usr/src/linux-2.6.18-128.1.1
+
'''''Note:''''' Installation of the new kernel is beyond the scope of this document; please consult your distribution vendor's documentation for details.
# ln -s ../lustre/kernel_patches/series/2.6-rhel5.series series
 
# ln -s ../lustre/kernel_patches/patches patches
 
  
2. Now apply all the patches listed in the series file that quilt added a symlink for in the kernel source tree:
+
== Lustre Kernel Patch Development ==
  
# cd /usr/src/linux-2.6.18-128.1.1
+
If you are going to be modifying existing kernel patches or contributing new kernel patches, follow the procedures in this section.  
# quilt push -av
 
  
3. If the right series file was chosen and the patches were up to date the destination Linux tree should now be able to act as a base Linux source tree for Lustre.
+
'''''Note:''''' If you plan to submit your modifications to Lustre Engineering for
 +
possible inclusion in future product releases, please be sure to follow
 +
the procedures described below.
  
''User Notes''
+
'''''Note:''''' As a general guideline, limit the scope of changes in a patch file to a group of related changes.
  
== Lustre Kernel Release Maintenance ==
+
=== Directory Layout ===
  
Typically a kernel developer will fix one kernel and introduce changes to the series for this kernel. Other developers in the QA or release engineering team need to fix up the other kernels.  It is critical that the procedure below is followed in detail to avoid enormous time wastage.
+
Patches are stored in the Lustre directory tree as follows:
  
=== Naming of Complete Patches ===
+
* ''patches/'' - Contains all the patch files. Each patch should correspond to a single functional change.
 +
* ''series/'' - The text files that ''patch-utils'' use to define the order that patches are applied to a tree. A series file exists for each distinctive variant of a kernel tree (corresponding to the source, such as kernel.org or Red Hat).
  
* Patches should be named '''<patchname><kernel-version>.patch''' ( for example vfs_intent-2.4.20-rh.patch) . We store these in a subdirectory ''patches'' of  ''lustre/kernel_patches/''
+
=== Naming completed patches ===
* ''class_obd.c'' should have a:
+
When naming patches, follow these guidelines :
** LUSTRE_MIN_VERSION  == Smallest acceptable Lustre-version.
+
* Use the format <patchname><kernel-version>''.patch'' (for example, ''vfs_intent-2.4.20-rh.patch''). Patches are stored in ''lustre/kernel_patches/patches''.
** LUSTRE_MAX_VERSION  == Highest acceptable Lustre-version.
 
** This allows a script to list obsolete complete patches and clean them up.
 
* ''which_patch'' should have one entry per line '''kernel-memnonic series''' so that the lbuild script can plough through it and build kernels.
 
  
When updating patches, it is best to keep new versions of the patches as similar to the old ones as possible, so that "cvs diff" will clearly show the changes that have been made to the patch.  Also, it is best to keep the different versions of each patch as close as possible, so  "diff -u foo-rhel4.patch foo-sles10.patch" will show as small a diff as possible and we can verify that each of the patches contain the fixes that have been applied to the others.
+
When updating patches, follow these guidelines:
 +
* Keep new versions of the patches as similar to the old ones as possible, so that ''git diff'' will clearly show the changes that have been made to the patch.   
 +
* Keep the different versions of each patch as close as possible so that ''diff -u foo-rhel4.patch foo-sles10.patch'' will show as small a diff as possible and we can verify that each of the patches contains the fixes that have been applied to the others.
  
To make this easier, options can be passed to quilt via $HOME/.quiltrc:
+
To make this easier, options can be passed to ''quilt'' via ''$HOME/.quiltrc'' (copied from ''build/quiltrc'':
  
 
  export QUILT_DIFF_OPTS="-upa"
 
  export QUILT_DIFF_OPTS="-upa"
 
  export QUILT_NO_DIFF_TIMESTAMPS=1
 
  export QUILT_NO_DIFF_TIMESTAMPS=1
  
=== Maintenance: Series Files ===
+
=== Maintaining series files ===
 +
The following conventions apply to series files:
  
 
* A series file lists patches that are part of the series.
 
* A series file lists patches that are part of the series.
* By Lustre convention, a series file supports '''one''' kernel, not two or more kernels.
+
* By convention, a series file supports ''one'' kernel, not two or more kernels.
* The number of patches used by all the different series needs to be kept to a minimum.
+
* When possible, patches should be applicable to multiple kernels to minimize the total number of series-specific patches.
 +
 
 +
=== Fixing a bug involving a kernel patch ===
 +
 
 +
The following example illustrates how to fix a bug that involves a kernel patch. In this example, the solution to the bug requires a change to a patch that affects ''fs/ext3/iopen.c''.
 +
 
 +
Complete these steps: 
 +
 
 +
1. Check the series file to find the name of the patch, in this case, ''iopen-2.6-rhel5''.
 +
 
 +
2. Pop to the patch by entering:
 +
<pre>
 +
quilt pop iopen-2.6-rhel5
 +
</pre>
 +
 
 +
3. Make the fix.
 +
 
 +
4. Update the patch in the lustre source by entering (where -o starts a new session):
 +
<pre>
 +
quilt refresh -o
 +
</pre>
 +
 
 +
5. Show the changes to fix the bug that were made in this session by entering:
 +
<pre>
 +
quilt gendiff > developers-fix.diff
 +
</pre>
 +
 
 +
6. Put the other patches back by entering:
 +
<pre>
 +
quilt push -a
 +
</pre>
 +
 
 +
7. Test the code. See [[Testing Lustre Code]] for details.
 +
 
 +
8. When you have built and tested your changes by running the ''acceptance-small'' test
 +
suite,submit your patch for review and possible inclusion in an upcoming Lustre release. See [[Submitting Patches]] for details.
 +
 
 +
The Lustre release engineering team will then apply ''developers-fix.diff'' to the ''iopen.c'' patches in each of the series with that patch. The QA team will then test  each of the affected the series.
 +
 
 +
=== Upgrading a kernel ===
 +
 
 +
To upgrade a kernel, following this procedure. In the example below, the new kernel is 2.6.25 and the patches to be applied are for 2.6.24.
 +
 
 +
1. Start pushing patches until the patch that fails is topmost (for example, ''foo-2.6.20.patch'').
  
=== Bug fixes involving kernel patches ===
+
2. If it is used in another series (for example, the ''sles'' series), fork it by entering:
 +
:<pre>
 +
;quilt fork foo-2.6.25-rhel.patch
 +
</pre>
  
Peter finds the solution to a bug, he needs to changes a patch that affects fs/ext3/iopen.c.  Peter works on a private branch. He proceeds as follows:
+
3. Force it in by entering:
 +
:<pre>
 +
;quilt push -f
 +
</pre>
  
* He looks in the series file and sees that iopen-2.6-rhel5 is the name of the patch
+
4. Fix conflicts.
* Pop to that patch: '''quilt pop iopen-2.6-rhel5'''
 
* Add the fix
 
* '''quilt refresh -o '''
 
** this updates the patch in the lustre source
 
** -o starts a new session, gendiff shows diffs made in a session
 
* '''quilt gendiff > peters-fix.diff'''
 
** to show what was done in this session to fix the bug
 
* '''quilt push -a'''
 
** put all other patches back in
 
* Test the code
 
  
Peter now assigns a bug to Alex asking him to apply peters-fix.diff to all iopen.c in all series. Alex proceeds similarly with a collection of kernel sources calling quilt refresh for each iopen patch file he needs to fix.
+
5. Update the patch in the source by entering:
 +
:<pre>
 +
;quilt refresh
 +
</pre>
  
Alex now files an issue on the QA team to test all series that have the affected iopen patches.
+
6. Verify that the series file has been updated with the new patch name.
  
=== Kernel upgrades ===
+
'''''Note:''''' To back out a forced patch, enter:
 +
<pre>
 +
quilt pop -R -f
 +
</pre>
  
Suppose Linus sends us a new kernel, 2.6.25 and we have patches for 2.6.24.
+
=== Adding a new file to the kernel ===
 +
To add a new file to the kernel:
  
* Update which_patch
+
1. To make sure the top patch is the patch you want, use ''quilt push {patch_name}'' or ''quilt pop {patch_name}''.
* Start pushing patches, suppose one foo-2.6.20.patch fails (it had been around since 2.6.20 probably):
 
** If the patch is shared with another series fork it:
 
** quilt fork foo-2.6.25.patch
 
** '''quilt push -f '''
 
*** This forces the patch in
 
*** fix conflicts
 
** '''quilt refresh '''
 
  
You can back out a forced patch with ''quilt pop -R -f''.
+
2. Edit the new file.
  
=== Making Changes to the Patches ===
+
3. To add the file to one of your patches, call ''quilt add [-p {patch_name}] {file_name}''. (If the top patch is the patch to which you want to add your new file, you can omit the '' -p {patch_name} '' option.)
  
* Push the patches up to including the one that needs changes.
+
=== Making changes in another source file ===
* Edit the source file that needs changing.
+
To make changes to another source file:
* Call ''quilt refresh''.
 
* Verify that the rest of the series applies by calling ''quilt push''.
 
  
=== Adding a New File to the Kernel ===
+
1. Make sure the top patch is the patch you want using ''quilt push {patch_name}'' or ''quilt pop {patch_name}''.
* Make sure the top patch is the patch you want. (''quilt push {patch_name}'' or ''quilt pop {patch_name}'')
 
* Edit the New file
 
* Call ''quilt add [-p {patch_name}] {file_name}'' to add the file to one of your patches, If the top patch is just the patch you want to add your new file in.the option '' -p {patch_name} '' can be omitted.
 
  
=== Making Changes in Another Source File ===
+
2. To add that source file to the top patch, call ''quilt add {that_source_name}''.
* Make sure the top patch is the patch you want. (''quilt push {patch_name}'' or ''quilt pop {patch_name}'')
 
* Call ''quilt add {that_source_name}'' to add that source file to the top patch
 
* Make changes in the source file
 
* Call ''quilt refresh'' to refresh the patch.
 
  
=== Adding a Patch that Someone Else Gave You ===
+
3. Make changes in the source file.
  
To Add a Patch into a Series:
+
4. To refresh the patch, call ''quilt refresh''.
*Ideally a patch can be added to the end of the series.  This is most easily done with ''quilt add''. After the patch is imported it still needs to be applied and refreshed with ''quilt push'' and ''quilt refresh''. '''Remember to ''cvs_add'' the patch with ''-ko'' so that tags in the context of the diff aren't changed by CVS, rendering the patch unusable.'''
 
*If introducing a new patch use ''quilt new'', editing, and ''quilt refresh''.
 
  
== General Guidelines ==
+
=== Adding a patch into a series ===
  
# Try to limit the scope of changes in a patch file to a group of related changes.
+
To add a patch into a series:
  
=== CVS Directory Layout ===
+
1. Ideally, add the patch to the end of the series using ''quilt add'' to avoid the risk of cascading patch modifications.
  
# ''patches/'' - Contains all the patch files themselves.  We should have a patch per functional change.
+
2. After the patch is imported, apply and refreshed it using ''quilt push'' and ''quilt refresh''.  
# ''series/'' - The text files that patch-utils use to define the ordering of patches that are applied to a tree. We have a series file for each kernel tree variant that requires wildly different patches (architecture differences, stock vs. redhat, etc).
 
  
=== Changing Patches ===
+
3. Verify that the series file was updated.
  
It may be necessary to study Andrew's documentation also. Hopefully the instructions below are complete.
+
4. Add the patch to the repository with "git add".  
  
* http://www.zip.com.au/~akpm/linux/patches/patch-scripts-0.8/docco.txt
+
'''''Note:''''' If you are introducing a new patch, use ''quilt new'', then edit the patch, and then use ''quilt refresh''.

Latest revision as of 05:13, 22 February 2010

(Updated: Jan 2010)

NOTICE: The transition from CVS to Git took place on Monday, December 14. For more information about the transition, see the Git Transition Notice. For details about how to migrate to Git, see Migrating to Git.


This page describes how to apply Lustre™ kernel patches to a tree, how to use Quilt (a package provided with most Linux distributions) to manage changes to patches, and how to modify an existing kernel patch or contribute a new kernel patch.

Overview of Lustre Patches

To support Lustre development and functionality, some changes must be made to the core Linux kernel. These changes are organized in a set of kernel patches kept in the Lustre repository in the directory lustre/kernel_patches/patches/.

For a given Linux distribution, such as RHEL5 or SLES10, the corresponding kernel karget file specifies details about the kernel for which Lustre is being built. These kernel target definitions are updated by the Lustre Engineering team whenever the supported kernel version changes.

For example, the file lustre/kernel_patches/targets/2.6-rhel5.target.in contains the following information:

  • Currently-supported kernel version (e.g., 2.6.18-128.7.1.el5)
  • Supported build architectures (e.g., i686, x86_64, ia64)
  • Name of the correct kernel patch series (e.g., 2.6-rhel5)
  • Version of OFED that will be used to build Infiniband drivers

The vanilla target is special in that it does not correspond to any specific Linux distribution. The target describes the latest unmodified kernel.org kernel which has been tested to work with this version of Lustre.

The patches to be applied depend on the kernel that is to be used. A series file is created in lustre/kernel_patches/series/ for each supported kernel to define and control the patches to be used for that kernel.

For example, the file lustre/kernel_patches/series/2.6-rhel5.series lists all the patches that must be applied to a Red Hat 2.6.18 kernel to build a Lustre compatible kernel. An excerpt from the current 2.6-rhel5.series is shown below:

lustre_version.patch
vfs_races-2.6-rhel5.patch
i_filter_data.patch
jbd-jcberr-2.6.18-vanilla.patch
export_symbols-2.6.18-vanilla.patch
...
quota-large-limits-rhel5.patch

Note: For more information about the set of patches developed to address issues with RAID-5 that are included in the 2.6-rhel5.series file, see RAID5 Patches.

Introduction to the Quilt package

The Quilt package can be used to manage many patches on a single source tree. You will need Quilt to apply and manage Lustre kernel patches. A general overview of how this works is as follows:

  • A series file lists an ordered collection of patches.
  • The patches in the series form a stack.
  • Quilt can be used to push and pop the patches.
  • When the stack is managed with Quilt, patches can be edited and refreshed (updated).
  • Inadvertent changes can be reverted and patches forked or cloned. Diffs allow before and after change comparisons.

Quilt is included in most Linux distributions and can be installed using a package management utility such as yum or apt-get. It can also be downloaded from the Quilt Project Site.

Applying Lustre Kernel Patches to a Tree

After you have checked out Lustre source code (see Accessing Lustre Code) and run the autogen script (see Building Lustre Code), follow these steps to apply the appropriate Lustre kernel patches to your tree.

Preparing to apply patches

1. Select a series file. First, choose the correct kernel target (see Overview of Lustre Patches) for your distribution and then determine the corresponding series file.

2. Unpack a kernel source tree (supported kernel sources can be found at http://downloads.lustre.org/public/kernels/). For example, enter:

tar -xf linux-2.6.18-128.1.1-el5.tar.bz2

The resulting source tree, referred to as the "destination tree" may be located in, for example, /tmp/kernels/linux-2.6.18-128.1.1.

3. Choose a .config file from the directory lustre/kernel_patches/kernel_configs. Each .config file corresponds to a supported kernel and contains the supported kernel build configuration for that kernel.

4. Copy the selected kernel config to the root of the kernel source tree, ensuring that the final file name is .config:

cp
lustre/kernel_patches/kernel_configs/kernel-2.6.18-2.6-rhel5-x86_64-smp.config /tmp/kernels/linux-2.6.18-128.1.1/.config

Applying the patches

You will need Quilt to setup the series to use for your kernel. Complete the steps below:

1. Add two symbolic links (symlinks) to your linux source tree. In this example, you will add:

  • symlink series -> ../lustre/kernel_patches/series/2.6-rhel5.series
  • symlink patches -> ../lustre/kernel_patches/patches

To add the symlinks, enter:

# cd /usr/src/linux-2.6.18-128.1.1
# ln -s ../lustre/kernel_patches/series/2.6-rhel5.series series
# ln -s ../lustre/kernel_patches/patches patches

2. Apply the patches to the kernel source tree.

# cd /usr/src/linux-2.6.18-128.1.1
# quilt push -av

The patched Linux source tree is now suitable for use during the Lustre server build process.

Building and installing a patched kernel

After successfully applying the Lustre kernel patches, you will need to build a new kernel in order to proceed with the Lustre build process. The new kernel must be installed and running before any Lustre server components can be used.

Note: The kernel patch, build, and install process does not need to be repeated unless the Lustre kernel patch set changes.

1. To build the new kernel, enter:

# cd /usr/src/linux-2.6.18-128.1.1
# make oldconfig
# make bzImage
# make modules

Completing these steps will result in a new Linux kernel (vmlinuz) and its associated modules.

Note: Installation of the new kernel is beyond the scope of this document; please consult your distribution vendor's documentation for details.

Lustre Kernel Patch Development

If you are going to be modifying existing kernel patches or contributing new kernel patches, follow the procedures in this section.

Note: If you plan to submit your modifications to Lustre Engineering for possible inclusion in future product releases, please be sure to follow the procedures described below.

Note: As a general guideline, limit the scope of changes in a patch file to a group of related changes.

Directory Layout

Patches are stored in the Lustre directory tree as follows:

  • patches/ - Contains all the patch files. Each patch should correspond to a single functional change.
  • series/ - The text files that patch-utils use to define the order that patches are applied to a tree. A series file exists for each distinctive variant of a kernel tree (corresponding to the source, such as kernel.org or Red Hat).

Naming completed patches

When naming patches, follow these guidelines :

  • Use the format <patchname><kernel-version>.patch (for example, vfs_intent-2.4.20-rh.patch). Patches are stored in lustre/kernel_patches/patches.

When updating patches, follow these guidelines:

  • Keep new versions of the patches as similar to the old ones as possible, so that git diff will clearly show the changes that have been made to the patch.
  • Keep the different versions of each patch as close as possible so that diff -u foo-rhel4.patch foo-sles10.patch will show as small a diff as possible and we can verify that each of the patches contains the fixes that have been applied to the others.

To make this easier, options can be passed to quilt via $HOME/.quiltrc (copied from build/quiltrc:

export QUILT_DIFF_OPTS="-upa"
export QUILT_NO_DIFF_TIMESTAMPS=1

Maintaining series files

The following conventions apply to series files:

  • A series file lists patches that are part of the series.
  • By convention, a series file supports one kernel, not two or more kernels.
  • When possible, patches should be applicable to multiple kernels to minimize the total number of series-specific patches.

Fixing a bug involving a kernel patch

The following example illustrates how to fix a bug that involves a kernel patch. In this example, the solution to the bug requires a change to a patch that affects fs/ext3/iopen.c.

Complete these steps:

1. Check the series file to find the name of the patch, in this case, iopen-2.6-rhel5.

2. Pop to the patch by entering:

quilt pop iopen-2.6-rhel5

3. Make the fix.

4. Update the patch in the lustre source by entering (where -o starts a new session):

quilt refresh -o

5. Show the changes to fix the bug that were made in this session by entering:

quilt gendiff > developers-fix.diff

6. Put the other patches back by entering:

quilt push -a

7. Test the code. See Testing Lustre Code for details.

8. When you have built and tested your changes by running the acceptance-small test suite,submit your patch for review and possible inclusion in an upcoming Lustre release. See Submitting Patches for details.

The Lustre release engineering team will then apply developers-fix.diff to the iopen.c patches in each of the series with that patch. The QA team will then test each of the affected the series.

Upgrading a kernel

To upgrade a kernel, following this procedure. In the example below, the new kernel is 2.6.25 and the patches to be applied are for 2.6.24.

1. Start pushing patches until the patch that fails is topmost (for example, foo-2.6.20.patch).

2. If it is used in another series (for example, the sles series), fork it by entering:

quilt fork foo-2.6.25-rhel.patch

3. Force it in by entering:

quilt push -f

4. Fix conflicts.

5. Update the patch in the source by entering:

quilt refresh

6. Verify that the series file has been updated with the new patch name.

Note: To back out a forced patch, enter:

quilt pop -R -f

Adding a new file to the kernel

To add a new file to the kernel:

1. To make sure the top patch is the patch you want, use quilt push {patch_name} or quilt pop {patch_name}.

2. Edit the new file.

3. To add the file to one of your patches, call quilt add [-p {patch_name}] {file_name}. (If the top patch is the patch to which you want to add your new file, you can omit the -p {patch_name} option.)

Making changes in another source file

To make changes to another source file:

1. Make sure the top patch is the patch you want using quilt push {patch_name} or quilt pop {patch_name}.

2. To add that source file to the top patch, call quilt add {that_source_name}.

3. Make changes in the source file.

4. To refresh the patch, call quilt refresh.

Adding a patch into a series

To add a patch into a series:

1. Ideally, add the patch to the end of the series using quilt add to avoid the risk of cascading patch modifications.

2. After the patch is imported, apply and refreshed it using quilt push and quilt refresh.

3. Verify that the series file was updated.

4. Add the patch to the repository with "git add".

Note: If you are introducing a new patch, use quilt new, then edit the patch, and then use quilt refresh.