Obsolete Lustre Wiki - User contributions [en]

Acceptance Small (acc-sm) Testing on Lustre

2009-05-05T14:08:42Z

Scjody: Add detail on first time run

The Lustre QE group and developers use acceptance-small (acc-sm) tests to catch bugs early in the development cycle. Within the Lustre group, acc-sm tests are run on YALA, an automated test system. This information is being published to describe the steps to perform acceptance small testing and encourage wider acc-sm testing in the Lustre community.

'''NOTE''': For your convenience, this document is also available as a [http://wiki.lustre.org/images/c/c6/AccSm_Testing.pdf PDF].

==What is acc-sm testing and why do we use it for Lustre?==

Acceptance small (acc-sm) testing is a suite of test cases used to verify different aspects of Lustre functionality.

* These tests are run using the acceptance-small.sh script.
* The script is run from the lustre/tests directory in a compiled Lustre tree.
* The acceptance-small.sh script runs a number of test scripts that are also run by the ltest (Buffalo) test harness on Lustre test clusters.

==What tests comprise the acc-sm test suite?==

Each Lustre CVS branch contains a lustre/tests sub-directory; all acc-sm tests are stored here. The acceptance-small.sh file contains a list of all tests in the acc-sm suite. To get the list, run:

$ grep TESTSUITE_LIST acceptance-small.sh

The acc-sm tests are listed below, by CVS branch.

====b1_6 branch====

This branch includes 17 acc-sm test suites.

$ grep TESTSUITE_LIST acceptance-small.sh
export TESTSUITE_LIST="RUNTESTS SANITY DBENCH BONNIE IOZONE FSX SANITYN LFSCK
LIBLUSTRE REPLAY_SINGLE CONF_SANITY RECOVERY_SMALL REPLAY_OST_SINGLE
REPLAY_DUAL INSANITY SANITY_QUOTA PERFORMANCE_SANITY"

====b1_8_gate branch====

This branch includes 18 acc-sm test suites.

$ grep TESTSUITE_LIST acceptance-small.sh
export TESTSUITE_LIST="RUNTESTS SANITY DBENCH BONNIE IOZONE FSX SANITYN LFSCK
LIBLUSTRE REPLAY_SINGLE CONF_SANITY RECOVERY_SMALL REPLAY_OST_SINGLE
REPLAY_DUAL REPLAY_VBR INSANITY SANITY_QUOTA PERFORMANCE_SANITY"

====HEAD branch====

This branch includes 19 acc-sm test suites.

$ grep TESTSUITE_LIST acceptance-small.sh
export TESTSUITE_LIST="RUNTESTS SANITY DBENCH BONNIE IOZONE FSX SANITYN LFSCK
LIBLUSTRE REPLAY_SINGLE CONF_SANITY RECOVERY_SMALL REPLAY_OST_SINGLE
REPLAY_DUAL INSANITY SANITY_QUOTA SANITY_SEC SANITY_GSS
PERFORMANCE_SANITY"

To see the test cases in a particular acc-sm test, run:

$ grep run_ <test suite script>

For example, to see the last 3 test cases that comprise the SANITY test:

$ grep run_ sanity.sh | tail -3

run_test 130c "FIEMAP (2-stripe file with hole)"
run_test 130d "FIEMAP (N-stripe file)"
run_test 130e "FIEMAP (test continuation FIEMAP calls)"

==For each acc-sm test, what does it measure or show?==

The acc-sm test suite are described below.

;'''RUNTESTS'''
: A basic regression test with unmount/remount.

;'''SANITY'''
: Verifies Lustre operation under normal operating conditions.

;'''DBENCH'''
:Dbench benchmark for simulating N clients to produce the filesystem load.

;'''BONNIE'''
:Bonnie++ benchmark for creation, reading and deleting many small files

;'''IOZONE'''
:IOzone benchmark for generating and measuring a variety of file operations.

;'''FSX'''
:Filesystem exerciser.

;'''SANITYN'''
:Verifies operations from two clients under normal operating conditions.

;'''LFSCK'''
:Tests e2fsck and lfsck to detect and fix filesystm corruption.

;'''LIBLUSTRE'''
:Runs a test linked to a liblustre client library.

;'''REPLAY_SINGLE'''
:Verifies recovery after an MDS failure.

;'''CONF_SANITY'''
:Verifies various Lustre configurations (including wrong ones), where the system must behave correctly.

;'''RECOVERY_SMALL'''
:Verifies RPC replay after a communications failure (message loss).

;'''REPLAY_OST_SINGLE'''
:Verifies recovery after an OST failure.

;'''REPLAY_DUAL'''
:Verifies recovery from two clients after a server failure.

;'''INSANITY'''
:Tests multiple concurrent failure conditions.

;'''SANITY_QUOTA'''
:Verifies filesystem quotas.

==How do you get the acc-sm tests?==

The acc-sm test suite is stored in the lustre/tests sub-directory on each CVS branch (b1_6, b1_8, and HEAD).

==Do you have to run every acc-sm test?==

No. You can choose to run only specified acc-sm tests. Tests can be run either with or without the acceptance-sm.sh (acc-sm.sh) wrapper script. Here are several examples:

To only run the RUNTESTS and SANITY tests:

ACC_SM_ONLY=”RUNTESTS SANITY” sh acceptance-small.sh

To only run test_1 and test_2 of the SANITYN tests:

ACC_SM_ONLY=”SANITYN” ONLY=”1 2” sh acceptance-small.sh

To only run the replay-single.sh test and except (not run) the test_3* and test_4* tests:

ACC_SM_ONLY=”REPLAY_SINGLE” REPLAY_SINGLE_EXCEPT=”3 4” sh acceptance-small.sh

To only run conf-sanity.sh tests after #15 (without the acceptance-small.sh wrapper script):

CONF_SANITY_EXCEPT=”$(seq 15)“ sh conf-sanity.sh

==Do the acc-sm tests have to be run in a specific order?==

The test order is defined in the acceptance-small.sh script and in each test script. Users do not have to (and should not) do anything to change the order of tests.

==Who runs the acc-sm tests?==

Currently, the QE group and Lustre developers run acc-sm as the main test suite for Lustre testing. Acc-sm tests are run on YALA, the automated test system, with test reports submitted to Buffalo (a web interface that allows for browsing various Lustre test results). We welcome external contributions to the Lustre acc-sm test efforts – either of the Lustre code base or new testing platforms.

==What type of Lustre environment is needed to run the acc-sm tests? Is anything special needed?==

The default Lustre configuration for acc-sm testing is a single node setup with one MDS and two OSTs. All devices are loop-back devices. YALA, the automated test system, uses a non-default configuration.

To run the acc-sm test suite on a non-default Lustre configuration, you have to modify the default settings in the acc-sm configuration file, lustre/tests/cfg/local.sh. The configuration variables include mds_HOST, ost_HOST, OSTCOUNT, MDS_MOUNT_OPTS and OST_MOUNT_OPTS, among others.

To create your own configuration file, copy cfg/local.sh to cfg/my_config.sh:

cp cfg/local.sh cfg/my_config.sh

Edit the necessary variables in the configuration file (my_config.sh) and run acc-sm as: NAME=my_config sh acceptance-small.sh

==What are the steps to run acc-sm?==

There are two methods to run the acc-sm tests.

1. Check out a Lustre branch (b1_6, b1_8 or HEAD).

2. Change directory to lustre/tests:

cd lustre/tests

3. Build lustre/tests.

4. Run acc-sm on a local, default Lustre configuration (1 MGS/MDT, 1 OST and 1 client):

sh acceptance-small.sh 2>&1 | tee /tmp/output

- OR -

1. Install the lustre-tests RPM (available at lts-head:/var/cache/cfs/PACKAGE/rpm/lustre).

2. Change directory to lustre/tests:

cd /usr/lib/lustre/tests

3. Create your own configuration file and edit it for your configuration.

cp cfg/local.sh cfg/my_config.sh

4. Run acc-sm on a local Lustre configuration.

Here is an example of running acc-sm on a non-default Lustre configuration (MDS is sfire7, OST is sfire8, OSCOUNT=1, etc). In this example, only the SANITY test cases are being run.

ACC_SM_ONLY=SANITY mds_HOST=sfire7 ost8_HOST=sfire8 MDSDEV1=/dev/sda1
OSTCOUNT=1 OSTDEV1=/dev/sda1 MDSSIZE=5000000 OSTSIZE=5000000
MDS_MOUNT_OPTS="-o user_xattr" OST_MOUNT_OPTS=" -o user_xattr"
REFORMAT="--reformat" PDSH="pdsh -S -w" sh acceptance-small.sh

==What if I hit a failure on an acc-sm test?==

* If you regularly hit a failure in any of these tests, check if a bug has been reported on the failure or file a new bug if one has not yet been opened.
* If the bug prevents you from completing the tests, set the environment variables to skip the specific test(s) until you or someone else fixes them.
** For example, to skip sanity.sh subtest 36g and 65, replay-single.sh subtest 42, and all of insanity.sh set in your environment:
**:
**:: <pre><nowiki>
**::
**:: export SANITY_EXCEPT="36g 65"
**:: export REPLAY_SINGLE_EXCEPT=42
**:: export INSANITY=no
**:: </nowiki></pre>
**::
** You can also skip tests on the command line. For example, when running acceptance-small:
**:
**:: <pre><nowiki>
**:: SANITY_EXCEPT="36g 65" REPLAY_SINGLE_EXCEPT=42 INSANITY=no ./acceptance-small.sh
**:: </nowiki></pre>
** The test framework is very flexible, and it is a very easy "hands-off" way of running testing while you are doing other things, like coding.
** Questions/problems with the test framework should be emailed to the lustre-discuss mailing list, so all Lustre users can benefit from improving and documenting it.
* If you do not run the entire test suite regularly, you will have no idea whether a bug is added from your code or not, and you will waste a lot of time looking.

==How do you run acc-sm on a mounted Lustre system?==

To run acc-sm on a Lustre system that is already mounted, you need to use the correct configuration file (according to the mounted Lustre system) and run acc-sm as:

SETUP=: CLEANUP=: FORMAT=: NAME=<config> sh acceptance-small.sh

==How do you run acc-sm with and without reformat?==

By default, the acc-sm test suite does not reformat Lustre. If you want to reformat Lustre, run acc-sm with REFORMAT="--reformat":

REFORMAT="--reformat" sh acceptance-small.sh

'''If this is a new system or you are using new devices for Lustre, reformatting must be done.'''

If needed, you can specify WRITECONF="writeconf", and then run acc-sm with WRITECONF="writeconf":

WRITECONF="writeconf" sh acceptance-small.sh

==How do you run acc-sm in a Lustre configuration with several clients?==

The default configuration file for acc-sm is cfg/local.sh, which uses only one client (local). To use additional remote clients, specify the RCLIENTS list and use the cfg/ncli.sh configuration file (or your own copy of ncli configuration).

NAME=ncli RCLIENT=<space-separated list of remote clients> sh acceptance-small.sh

For example:

NAME=ncli RCLIENT="client2 client3 client11" sh acceptance-small.sh

==What is the SLOW variable and how is it used with acc-sm?==

The SLOW variable is used to run a subset of acc-sm tests. By default, the variable is set to SLOW=no, which causes some of the longer acc-sm tests to be skipped and acc-sm test run to complete in less than 2 hours. To run all of the acc-sm tests, set the variable to SLOW=yes:

SLOW=yes sh acceptance-small.sh

==What is the FAIL_ON_ERROR variable and how is it used with acc-sm?==

The FAIL_ON_ERROR variable is used to "stop" or "continue" running acc-sm tests after a test failure occurs. If the variable is set to "true" (FAIL_ON_ERROR=true), then acc-sm stops after test_N fails and test_N+1 does not run. If the variable is set to "false" (FAIL_ON_ERROR=false), then acc-sm continues after test_N fails and test_N+1 does run.

FALSE_ON_ERROR=false, by default, for the sanity, sanityn and sanity-quota tests. FALSE_ON_ERROR=true for the replay/recovery tests.

==What is the PDSH variable and how it is used with acc-sm?==

The PDSH variable is used to provide remote shell access. If acc-sm is run on a Lustre configuration with remote servers, specify PDSH like this:

PDSH="pdsh -S w" sh acceptance-small.sh

If the client has no access to the servers, you can run acc-sm without PDSH, but the tests which need PDSH access are skipped. A summary report is generated which lists the skipped tests.

==What is the CMD configuration for HEAD?==

For the HEAD branch, specify the MDSCOUNT variable (number of MDTs). By default, the variable is set to 1. If you have a Lustre configuration with several MDT nodes, they need to be specified in the configuration file as mds1_HOST, mds2_HOST, ...

By default, all of these variables are set to the mds_HOST value.

==What do we do with the acc-sm test results?==

Acc-sm results are sent to Buffalo, a web interface for Lustre test results. The default Buffalo display shows a summary of tests run on different hardware configurations for various CVS branches for the past 24 hours, with links to the various reports. For more information on reporting test results
to Buffalo, see [http://wiki.lustre.org/index.php?title=Buffalizing_Tests Buffalizing Tests].

If an acc-sm test fails, then the failure is investigated. If the investigation reveals there is a Lustre defect, a bug is opened in Bugzilla to fix the problem and also the acc-sm issue.

Acceptance Small (acc-sm) Testing on Lustre

2009-05-05T14:07:32Z

Scjody: Fix examples

The Lustre QE group and developers use acceptance-small (acc-sm) tests to catch bugs early in the development cycle. Within the Lustre group, acc-sm tests are run on YALA, an automated test system. This information is being published to describe the steps to perform acceptance small testing and encourage wider acc-sm testing in the Lustre community.

'''NOTE''': For your convenience, this document is also available as a [http://wiki.lustre.org/images/c/c6/AccSm_Testing.pdf PDF].

==What is acc-sm testing and why do we use it for Lustre?==

Acceptance small (acc-sm) testing is a suite of test cases used to verify different aspects of Lustre functionality.

* These tests are run using the acceptance-small.sh script.
* The script is run from the lustre/tests directory in a compiled Lustre tree.
* The acceptance-small.sh script runs a number of test scripts that are also run by the ltest (Buffalo) test harness on Lustre test clusters.

==What tests comprise the acc-sm test suite?==

Each Lustre CVS branch contains a lustre/tests sub-directory; all acc-sm tests are stored here. The acceptance-small.sh file contains a list of all tests in the acc-sm suite. To get the list, run:

$ grep TESTSUITE_LIST acceptance-small.sh

The acc-sm tests are listed below, by CVS branch.

====b1_6 branch====

This branch includes 17 acc-sm test suites.

$ grep TESTSUITE_LIST acceptance-small.sh
export TESTSUITE_LIST="RUNTESTS SANITY DBENCH BONNIE IOZONE FSX SANITYN LFSCK
LIBLUSTRE REPLAY_SINGLE CONF_SANITY RECOVERY_SMALL REPLAY_OST_SINGLE
REPLAY_DUAL INSANITY SANITY_QUOTA PERFORMANCE_SANITY"

====b1_8_gate branch====

This branch includes 18 acc-sm test suites.

$ grep TESTSUITE_LIST acceptance-small.sh
export TESTSUITE_LIST="RUNTESTS SANITY DBENCH BONNIE IOZONE FSX SANITYN LFSCK
LIBLUSTRE REPLAY_SINGLE CONF_SANITY RECOVERY_SMALL REPLAY_OST_SINGLE
REPLAY_DUAL REPLAY_VBR INSANITY SANITY_QUOTA PERFORMANCE_SANITY"

====HEAD branch====

This branch includes 19 acc-sm test suites.

$ grep TESTSUITE_LIST acceptance-small.sh
export TESTSUITE_LIST="RUNTESTS SANITY DBENCH BONNIE IOZONE FSX SANITYN LFSCK
LIBLUSTRE REPLAY_SINGLE CONF_SANITY RECOVERY_SMALL REPLAY_OST_SINGLE
REPLAY_DUAL INSANITY SANITY_QUOTA SANITY_SEC SANITY_GSS
PERFORMANCE_SANITY"

To see the test cases in a particular acc-sm test, run:

$ grep run_ <test suite script>

For example, to see the last 3 test cases that comprise the SANITY test:

$ grep run_ sanity.sh | tail -3

run_test 130c "FIEMAP (2-stripe file with hole)"
run_test 130d "FIEMAP (N-stripe file)"
run_test 130e "FIEMAP (test continuation FIEMAP calls)"

==For each acc-sm test, what does it measure or show?==

The acc-sm test suite are described below.

;'''RUNTESTS'''
: A basic regression test with unmount/remount.

;'''SANITY'''
: Verifies Lustre operation under normal operating conditions.

;'''DBENCH'''
:Dbench benchmark for simulating N clients to produce the filesystem load.

;'''BONNIE'''
:Bonnie++ benchmark for creation, reading and deleting many small files

;'''IOZONE'''
:IOzone benchmark for generating and measuring a variety of file operations.

;'''FSX'''
:Filesystem exerciser.

;'''SANITYN'''
:Verifies operations from two clients under normal operating conditions.

;'''LFSCK'''
:Tests e2fsck and lfsck to detect and fix filesystm corruption.

;'''LIBLUSTRE'''
:Runs a test linked to a liblustre client library.

;'''REPLAY_SINGLE'''
:Verifies recovery after an MDS failure.

;'''CONF_SANITY'''
:Verifies various Lustre configurations (including wrong ones), where the system must behave correctly.

;'''RECOVERY_SMALL'''
:Verifies RPC replay after a communications failure (message loss).

;'''REPLAY_OST_SINGLE'''
:Verifies recovery after an OST failure.

;'''REPLAY_DUAL'''
:Verifies recovery from two clients after a server failure.

;'''INSANITY'''
:Tests multiple concurrent failure conditions.

;'''SANITY_QUOTA'''
:Verifies filesystem quotas.

==How do you get the acc-sm tests?==

The acc-sm test suite is stored in the lustre/tests sub-directory on each CVS branch (b1_6, b1_8, and HEAD).

==Do you have to run every acc-sm test?==

No. You can choose to run only specified acc-sm tests. Tests can be run either with or without the acceptance-sm.sh (acc-sm.sh) wrapper script. Here are several examples:

To only run the RUNTESTS and SANITY tests:

ACC_SM_ONLY=”RUNTESTS SANITY” sh acceptance-small.sh

To only run test_1 and test_2 of the SANITYN tests:

ACC_SM_ONLY=”SANITYN” ONLY=”1 2” sh acceptance-small.sh

To only run the replay-single.sh test and except (not run) the test_3* and test_4* tests:

ACC_SM_ONLY=”REPLAY_SINGLE” REPLAY_SINGLE_EXCEPT=”3 4” sh acceptance-small.sh

To only run conf-sanity.sh tests after #15 (without the acceptance-small.sh wrapper script):

CONF_SANITY_EXCEPT=”$(seq 15)“ sh conf-sanity.sh

==Do the acc-sm tests have to be run in a specific order?==

The test order is defined in the acceptance-small.sh script and in each test script. Users do not have to (and should not) do anything to change the order of tests.

==Who runs the acc-sm tests?==

Currently, the QE group and Lustre developers run acc-sm as the main test suite for Lustre testing. Acc-sm tests are run on YALA, the automated test system, with test reports submitted to Buffalo (a web interface that allows for browsing various Lustre test results). We welcome external contributions to the Lustre acc-sm test efforts – either of the Lustre code base or new testing platforms.

==What type of Lustre environment is needed to run the acc-sm tests? Is anything special needed?==

The default Lustre configuration for acc-sm testing is a single node setup with one MDS and two OSTs. All devices are loop-back devices. YALA, the automated test system, uses a non-default configuration.

To run the acc-sm test suite on a non-default Lustre configuration, you have to modify the default settings in the acc-sm configuration file, lustre/tests/cfg/local.sh. The configuration variables include mds_HOST, ost_HOST, OSTCOUNT, MDS_MOUNT_OPTS and OST_MOUNT_OPTS, among others.

To create your own configuration file, copy cfg/local.sh to cfg/my_config.sh:

cp cfg/local.sh cfg/my_config.sh

Edit the necessary variables in the configuration file (my_config.sh) and run acc-sm as: NAME=my_config sh acceptance-small.sh

==What are the steps to run acc-sm?==

There are two methods to run the acc-sm tests.

1. Check out a Lustre branch (b1_6, b1_8 or HEAD).

2. Change directory to lustre/tests:

cd lustre/tests

3. Build lustre/tests.

4. Run acc-sm on a local, default Lustre configuration (1 MGS/MDT, 1 OST and 1 client):

sh acceptance-small.sh 2>&1 | tee /tmp/output

- OR -

1. Install the lustre-tests RPM (available at lts-head:/var/cache/cfs/PACKAGE/rpm/lustre).

2. Change directory to lustre/tests:

cd /usr/lib/lustre/tests

3. Create your own configuration file and edit it for your configuration.

cp cfg/local.sh cfg/my_config.sh

4. Run acc-sm on a local Lustre configuration.

Here is an example of running acc-sm on a non-default Lustre configuration (MDS is sfire7, OST is sfire8, OSCOUNT=1, etc). In this example, only the SANITY test cases are being run.

ACC_SM_ONLY=SANITY mds_HOST=sfire7 ost8_HOST=sfire8 MDSDEV1=/dev/sda1
OSTCOUNT=1 OSTDEV1=/dev/sda1 MDSSIZE=5000000 OSTSIZE=5000000
MDS_MOUNT_OPTS="-o user_xattr" OST_MOUNT_OPTS=" -o user_xattr"
REFORMAT="--reformat" PDSH="pdsh -S -w" sh acceptance-small.sh

==What if I hit a failure on an acc-sm test?==

* If you regularly hit a failure in any of these tests, check if a bug has been reported on the failure or file a new bug if one has not yet been opened.
* If the bug prevents you from completing the tests, set the environment variables to skip the specific test(s) until you or someone else fixes them.
** For example, to skip sanity.sh subtest 36g and 65, replay-single.sh subtest 42, and all of insanity.sh set in your environment:
**:
**:: <pre><nowiki>
**::
**:: export SANITY_EXCEPT="36g 65"
**:: export REPLAY_SINGLE_EXCEPT=42
**:: export INSANITY=no
**:: </nowiki></pre>
**::
** You can also skip tests on the command line. For example, when running acceptance-small:
**:
**:: <pre><nowiki>
**:: SANITY_EXCEPT="36g 65" REPLAY_SINGLE_EXCEPT=42 INSANITY=no ./acceptance-small.sh
**:: </nowiki></pre>
** The test framework is very flexible, and it is a very easy "hands-off" way of running testing while you are doing other things, like coding.
** Questions/problems with the test framework should be emailed to the lustre-discuss mailing list, so all Lustre users can benefit from improving and documenting it.
* If you do not run the entire test suite regularly, you will have no idea whether a bug is added from your code or not, and you will waste a lot of time looking.

==How do you run acc-sm on a mounted Lustre system?==

To run acc-sm on a Lustre system that is already mounted, you need to use the correct configuration file (according to the mounted Lustre system) and run acc-sm as:

SETUP=: CLEANUP=: FORMAT=: NAME=<config> sh acceptance-small.sh

==How do you run acc-sm with and without reformat?==

By default, the acc-sm test suite does not reformat Lustre. If you want to reformat Lustre, run acc-sm with REFORMAT="--reformat":

REFORMAT="--reformat" sh acceptance-small.sh

If needed, you can specify WRITECONF="writeconf", and then run acc-sm with WRITECONF="writeconf":

WRITECONF="writeconf" sh acceptance-small.sh

==How do you run acc-sm in a Lustre configuration with several clients?==

The default configuration file for acc-sm is cfg/local.sh, which uses only one client (local). To use additional remote clients, specify the RCLIENTS list and use the cfg/ncli.sh configuration file (or your own copy of ncli configuration).

NAME=ncli RCLIENT=<space-separated list of remote clients> sh acceptance-small.sh

For example:

NAME=ncli RCLIENT="client2 client3 client11" sh acceptance-small.sh

==What is the SLOW variable and how is it used with acc-sm?==

The SLOW variable is used to run a subset of acc-sm tests. By default, the variable is set to SLOW=no, which causes some of the longer acc-sm tests to be skipped and acc-sm test run to complete in less than 2 hours. To run all of the acc-sm tests, set the variable to SLOW=yes:

SLOW=yes sh acceptance-small.sh

==What is the FAIL_ON_ERROR variable and how is it used with acc-sm?==

The FAIL_ON_ERROR variable is used to "stop" or "continue" running acc-sm tests after a test failure occurs. If the variable is set to "true" (FAIL_ON_ERROR=true), then acc-sm stops after test_N fails and test_N+1 does not run. If the variable is set to "false" (FAIL_ON_ERROR=false), then acc-sm continues after test_N fails and test_N+1 does run.

FALSE_ON_ERROR=false, by default, for the sanity, sanityn and sanity-quota tests. FALSE_ON_ERROR=true for the replay/recovery tests.

==What is the PDSH variable and how it is used with acc-sm?==

The PDSH variable is used to provide remote shell access. If acc-sm is run on a Lustre configuration with remote servers, specify PDSH like this:

PDSH="pdsh -S w" sh acceptance-small.sh

If the client has no access to the servers, you can run acc-sm without PDSH, but the tests which need PDSH access are skipped. A summary report is generated which lists the skipped tests.

==What is the CMD configuration for HEAD?==

For the HEAD branch, specify the MDSCOUNT variable (number of MDTs). By default, the variable is set to 1. If you have a Lustre configuration with several MDT nodes, they need to be specified in the configuration file as mds1_HOST, mds2_HOST, ...

By default, all of these variables are set to the mds_HOST value.

==What do we do with the acc-sm test results?==

Acc-sm results are sent to Buffalo, a web interface for Lustre test results. The default Buffalo display shows a summary of tests run on different hardware configurations for various CVS branches for the past 24 hours, with links to the various reports. For more information on reporting test results
to Buffalo, see [http://wiki.lustre.org/index.php?title=Buffalizing_Tests Buffalizing Tests].

If an acc-sm test fails, then the failure is investigated. If the investigation reveals there is a Lustre defect, a bug is opened in Bugzilla to fix the problem and also the acc-sm issue.

Coding Guidelines

2007-08-28T13:38:15Z

Scjody: Add autoconf info

All Lustre developers should follow the guidelines in this page very strictly to avoid problems during code merges later on. Please make the required changes to the default formatting rules in the editor you use to comply to the guidelines below.

== Guidelines ==

1. There should be no tabs in any Lustre or LNET files. The exceptions are libsysio (maintained by someone else), ldiskfs and kernel patches (also part of a non-CFS project).

2. Blocks should be indented by 8 spaces.

3. New files should contain the following along with the license boilerplate. This will cause vim and emacs to use spaces instead of tabs for indenting. If you use a different editor, it also needs to be set to use spaces for indening Lustre code.
<pre style="background:lightgrey;">
/* -*- mode: c; c-basic-offset: 8; indent-tabs-mode: nil; -*-
* vim:expandtab:shiftwidth=8:tabstop=8:
*/
</pre>
4. All lines should wrap at 80 characters. If it's getting too hard to wrap there, you probably need to break it up into more functions. In some cases, it is acceptable to remove a few spaces between function arguments to avoid overflowing onto the next line.

5. Don't have spaces or tabs on blank lines or at the end of lines. Find these with some regexps in your patch (grep, or in vim) before attaching it to bugzilla:

/[ \t]$/

6. Don't use "inline" unless you're doing something so performance critical that the function call overhead will make a difference -- in other words: never. It makes debugging harder.

All of our wrapping, parenthesis, brace placement, etc. rules are basically Linux kernel rules, which are basically K&R. For those of you in need of a refresher, great detail is provided below.

7. For Autoconf macros, follow the [http://www.gnu.org/software/autoconf/manual/html_node/Coding-Style.html style suggested in the autoconf manual].

== Great detail ==

a. When you wrap, the next line should start after the parenthesis:
<pre style="background:lightgrey;">
right:

variable = do_something_complicated(long_argument, longer_argument,
longest_argument(sub_argument,
foo_argument),
last_argument);

if (some_long_condition(arg1, arg2, arg3) < some_long_value &&
another_long_condition(very_long_argument_name,
another_long_argument_name) >
second_long_value) {
}

wrong:

variable = do_something_complicated(long_argument, longer_argument,
longest_argument(sub_argument, foo_argument),
last_argument);

if (some_long_condition(arg1, arg2, arg3) < some_long_value &&
another_long_condition(very_long_argument_name,
another_long_argument_name) >
second_long_value) {
}
</pre>
b. If you're wrapping put the operators at the end of the line, and if there are no parentheses indent 8 more:
<pre style="background:lightgrey;">
off = le32_to_cpu(fsd->fsd_client_start) +
cl_idx * le16_to_cpu(fsd->fsd_client_size);
</pre>
c. Binary and ternary (but not unary) operators should be separated from their arguments by one space.
<pre style="background:lightgrey;">
a++;
b |= c;
d = f > g ? 0 : 1;
</pre>
d. Function calls should be nestled against the parentheses, the parentheses should crowd the arguments, and one space after commas:
<pre style="background:lightgrey;">
right: do_foo(bar, baz);
wrong: do_foo ( bar,baz );
</pre>

e. All ''if'', ''for'', ''while'', etc. expressions should be separated by a space from the parenthesis, one space after the semicolons:
<pre style="background:lightgrey;">
for (a = 0; a < b; a++)
if (a < b || a == c)
while (1)
</pre>
f. Opening braces should be on the same line as the line that introduces the block, except for function calls. Closing braces get their own line, except for "else".
<pre style="background:lightgrey;">
int foo(void)
{
if (bar) {
this();
that();
} else if (baz) {
;
} else {
;
}
do {
cow();
} while (0);
}
</pre>
g. If one part of a compound ''if'' block has braces, all should.
<pre style="background:lightgrey;">
right:

if (foo) {
bar();
baz();
} else {
salmon();
}

wrong:

if (foo) {
bar();
baz();
} else
moose();
</pre>
h. When you make a macro, protect those who might call it by using do/while and parentheses; line up your backslashes:
<pre style="background:lightgrey;">
right:

#define DO_STUFF(a) \
do { \
int b = (a) + MAGIC; \
do_other_stuff(b); \
} while (0)

wrong:

#define DO_STUFF(a) \
{ \
int b = a + MAGIC; \
do_other_stuff(b); \
}
</pre>
i. If you nest preprocessor commands, use spaces to visually delineate:
<pre style="background:lightgrey;">
#ifdef __KERNEL__
# include <goose>
# define MOOSE steak
#else
# include <mutton>
# define MOOSE prancing
#endif
</pre>
j. For very long #ifdefs include the conditional with each #endif to make it readable:
<pre style="background:lightgrey;">
#ifdef __KERNEL__
# if LINUX_VERSION_CODE >= KERNEL_VERSION(2,5,0)
/* lots
of
stuff */
# endif /* KERNEL_VERSION(2,5,0) */
#else /* !__KERNEL__ */
# if HAVE_FEATURE
/* more
* stuff */
# endif
#endif /* __KERNEL__ */
</pre>
k. Comments should have the leading '''/*''' on the same line as the comment, and the trailing '''*/''' at the end of the last comment line. Intermediate lines should start with a '''*''' aligned with the first line's '''*''':
<pre style="background:lightgrey;">
/* This is a short comment */

/* This is a multi-line comment. I wish the line would wrap already,
* as I don't have much to write about. */
</pre>
l. Function declarations absolutely should NOT go into .c files, unless they are forward declarations for static functions that can't otherwise be moved before the caller. Instead, the declaration should go into the most "local" header available (preferrably *_internal.h for a given piece of code).

m. Structure and constant declarations should not be declared in multiple places. Put the struct into the most "local" header possible. If it is something that is passed over the wire it needs to go into lustre_idl.h, and needs to be correctly swabbed when the RPC message is unpacked.

n. The types and printf/printk formats used by Lustre code are:
<pre style="background:lightgrey;">
__u64 LPU64/LPX64/LPD64 (unsigned, hex, signed)
size_t LPSZ (or cast to int and use %u / %d)
__u32/int %u/%x/%d (unsigned, hex, signed)
(unsigned) long long %llu/%llx/%lld
loff_t %lld after a cast to long long (unfortunately)
</pre>
----
*'''[http://wiki.lustre.org/index.php?title=Front_Page FrontPage]'''

Coding Guidelines

2007-06-01T02:11:48Z

Scjody: Coding Guide lines moved to Coding Guidelines: Remove extraneous space in title

All Lustre developers should follow the guidelines in this page very strictly to avoid problems during code merges later on. Please make the required changes to the default formatting rules in the editor you use to comply to the guidelines below.

== Guidelines ==

1. There should be no tabs in any files.

2. Blocks should be indented by 8 spaces.

3. New files should contain the following along with the license boilerplate. This will cause vim and emacs to use spaces instead of tabs for indenting. If you use a different editor, it also needs to be set to use spaces for indening Lustre code.

/* -*- mode: c; c-basic-offset: 8; indent-tabs-mode: nil; -*-
* vim:expandtab:shiftwidth=8:tabstop=8:
*/

4. All lines should wrap at 80 characters. If it's getting too hard to wrap there, you probably need to break it up into more functions. In some cases, it is acceptable to remove a few spaces between function arguments to avoid overflowing onto the next line.

5. Don't have spaces or tabs on blank lines or at the end of lines.

6. Don't use "inline" unless you're doing something so performance critical that the function call overhead will make a difference -- in other words: never. It makes debugging harder.

All of our wrapping, parenthesis, brace placement, etc. rules are basically Linux kernel rules, which are basically K&R. For those of you in need of a refresher, great detail is provided below.

== Great detail ==

a. When you wrap, the next line should start after the parenthesis:

right:

variable = do_something_complicated(long_argument, longer_argument,
longest_argument(sub_argument,
foo_argument),
last_argument);

if (some_long_condition(arg1, arg2, arg3) < some_long_value &&
another_long_condition(very_long_argument_name,
another_long_argument_name) >
second_long_value) {
}

wrong:

variable = do_something_complicated(long_argument, longer_argument,
longest_argument(sub_argument, foo_argument),
last_argument);

if (some_long_condition(arg1, arg2, arg3) < some_long_value &&
another_long_condition(very_long_argument_name,
another_long_argument_name) >
second_long_value) {
}

b. If you're wrapping put the operators at the end of the line, and if there are no parentheses indent 8 more:

off = le32_to_cpu(fsd->fsd_client_start) +
cl_idx * le16_to_cpu(fsd->fsd_client_size);

c. Binary and ternary (but not unary) operators should be separated from their arguments by one space.

a++;
b |= c;
d = f > g ? 0 : 1;

d. Function calls should be nestled against the parentheses, the parentheses should crowd the arguments, and one space after commas:

right: do_foo(bar, baz);
wrong: do_foo ( bar,baz );

e. All ''if'', ''for'', ''while'', etc. expressions should be separated by a space from the parenthesis, one space after the semicolons:

for (a = 0; a < b; a++)
if (a < b || a == c)
while (1)

f. Opening braces should be on the same line as the line that introduces the block, except for function calls. Closing braces get their own line, except for "else".

int foo(void)
{
if (bar) {
this();
that();
} else if (baz) {
;
} else {
;
}
do {
cow();
} while (0);
}

g. If one part of a compound ''if'' block has braces, all should.

right:

if (foo) {
bar();
baz();
} else {
salmon();
}

wrong:

if (foo) {
bar();
baz();
} else
moose();

h. When you make a macro, protect those who might call it by using do/while and parentheses; line up your backslashes:

right:

#define DO_STUFF(a) \
do { \
int b = (a) + MAGIC; \
do_other_stuff(b); \
} while (0)

wrong:

#define DO_STUFF(a) \
{ \
int b = a + MAGIC; \
do_other_stuff(b); \
}

i. If you nest preprocessor commands, use spaces to visually delineate:

#ifdef __KERNEL__
# include <goose>
# define MOOSE steak
#else
# include <mutton>
# define MOOSE prancing
#endif

j. For very long #ifdefs include the conditional with each #endif to make it readable:

#ifdef __KERNEL__
# if LINUX_VERSION_CODE >= KERNEL_VERSION(2,5,0)
/* lots
of
stuff */
# endif /* KERNEL_VERSION(2,5,0) */
#else /* !__KERNEL__ */
# if HAVE_FEATURE
/* more
* stuff */
# endif
#endif /* __KERNEL__ */

k. Comments should have the leading '''/*''' on the same line as the comment, and the trailing '''*/''' at the end of the last comment line. Intermediate lines should start with a '''*''' aligned with the first line's '''*''':
/* This is a short comment */

/* This is a multi-line comment. I wish the line would wrap already,
* as I don't have much to write about. */

l. Function declarations absolutely should NOT go into .c files, unless they are forward declarations for static functions that can't otherwise be moved before the caller. Instead, the declaration should go into the most "local" header available (preferrably *_internal.h for a given piece of code).

m. Structure and constant declarations should not be declared in multiple places. Put the struct into the most "local" header possible. If it is something that is passed over the wire it needs to go into lustre_idl.h, and needs to be correctly swabbed when the RPC message is unpacked.

n. The types and printf/printk formats used by Lustre code are:

__u64 LPU64/LPX64/LPD64 (unsigned, hex, signed)
size_t LPSZ (or cast to int and use %u / %d)
__u32/int %u/%x/%d (unsigned, hex, signed)
(unsigned) long long %llu/%llx/%lld
loff_t %lld after a cast to long long (unfortunately)

----
*'''[http://wiki.lustre.org/index.php?title=Front_Page FrontPage]'''

Lustre Documentation

2007-05-01T23:39:45Z

Scjody: Fix PDF links

= Lustre Documentation =
== Lustre Manual ==

=== Lustre 1.6 ===
* '''Lustre Manual - HTML '''[https://mail.clusterfs.com/wikis/attachments/LustreManual1_6.html Lustre 1.6 manual V1.1]
* '''Lustre Manual - PDF'''
. ''''''[https://mail.clusterfs.com/wikis/attachments/LustreManual1_6.pdf Lustre 1.6 manual V1.1(A4)]
. ''''''[https://mail.clusterfs.com/wikis/attachments/LustreManual-letter1_6.pdf Lustre 1.6 manual V1.1(Letter)]
* '''Lustre Manual Changelog '''[https://mail.clusterfs.com/wikis/attachments/manual-changelog1_6.html Lustre Manual-Change log]
* '''[http://wiki.lustre.org/index.php?title=Mount_Conf MountConf wiki]''' - A quick reference for those familiar with older versions of Lustre

=== Lustre 1.4.8 ===
* '''Lustre Manual - HTML '''[https://mail.clusterfs.com/wikis/attachments/LustreManual.html Lustre 1.4.8 manual V1.37]
* '''Lustre Manual - PDF'''
. ''''''[https://mail.clusterfs.com/wikis/attachments/LustreManual37.pdf Lustre 1.4.8 manual V1.37(A4)]
* '''Lustre Manual Changelog '''[https://mail.clusterfs.com/wikis/attachments/manual-changelog.html Lustre Manual-Change log]

== Interim Lustre 1.8 Documentation ==
* '''[http://wiki.lustre.org/index.php?title=Kerb_Lustre KerbLustre]''' - The 1.8 interim Lustre documentation (Kerberos....)

== Older Documentation ==
This documentation will be incorporated into the manual. Until that is done you may find useful bits and pieces here.

* '''Frequently Asked Questions'''
. [http://www.clusterfs.com/faq.html Lustre Faq]
* '''Knowledge Base'''
. [http://bugzilla.lustre.org/showdependencytree.cgi?id=2374 Lustre Knowledge Base] - questions and answers
* '''Configuration'''
* '''[http://wiki.lustre.org/index.php?title=Lustre_Howto LustreHowto]''' - A guide to getting Lustre cluster started.
* '''[http://wiki.lustre.org/index.php?title=Lustre_LDAP LustreLDAP]''' - A guide for using LDAP with Lustre.
* '''[http://wiki.lustre.org/index.php?title=Lustre_Wizard LustreWizard]''' - The ''Lustre wizard'' or ''lwizard'' is a utility that helps with creation of configuration file for a cluster through asking some simple questions.
* '''[http://wiki.lustre.org/index.php?title=Lustre_Failover LustreFailover]''' - Managing failover
* '''[http://wiki.lustre.org/index.php?title=Filesystem_Backup FilesystemBackup]''' - How to back up a Lustre filesystem
* '''[http://wiki.lustre.org/index.php?title=Fsck_Suppor FsckSupport]''' - The lustre fsck tool and Lustre-patched e2fsck (extents, large inode/EA support)
* '''Filesystem Tuning'''
* [https://mail.clusterfs.com/wikis/attachments/LustreManual.html#Chapter_III-2._LustreProc LustreProc] - A guide on the '''proc''' tunables parameters for Lustre and their usage. It describes several of the ''proc'' tunables including those that effect the client's RPC behaviour and prepare for a substantial reorganization of proc entries.
* '''[https://mail.clusterfs.com/wikis/attachments/LustreManual.html#3.2_DDN_Tuning LustreDdnTuning]''' - A brief guide on tuning DDN S2A 8500 (and maybe 9500) storage optimally for Lustre