1.1

1. Copyright Page: Changed copyright year from 2004 to 2005, first draft to draft 1.1

2. On page 14: As Nathan requested, heading changed to upgrading a cluster live

3. Bookmarks generated by OpenOffice automatically, actually they are generated from table of contents of OpenOffice file.

Eli Li

12/02/05

1.2

1. Cover and Copyright Page: Changed the draft version from 1.1 to 1.2. Changed the date from Dec. 2,2005 to Dec. 9, 2005.

2. Changed the color of all titles from red to black as per the communication between Dr. Braam and Mr. Bojanic.

3. Completed basic editing for Chapter I.

4. Incorporated some of the changes suggested by Dr. Braam and Mr. Bojanic in Chapter I and Chapter II-1.

SPSOFT

12/09/05

1.3

1. Reformatted as per the ORA_Template.

2. Incorporated Dr. Braam and Mr. Bojanic's suggestions wherever possible.

3. Included Eli' ChangeLog and Updated mine.

4. Cover and Copyright page: Updated the date and version.

5. Numbered all the chapters.

6. Generated the TOC.

SPSOFT

12/13/05

1.4

1. Cover Page and Copyright Page: Changed the date and version

2. Drafted and Inserted Part 1 - Chapter 1 A Cluster with Lustre as per ManualProject.mpp

3. ChangeLog: Updated the ChangeLog.

SPSOFT

12/16/05

1.5

1. Transformed in to the new format

SPSOFT

12/19/05

1.6

1. Changed the version no. and date on Cover Page and Copyright Page

2. Added the Parts 2-5 and their headings making the skeleton complete

3. Added the contents under 1.2 Lustre Server Nodes

4. Enhanced the format further for the complete document and also for the Table of Contents

5. Created 5 diagrams in Visio and pasted them at appropriate places

6. Inserted some of the contents from LustreManual.pdf (developed by Eli) till page 20

7. Converted the ChangeLog page into the VersionLog

SPSOFT

12/23/05

1.7

1. Inserted a new Cover Page

2. Changed the version no. and date on Cover Page, Copyright Page and VersionLog

3. Deleted 2 previous versions and transformed version 1.4 into the latest format and prepared a Master Document

4. Incorporated majority of the suggestions from Dr. Braam-Mr. Bojanic conversation

SPSOFT

12/28/05

1.8

1. Changed the version no. and date

2. Created a simpler template and format

3. Reformatted all the chapters in the new template and format

4. Incorporated all the review comments received from Cliff

5. Inserted all the new chapters created by SPSOFT at appropriate places

6. Generated a new TOC

SPSOFT

01/31/06

1.9

1. Changed the version no. and date

2. Drafted new Task / Feature Lists

3. Introduced Glossary

4. Introduced various page styles to reflect proper headers and footers

5. Created the master document in a new way as directed in the Writer Manual.

6. Generated an alphabetical Index.

SPSOFT

02/10/06

1.10

1. Changed the version no. and date

2. Updated Task / Feature List

3. Changed the Glossary as given by Peter

4. Changed the entries in TOC and Bookmarks

5. Reformatted the Index

SPSOFT

02/13/06

1.11

1. Changed the version no. and date

2. Updated the glossary as resent by Dr. Braam

3. Changed the attributes of the tables and figures, and their titles for better HTML generation

SPSOFT

02/20/06

1.12

1. Change the version no. and date

2. Changed Part I - Chapter 1 as per Peter's instructions.

3. Replaced POSIX ACLs section in Part IV - Chapter 1.

4. Changed the attributes of figures and tables for better HTML generation

5. Updated the Task and Feature Lists as the references to TOC changed

6. Updated the Index and TOC

7. Enhanced the styles of Headings for better HTML generation

SPSOFT

02/23/06

1_4_6_manv 1_13

1. Changed the version

2. Mentioned about the registered trademark on the Copyright page

3. Introduced 'About the Manaul' page

4. Moved the Version Log to the end

5. Reshuffled the contents and chapters as instructed by Peter

6. Edited the contents as instructed by Peter

7. Increased the Task List and the Feature List, and introduced Cross references in both of them

8. Hyper-linked the TOC

SPSOFT

03/20/06

Version 1.4.6.1-man-v14

1. Changed the version and date

2. Improved the following chapters -

   I. Chapter II - 4. Failover

   II. Chapter III - 1. Lustre I/O Kit

   III. Chapter IV - 2. Striping and I/O

   IV. Chapter V - 1. User Utilities (man1)

3. Put the headers and footers in place

4. Incorporated Cameron Harr's comments

5. Inserted the text regarding Trademarks on the Copyright page

6. Changed the cover page

SPSOFT

03/24/06

Version 1.4.6.1-man-v15

1. Changed the version and date

2. Improved following Chapters as per Nathan's suggestions -

   I. Chapter I-2 - Understanding Lustre Networking

   II. Chapter II-1 - Configuring Lustre Network

   III. Chapter II-2 - Configuring Lustre-Examples

   IV. Chapter II-3 - More Complicated Configurations

   V. Chapter II-4 - Failover

SPSOFT

03/28/06

Version 1.4.6.1-man-v16

1. Changed the version and date

2. Inserted a new title called Portals LND in the Chapter IV - 3. Config Files and Module Parameters

3. Reformatted the Feature List to have the correct reference to Portals LND

SPSOFT

03/31/06

Version 1.4.6.1-man-v17

1. Changed the version and date

2. Improved following chapters -

   I. Chapter I-1 - A Cluster With Lustre

   II. Chapter II-1 - Configuring Lustre Network

   III. Chapter II-5 - Configuring Quotas

   IV. Chapter IV-3 - Lustre Security

   V. Chapter V-1 - User Utilities

   VI. Chapter V-3 - Config Files and Module Parameters

   VII. Chapter V-4 - System Configuration Utilities

SPSOFT

04/07/06

Version 1.4.6.1-man-v18

1. Changed the version and date

2. Introduced the chapter -

   Part2-chapter1 - Prerequisites

3. Improved following chapters -

   I. Chapter II-3 - More Complicated Configurations

   II. Chapter II-4 - Failover

SPSOFT

04/14/06

Version 1.4.6.1-man-v19

1. Changed the version and date

2. Introduced the chapter - Part3 - chapter2 - LustreProc

3. Incorporated Tom Fenton's review comments, which included corrections of wrong or misleading content, grammar or formatting mistakes, etc.

4. Updated the LMC commands in Chapter V-4 System Configuration Utilities

SPSOFT

04/21/06

Version 1.4.6.1-man-v20

1. Changed the version and date

2. Changed the contents under Chapter II-6 Configuring Quotas - 6.1.2 Remounting the File Systems

3. Changed the Title of the Chapter IV-2 - Striping and I/O to Chapter IV-2 - Striping and Other I/O Options

4. Changed the contents under Chapter II-1 - Prerequisites - 1.3.2 Building Lustre

5. Added the information about how to install Lustre RPMs while upgrading Lustre Chapter II-7 - Upgrading from 1.4.5 in the section '7.1.2 Upgrade a Cluster Using Shutdown'

6. Added Chapter II-8 - RAID

SPSOFT

04/28/06

Version 1.4.6.1-man-v21

1. Changed the version and date.

2. Included information on I/O, Locking and Debug Support in Chapter III-2 - LustreProc.

3. Included 'Conventions for Command Syntax' in the front matter.

SPSOFT

Version 1.4.6.1-man-v22

1. Changed the version and date

2. Corrected all the links that appear in the manual

3. Converted the utility commands from tabular format to textual format for ease of generating man pages directly

4. Modified the section of fstab from labels to MDS string in Chapter II-6 - Configuring Quotas

5. Added more commands for stopping LNET in Chapter II-2 - Configuring the Lustre Network

SPSOFT

1. Changed the version and date

2. Added Chapter III-3 - Lustre Tuning

1. Changed the version and date

2. Added Chapter IV-4 -Other Lustre Operating Tips

3. Inserted four sample outputs in Chapter V-1 User Utilities

Version 1.4.6.1-man-v25

1. Changed the version and date

2. Revised the chapters below as per Cliff's instructions -

   Chapter III - 2 - Lustre Proc

   Chapter III - 3 - Lustre Tuning

   Chapter IV - 4 - Other Lustre Operating Tips

3. In Chapter II - 4 - More Complicated Configurations, added notes on LNET lines and -add net option

4. In Chapter V - 4 - System Configuration Utilities, added notes about mountfs option in LMC and LCONF.

SPSOFT

Version 1.4.6.1-man-v26

1. Changed the version and date

2. Edited the section on LCONF utility in Chapter V - 4 - System Configuration Utilities

3. Deleted the material on Proc elements from Chapter III - 2 Lustre Proc and created a new chapter in Part V - Reference as Chapter V - 5 Lustre Proc Elements

4. Added Chapter V - 6 Elevator

5. Updated Chapter II - 6 Configuring Quotas

SPSOFT

06/12/06

Version 1.4.6.1-man-v27

1. Changed the version and date

2. Deleted Chapter V - 6 Elevator and introduced the information in Chapter II - 1 Prerequisites as 1.4.3 Proper Kernel I/O Elevator

3. Included the 'lfs df' command in the Synopsis section of Chapter V - 1 User Utilities

4. Edited the notes on 'asyncdel' in Chapter V - 4 System Configuration Utilities

5. Introduced information on Liblustre Network parameters in 2.2.2 Module Parameters under Chapter II - 2  Configuring Lustre Network

06/16/06

1. Changed the version and date

2. Removed 1.3.2 obd survey from Chapter III - 1 Lustre I/O Kit

3. Added information of LNET_ROUTES setting of Liblustre client in section 2.2.2 Module Parameters of Chapter II - 2 Configuring Lustre Network

4. Inserted the information on order of LNET lines in modprobe.conf in section 4.1.1 Modprobe.conf of Chapter II - 4 More Complicated Configurations

1. Changed the version and date

2. Changed the format of examples and the reference of 'RHEL' to 'Red Hat Enterprise Linux v3 Update 3' in Chapter II - 1 Prerequisites - 1.4.3 Proper Kernel I/O Elevator

3. Added a note on loopback interface in Chapter II - 2 Configuring Lustre Network - 2.2.2 Module Parameters

4. Gave the full pathname instead of * in Chapter III - 2 Lustre Proc - 2.2.1 Client I/O RPC Stream Tunables

5. Changed 'portals' to 'lnet' in Chapter III - 2 Lustre Proc - 2.4 Debug Support

6. Corrected the setstripe command in 1.1.1 Synopsis and elaborated information on Stripe Size, Stripe Start and Stripe Count in 1.1.3 Examples in the Chapter V - 1 User Utilities

7. Added a note on loopback interface in Chapter V - 3 Config Files and Module Parameters - 3.2.1.1 Network Topology

8. Added a sample output for invoking lconf on the OST node in Chapter V - 4 System Configuration Utilities - 4.2.3 Examples

1. Changed the version and date

2. Added Chapter II - 9 Bonding

3. Added sample output of peer list for lctl in Chapter V - 4 System Configuration Utilities.

4. Added sample output of setstripe and getstripe for lfs in Chapter V - 1 User Utilities.

5. Added a note on the pathnames of /proc variables under 2.2.1 Client I/O RPC Stream Tunables in Chapter III - 2 LustreProc. In the same chapter changed 'portals' to 'lnet' all over and '/proc/fs/ldiskfs/xxxx/mb_history' to '/proc/fs/ldiskfs/loop0/mb_history' in 2.2.4 mballoc history.

6. Deleted 2 hanging lines and corrected the spelling of passive in 5.1.3 Heartbeat in Chapter II - 5 Failover

1. Changed the version and date
2. Added Chapter II - 2. Lustre Installation
3. Edited Chapter II - 1. Prerequisites for proper flow of information considering insertion of Chapter II - 2. Lustre Installation

1. Changed the version and date
2. Edited Chapter I - 1. A Cluster With Lustre to improve readability
3. Added information on LNET lnd interface number indexing in 3.2.2 Module Parameters in Chapter II - 3. Configuring Lustre Network
4. Added 9.2 Disk Performance Measurement in Chapter II - 9. RAID
5. Added Chapter V - 6. System Limits

1. Changed the version and date
2. Replaced 'Portals' with LNET in section 2.3 of Chapter I - 2. Understanding Lustre Networking
3. Upgraded the output of llmount.sh in section 2.2.1, and changed the format of the example in section 2.4.2 in Chapter II - 2. Lustre Installation

4. Added sample graphs of Read and Write Performance as section 9.2.1 in Chapter II - 9. RAID

   Also, edited point no. 10 of section 9.2 for correct description of mdadm in Chapter II - 9. RAID

5. Explained how to set the debug level and added a note in section 2.4 of Chapter III - 2. Lustre Proc
6. Replaced 'Portals' with LNET in section 4.3.2 of Chapter V - 4. System Configuration Utilities

1. Changed the version and date
2. Updated Chapter V - 1. User Utilities as per the latest LFS man page
3. Added a table describing various networks and supported software stacks under section 3.2.1.1 Network Topology in Chapter V - 3. Config Files and Module Parameters

Version 1.4.7.1-man-v35

1. Changed the version and date
2. Changed the Title from “Lustre 1.4.6 Operations Manual” to “Lustre 1.4.7 Operations Manual”
3. Changed the mentions of 1.4.6 to 1.4.7 wherever applicable
4. Updated following chapters -

1. Chapter I - 1. A Cluster with Lustre
2. Chapter I - 2. Understanding Lustre Networking
3. Chapter II - 1. Prerequisites
4. Chapter II - 2. Lustre Installation (Corrected information in “Patch Series Selection”)
5. Chapter II - 3. Configuring Lustre Network
6. Chapter II - 4. Configuring Lustre - Examples
7. Chapter II - 5. More Complicated Configurations
8. Chapter II - 6. Failover (Included “Instructions to setup Failover with Heartbeat V1 and V2”)
9. Chapter II - 7. Configuring Quotas
10. Chapter II - 8. Upgrading from 1.4.5
11. Chapter II - 9. RAID
12. Chapter III - 1. Lustre I/O Kit
13. Chapter III - 2. Lustre Proc (full path names)
14. Chapter III - 3. Lustre Tuning (Added information on OST threads)
15. Chapter IV - 2. Striping and I/O (Introduced information on “MDS Space Utilization”)
16. Chapter IV - 4. Other Lustre Operating Tips
17. Chapter V - 1. User Utilities (Edited description of “find”)
18. Chapter V - 4. System Configuration Utilities (Changed the references of Portals to LNET wherever applicable)

Version 1.4.7.1-man-v36

1. Changed the version and date

2. Moved the chapter on Upgrading from 1.4.5 to a new section called Appendices. As a result, Part II - Chapter 9. RAID became Part II – Chapter 8. RAID and Part II – Chapter 10. Bonding became Part II - Chapter 9. Bonding

3. Improved PartIV - Chapter 2. Striping and Other I/O Options for information on following -

• aggregate striping

• formatting the MDS for better space utilization

• lfs setstripe

4. Improved Part III. Lustre Tuning, Monitoring and Troubleshooting

5. Improved Part IV. Lustre for Users

6. Corrected Section 8.2 of Part II - Chapter 8. RAID for information on sgpdd survey and sgp_dd tool

7. Added information on I/O scheduler in Part II - Chapter 1. Prerequisites in the section 1.3.3 Proper Kernel I/O Elevator

8. Introduced obdfilter survey in Part III - Chapter 1. Lustre I/O Kit, section 1.2.2 obdfilter_survey

9. Documented exceptional failover conditions in Part II - Chapter 6. Failover in section 6.7 Considerations With Failover Software and Solutions. Also added information on errors=panic option in section 6.4 Configuring MDS and OSTs for Failover

10. Removed Part V - Chapter 5. Lustre Proc Elements, and as a result Part 5 - Chapter 6. System Limits is now Part V - Chapter 5. System Limits

11. Corrected Part V - Chapter 3. Lustre Security for information on ACLs

12. Improved information on mballoc tunables in section 2.2.4 mballoc History of Part III - Chapter 2. Lustre Proc

13. Replaced lfs examples with correct tested examples, corrected several other errors and removed XML example in Part V - Chapter 1. User Utilities

14. Added a new chapter: Part III - Chapter 4. Lustre Troubleshooting and Tips

15. Added directives for IB and re-written the section 5.1.1 Modprobe.conf of Part II - Chapter 5. More Complicated Configurations

16. Added section 3.2.4 Downed Routers in Part II - Chapter 3. Configuring the Lustre Network. Also added information on router checker in section 3.2.3 Module Parameters - Routing

17. Improved Part V - Chapter 3. Config Files and Module Parameters for information on LNET configurations

Version 1.4.8.1-man-v37

1. Changed the version and date

2. Upgraded the manual to Lustre version 1.4.8

3. Added information on how to set SCSI IO sizes in Part III - Chapter 4.

4. Added information on sgpdd survey in Part III - Chapter 1.

5. Updated the Quilt section in Part II - Chapter 2.

6. Updated information on ksocklnd defaults in Part V - Chapter 3.

7. Added LNET information in Part III - Chapter 2.

8. Added a section on Large-scale Tuning in Part III - Chapter 3.

9. Added information on the Mount command in Part V - Chapter 1.

10. Updated entire section on lctl in Part V - Chapter 4 as per the obsolete lctl commands.

SPSOFT

02/09/2007