ZFS and Lustre - Revision history

Sbarthel at 18:26, 10 September 2010

2010-09-10T18:26:38Z

← Older revision		Revision as of 11:26, 10 September 2010
Line 1:		Line 1:
	<small>''(Updated: ~~Dec 2009~~)''</small>		<small>''(Updated: Sep 2010)''</small>
	__TOC__		__TOC__
	The Lustre™ node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.		The Lustre™ node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.

Adilger at 14:12, 5 September 2010

2010-09-05T14:12:37Z

← Older revision		Revision as of 07:12, 5 September 2010
Line 16:		Line 16:
	In Lustre, ZFS checksumming will be done by the Lustre client on the application node. This will detect any data corruption introduced into the network between the application node and the disk drive in the Lustre storage system.		In Lustre, ZFS checksumming will be done by the Lustre client on the application node. This will detect any data corruption introduced into the network between the application node and the disk drive in the Lustre storage system.

	~~Testing ZFS~~ with ~~Lustre~~ has resulted in the detection of previously unknown ~~problems~~ in network cards. These cards silently introduced data corruption that went undetected without the use of ~~ZFS~~. It should be noted that the checksum computation does consume some processor cycles, approximately 1 GHz of CPU time to process 500 MB/sec of I/O.		Previous testing of Lustre with network checksums has resulted in the detection of previously unknown corruption in network cards. These cards silently introduced data corruption that went undetected without the use of checksums. It should be noted that the checksum computation does consume some processor cycles, approximately 1 GHz of CPU time to process 500 MB/sec of I/O.

	''An implementation note:'' Previously, ZFS support was being developed and tested with a user space implementation of the ZFS DMU. Currently, we are running the DMU in kernel space. Also, the Lustre DMU code is almost entirely common with the Solaris version of ZFS, so Lustre support for ZFS will closely parallel the Solaris release of ZFS.		''An implementation note:'' Previously, ZFS support was being developed and tested with a user space implementation of the ZFS DMU. Currently, we are running the DMU in kernel space. Also, the Lustre DMU code is almost entirely common with the Solaris version of ZFS, so Lustre support for ZFS will closely parallel the Solaris release of ZFS.
Line 23:		Line 23:
	* ''Self-healing capability'' - In a mirrored or RAID configuration, ZFS not only detects data corruption, but it automatically corrects the bad data.		* ''Self-healing capability'' - In a mirrored or RAID configuration, ZFS not only detects data corruption, but it automatically corrects the bad data.
	* ''Improved administration'' - Because ZFS detects and reports data corruption on all read and write errors at the block level, it is easier for system administrators to quickly identify which hardware components are corrupting data. ZFS also has very easy-to-use command-line administration utilities.		* ''Improved administration'' - Because ZFS detects and reports data corruption on all read and write errors at the block level, it is easier for system administrators to quickly identify which hardware components are corrupting data. ZFS also has very easy-to-use command-line administration utilities.
	* ''SSD support'' - ZFS supports the addition of high-speed I/O devices, such as SSDs, to the storage pool. The Read Cache Pool or L2ARC acts as a cache layer between memory and the disk. This support can substantially improve the performance of random read operations. SSDs can also be used to improve synchronous write performance, by adding them to the pool as log devices. You can add as many ~~SDDs~~ to your storage pool as you need to increase your read cache size and IOPS, your synchronous write IOPS, or both.		* ''SSD support'' - ZFS supports the addition of high-speed I/O devices, such as SSDs, to the storage pool. The Read Cache Pool or L2ARC acts as a cache layer between memory and the disk. This support can substantially improve the performance of random read operations. SSDs can also be used to improve synchronous write performance, by adding them to the pool as log devices. You can add as many SSDs to your storage pool as you need to increase your read cache size and IOPS, your synchronous write IOPS, or both.
	* ''Scalability'' - ZFS is a 128-bit file system. This means that current restrictions on maximum-size file systems for a single ~~MDS~~ or OST, maximum stripe size, ~~and maximum~~ size of a single file will be removed. ZFS support will also remove the current 8 TB limitation on LUNs.		* ''Scalability'' - ZFS is a 128-bit file system. This means that current restrictions on maximum-size file systems for a single MDT or OST, maximum stripe size, andmaximum size of a single file will be removed. ZFS support will also remove the current 16 TB limitation on LUNs.

	For more general information about ZFS, see [[ZFS Resources]].		For more general information about ZFS, see [[ZFS Resources]].

Sbarthel at 19:25, 22 February 2010

2010-02-22T19:25:36Z

← Older revision		Revision as of 12:25, 22 February 2010
Line 1:		Line 1:
			<small>''(Updated: Dec 2009)''</small>
	__TOC__		__TOC__
	The Lustre™ node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.		The Lustre™ node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.

Docadmin at 20:39, 17 December 2009

2009-12-17T20:39:26Z

← Older revision		Revision as of 13:39, 17 December 2009
Line 1:		Line 1:
			__TOC__
	The Lustre™ node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.		The Lustre™ node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.

Docadmin at 20:47, 16 December 2009

2009-12-16T20:47:04Z

← Older revision		Revision as of 13:47, 16 December 2009
Line 1:		Line 1:
	Lustre™~~'s current~~ node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.		The Lustre™ node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.

	Lustre supporting ZFS will offer a number of advantages, such as improved data integrity with transaction-based, copy-on-write operations and an end-to-end checksum on every block.		Lustre supporting ZFS will offer a number of advantages, such as improved data integrity with transaction-based, copy-on-write operations and an end-to-end checksum on every block.

Docadmin: Protected "ZFS and Lustre" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

2009-12-16T20:44:57Z

Protected "ZFS and Lustre" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

← Older revision	Revision as of 13:44, 16 December 2009
(No difference)

Docadmin at 20:44, 16 December 2009

2009-12-16T20:44:40Z

← Older revision		Revision as of 13:44, 16 December 2009
Line 1:		Line 1:
	~~Lustre~~'s current node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.		Lustre™'s current node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.

	Lustre supporting ZFS will offer a number of advantages, such as improved data integrity with transaction-based, copy-on-write operations and an end-to-end checksum on every block.		Lustre supporting ZFS will offer a number of advantages, such as improved data integrity with transaction-based, copy-on-write operations and an end-to-end checksum on every block.

Rcorreia: Corrected disk filesystem name

2009-12-03T23:01:13Z

Corrected disk filesystem name

← Older revision		Revision as of 16:01, 3 December 2009
Line 1:		Line 1:
	Lustre's current node file system ''~~ext3~~'' is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.		Lustre's current node file system ''ldiskfs'' (based on ext3/ext4) is limited to an 8 TB maximum file system size and offers no guarantee of data integrity. To improve the reliability and resilience of the underlying file system on the OSS and MDS components, Lustre will add ZFS support.

	Lustre supporting ZFS will offer a number of advantages, such as improved data integrity with transaction-based, copy-on-write operations and an end-to-end checksum on every block.		Lustre supporting ZFS will offer a number of advantages, such as improved data integrity with transaction-based, copy-on-write operations and an end-to-end checksum on every block.

Docadmin at 21:59, 3 December 2009

2009-12-03T21:59:17Z

← Older revision		Revision as of 14:59, 3 December 2009
Line 10:		Line 10:
	* DMA parity errors between the array and server memory or from the driver, since the checksum validates data inside the array.		* DMA parity errors between the array and server memory or from the driver, since the checksum validates data inside the array.
	* Driver errors, where data winds up in the wrong buffer inside the kernel.		* Driver errors, where data winds up in the wrong buffer inside the kernel.
	* Accidental overwrites, such as swapping to a live file system~~, are all detected by ZFS~~.		* Accidental overwrites, such as swapping to a live file system.

	In Lustre, ZFS checksumming will be done by the Lustre client on the application node. This will detect any data corruption introduced into the network between the application node and the disk drive in the Lustre storage system.		In Lustre, ZFS checksumming will be done by the Lustre client on the application node. This will detect any data corruption introduced into the network between the application node and the disk drive in the Lustre storage system.

Rcorreia: Updated text and added comments

2009-12-03T14:51:30Z

Updated text and added comments

← Older revision		Revision as of 07:51, 3 December 2009
Line 16:		Line 16:
	Testing ZFS with Lustre has resulted in the detection of previously unknown problems in network cards. These cards silently introduced data corruption that went undetected without the use of ZFS. It should be noted that the checksum computation does consume some processor cycles, approximately 1 GHz of CPU time to process 500 MB/sec of I/O.		Testing ZFS with Lustre has resulted in the detection of previously unknown problems in network cards. These cards silently introduced data corruption that went undetected without the use of ZFS. It should be noted that the checksum computation does consume some processor cycles, approximately 1 GHz of CPU time to process 500 MB/sec of I/O.

	''An implementation note:'' ZFS support is being developed and tested with a user space implementation of the ZFS DMU. ~~In the future~~, the DMU ~~will run~~ in kernel space. Also, the Lustre DMU code is almost entirely common with the Solaris version of ZFS, so Lustre support for ZFS will closely parallel the Solaris release of ZFS.		''An implementation note:'' Previously, ZFS support was being developed and tested with a user space implementation of the ZFS DMU. Currently, we are running the DMU in kernel space. Also, the Lustre DMU code is almost entirely common with the Solaris version of ZFS, so Lustre support for ZFS will closely parallel the Solaris release of ZFS.

	Lustre support of ZFS will offer several specific advantages:		Lustre support of ZFS will offer several specific advantages:
	* ''Self-healing capability'' - In a mirrored or RAID configuration, ZFS not only detects data corruption, but it automatically corrects the bad data.		* ''Self-healing capability'' - In a mirrored or RAID configuration, ZFS not only detects data corruption, but it automatically corrects the bad data.
	* ''Improved administration'' - Because ZFS detects and reports data corruption on all read and write errors at the block level, it is easier for system administrators to quickly identify which hardware components are corrupting data.		* ''Improved administration'' - Because ZFS detects and reports data corruption on all read and write errors at the block level, it is easier for system administrators to quickly identify which hardware components are corrupting data. ZFS also has very easy-to-use command-line administration utilities.
	* ''SSD support'' - ZFS supports the addition of high-speed I/O devices, such as SSDs, to the storage pool. The Read Cache Pool or L2ARC acts as a cache layer between memory and the disk. This support can substantially improve the performance of random read operations.		* ''SSD support'' - ZFS supports the addition of high-speed I/O devices, such as SSDs, to the storage pool. The Read Cache Pool or L2ARC acts as a cache layer between memory and the disk. This support can substantially improve the performance of random read operations. SSDs can also be used to improve synchronous write performance, by adding them to the pool as log devices. You can add as many SDDs to your storage pool as you need to increase your read cache size and IOPS, your synchronous write IOPS, or both.
	* ''Scalability'' - ZFS is a 128-bit file system. This means that current restrictions on maximum-size file systems for a single MDS or OST, maximum stripe size, and maximum size of a single file will be removed. ZFS support will also remove the current 8 TB limitation on LUNs.		* ''Scalability'' - ZFS is a 128-bit file system. This means that current restrictions on maximum-size file systems for a single MDS or OST, maximum stripe size, and maximum size of a single file will be removed. ZFS support will also remove the current 8 TB limitation on LUNs.

	For more general information about ZFS, see [[ZFS Resources]].		For more general information about ZFS, see [[ZFS Resources]].