Architecture - Wide Striping - Revision history

Docadmin at 21:26, 22 January 2010

2010-01-22T21:26:05Z

← Older revision		Revision as of 14:26, 22 January 2010
Line 1:		Line 1:
	'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.		'''''Note:''''' ''The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.''

	There are several use cases where Lustre wants to write exceptionally many stripes in files:		There are several use cases where Lustre wants to write exceptionally many stripes in files:

Docadmin: Protected "Architecture - Wide Striping" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

2010-01-22T19:42:07Z

Protected "Architecture - Wide Striping" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

← Older revision	Revision as of 12:42, 22 January 2010
(No difference)

Docadmin at 00:23, 19 January 2010

2010-01-19T00:23:04Z

← Older revision		Revision as of 17:23, 18 January 2010
Line 1:		Line 1:
			'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.

	There are several use cases where Lustre wants to write exceptionally many stripes in files:		There are several use cases where Lustre wants to write exceptionally many stripes in files:

Docadmin: /* APIs required */

2010-01-15T00:09:27Z

APIs required

← Older revision		Revision as of 17:09, 14 January 2010
Line 24:		Line 24:
	#* OST(stripe_idx) = FLDB(seq_start + stripe_idx*seq_skip)		#* OST(stripe_idx) = FLDB(seq_start + stripe_idx*seq_skip)
	#* OBJ(stripe_idx) = FID{seq_start + stripe_idx*seq_skip,fid_number,obj_version}		#* OBJ(stripe_idx) = FID{seq_start + stripe_idx*seq_skip,fid_number,obj_version}

	~~[[Category:Architecture\|Wide Striping]]~~

Docadmin: moved Very Wide Striping to Architecture - Wide Striping

2010-01-14T21:12:40Z

moved Very Wide Striping to Architecture - Wide Striping

← Older revision	Revision as of 14:12, 14 January 2010
(No difference)

Docadmin: 1 revision

2010-01-14T21:01:06Z

1 revision

← Older revision	Revision as of 14:01, 14 January 2010
(No difference)

Adilger: add raid parameters to EA, offsete calculations

2007-08-08T14:44:15Z

add raid parameters to EA, offsete calculations

New page

There are several use cases where Lustre wants to write exceptionally many stripes in files:

# Major HPC installations may have many hundreds or thousands of OSTs and we need to be able to stripe files over all of them
# Server Network Striping (SNS) will use parity declustering, resulting in an very large number of objects building up the striped file.

Therefore, wide striping will be a commonly encountered case. The goal is to encode the striping information in a very compact way.

== Definitions (see fid-hld) ==

; A pool: defines an un-ordered sets of OSTs and will be used to describe the striping in a manageable way.
; fid seq number: part of fully specified FID, contains sequence in which object was created
; fid number: part of fully specified FID, contains object id within its sequence
; object version: part of fully specified FID, contains object version number
; FID: fully specified object identification structure: FID = {f-sequence, f-number, f-version}
; FLDB: FID Location DataBase, provides fid sequence to server (OST, MDS) mapping

== APIs required ==

# Get a consecutive set of fid sequence numbers from the FLDB
# define an on-disk EA that contains a pool name and other RAID striping parameters, for use as a default directory EA
# define an on-disk EA that contains a RAID type, raid parameters, a starting fid sequence number, a count of objects over which the object may be striped, a sequence skip count, a single fid number used by this file in all specified sequences, the object version, possibly the pool from which this object was allocated (for future reference)
# offsets within the file are {lov_offset, stripe_index} = fn(file_offset, raid_type, raid_parameters}
# individual objects OBJ{0, ..., num_obj - 1} in the file can be located:
#* OST(stripe_idx) = FLDB(seq_start + stripe_idx*seq_skip)
#* OBJ(stripe_idx) = FID{seq_start + stripe_idx*seq_skip,fid_number,obj_version}

[[Category:Architecture|Wide Striping]]