Docadmin at 21:17, 22 January 2010

2010-01-22T21:17:18Z

← Older revision		Revision as of 14:17, 22 January 2010
Line 1:		Line 1:
	'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.		'''''Note:''''' ''The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.''

	== Summary ==		== Summary ==

Docadmin: Protected "Architecture - Migration (2)" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

2010-01-22T18:26:52Z

Protected "Architecture - Migration (2)" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

← Older revision	Revision as of 11:26, 22 January 2010
(No difference)

Docadmin at 00:16, 19 January 2010

2010-01-19T00:16:38Z

← Older revision		Revision as of 17:16, 18 January 2010
Line 1:		Line 1:
			'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.

	== Summary ==		== Summary ==

Docadmin: Created page with '== Summary == Migration is a process of data and metadata moving within one cluster (one namespace) as well as to/from external non-Lustre storage servers. HSM, free space bal...'

2010-01-15T19:14:13Z

Created page with '== Summary == Migration is a process of data and metadata moving within one cluster (one namespace) as well as to/from external non-Lustre storage servers. HSM, free space bal...'

New page

== Summary ==

Migration is a process of data and metadata moving within one cluster (one namespace) as well as to/from external non-Lustre storage servers. HSM,
free space balance, file restriping are examples of migration.

== Definitions ==

;'''Agent''' : actually copies objects
;'''Feed''' : object enumeration
;'''Feed Generator''': produces object enumeration for migration agent
;'''Coordinator''' : menages migrations and running migration agents
;'''Initiator''': initiates a migration, issues a migration request

== Use cases ==

{| cellspacing="0" border="1"
! ID !! Quality Attribute !! Summary
|-
| simple migration
| usability, performance
| a simple migration within one name space
|-
| duplicate requests are merged
| performance
| duplicate migration requests are proccessed as one request
|-
| conflicting requests abort in-progress migration
| performance
| when an object in process of migrating to HSM, any access to the object aborts migration
|-
| coherent access to moving objects
| usability
| moving object continues to be accessible to clients
|-
| propagate punch and trunc to source, as well as llog on sink
| performance
| don't copy truncated data
|-
| recovery
| availability
| restore moving object state after a server crash
|-
| single namespace at all times
| usability
| moving and migrated objects are in the same namespace
|-
| scalability
| scalability
| more servers - faster migration
|-
| reconnect with dirty cache after server migration has completed (FLDB)
| usability
| dirty cache reintegration when the objects are moved already
|-
| support partial file reclamations
| performance, usability
| file can't fit fully in cache
|-
| cache full / master full management (grants?)
| usability
| we don't want destination server to run out of space in the middle of migration
|-
| IO optimization
| performance
| agents will somehow create large IOs
|}

== Quality Attribute Scenarios ==

Simple migration

{|border=1 cellspacing="0"
|-align="left"
|colspan=2|'''Scenario:''' || simple data and md migration within one namespace
|-align="left"
|colspan=2|'''Business Goals:'''|| advanced control over Lustre object placement
|-align="left"
|colspan=2|'''Relevant QA's:'''|| usability, performance
|-align="left"
|rowspan="6" writing-mode="vertical"|'''details'''
| '''Stimulus:'''|| a migration request
|-align="left"
|'''Stimulus source:'''|| administrator, lustre control utilities, client
|-align="left"
|'''Environment:'''|| a Lustre cluster
|-align="left"
|'''Artifact:'''|| migration coordinator
|-align="left"
|'''Response:'''
| an administrator issues a migration request to a coordinator. The
coordinator starts or wakes up one or more migration agents. The
coordinator asks migration agents to move Lustre objects with given
IDs. The migration agents do actual migration.
|-align="left"
|'''Response measure:'''
| successful migration, achieving good performance by moving objects in parallel
|-align="left"
|colspan=2|'''Questions:'''||
|-align="left"
|colspan=2|'''Issues:'''||
|-
|}

duplicate requests are merged

{|border=1 cellspacing="0"
|-align="left"
|colspan=2|'''Scenario:''' || second migration request is issued when the object is being migrated
|-align="left"
|colspan=2|'''Business Goals:'''|| handle duplicated requests efficiently
|-align="left"
|colspan=2|'''Relevant QA's:'''|| performance
|-align="left"
|rowspan="6" writing-mode="vertical"|'''details'''
| '''Stimulus:'''|| a migration request
|-align="left"
|'''Stimulus source:'''|| client, administrator, a control utility
|-align="left"
|'''Environment:'''|| an object being migrated
|-align="left"
|'''Artifact:'''|| coordinator
|-align="left"
|'''Response:'''||
the coordinator detects duplication of the requests and execute them as one
|-align="left"
|'''Response measure:'''|| successful execution of two requests, no duplicated requrests disturbing each other
|-align="left"
|colspan=2|'''Questions:'''||
|-align="left"
|colspan=2|'''Issues:'''||
|-
|}

conflicting requests abort in-progress migration

{|border=1 cellspacing="0"
|-align="left"
|colspan=2|'''Scenario:''' || archiving to a tape is aborted if conflicting migration request is issued
|-align="left"
|colspan=2|'''Business Goals:'''|| eliminate useless arhive operation
|-align="left"
|colspan=2|'''Relevant QA's:'''|| performance
|-align="left"
|rowspan="6" writing-mode="vertical"|'''details'''
| '''Stimulus:'''|| client's file access (for the HSM case)
|-align="left"
|'''Stimulus source:'''|| client application
|-align="left"
|'''Environment:'''|| HSM, a file is being archived to a tape, someone wants to write to the file
|-align="left"
|'''Artifact:'''|| coordinator
|-align="left"
|'''Response:'''|| coordinator aborts the archiving operation
|-align="left"
|'''Response measure:'''|| archiving op is aborted
|-align="left"
|colspan=2|'''Questions:'''||
|-align="left"
|colspan=2|'''Issues:'''||
|-
|}

coherent access to moving objects

{|border=1 cellspacing="0"
|-align="left"
|colspan=2|'''Scenario:''' || transparent access to objects being migrated
|-align="left"
|colspan=2|'''Business Goals:'''|| concurrent access to the moving objects
|-align="left"
|colspan=2|'''Relevant QA's:'''|| performance, scalability
|-align="left"
|rowspan="6" writing-mode="vertical"|'''details'''
| '''Stimulus:'''|| file access
|-align="left"
|'''Stimulus source:'''|| client application
|-align="left"
|'''Environment:'''|| an object being moved, a client application accesses the object
|-align="left"
|'''Artifact:'''|| moving object
|-align="left"
|'''Response:'''|| The migration agent(s) move the objects in chunks, protecting the currently moved chunks by exclusive locks and redirect application requests to appropriate data location (source or target)
|-align="left"
|'''Response measure:'''|| client is able to access an object being moved,
client access isn't blocked for the period of object migration.
|-align="left"
|colspan=2|'''Questions:'''|| always redirect to target?
|-align="left"
|colspan=2|'''Issues:'''||
|-
|}

Empty UC

{|border=1 cellspacing="0"
|-align="left"
|colspan=2|'''Scenario:''' ||
|-align="left"
|colspan=2|'''Business Goals:'''||
|-align="left"
|colspan=2|'''Relevant QA's:'''||
|-align="left"
|rowspan="6" writing-mode="vertical"|'''details'''
| '''Stimulus:'''||
|-align="left"
|'''Stimulus source:'''||
|-align="left"
|'''Environment:'''||
|-align="left"
|'''Artifact:'''||
|-align="left"
|'''Response:'''||
|-align="left"
|'''Response measure:'''||
|-align="left"
|colspan=2|'''Questions:'''||
|-align="left"
|colspan=2|'''Issues:'''||
|-
|}

== Implementation details ==

* '''IMP1''' all IO involving Lustre servers uses client API (exploit existing locking and sync '''LLITE''' infrastructure)
** avoid reimplementing client
* '''IMP2''' separate problem into coordinators and agents
** agents have datamover plugins
* '''IMP3''': pull model if target is Lustre OSD (run agent on sink)
* '''IMP4''': if target is Lustre OSD, send all requests to target (block client requests until they can be filled on target). source OSD must redirect
** let MDTmaintain the redirection, in line with flash cache
** callback layout (stripe descriptor) when we initiate migration, sends
** blocking asts to all clients with any locks on the file
** 3 phase: old layout, dual layout, final layout
* '''IMP5''': need a lock bit for layouts
** means we need to drop the client lock when we flush inode
* '''IMP6''': creation of target object results in llog entry with old and new EA (SOM-style recovery)
** IMP7: record (llog) and execute trunc/punch on tgt, propagate to source
* '''IMP8''': bit (or extent log) on MDT (master copy) indicating copy or tape is current ("can I reclaim this space?") 1 llog of extents indicating which files are on tape (for fast space reclamation). Similar to WBC on clients
* '''IMP9''': Use commit CB on EA with tape FID to tell the tape "not an orphan" (i.e. "we're counting on this tape fid")
* '''IMP10''': MDS inode objects never change fids
* '''IMP11''' locks a migrator might take automatically interact with locks other clients may take

== References ==
[https://bugzilla.lustre.org/show_bug.cgi?id=14698 bug 14698]

Architecture - Migration (2) - Revision history

Docadmin at 21:17, 22 January 2010

Docadmin: Protected "Architecture - Migration (2)" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

Docadmin at 00:16, 19 January 2010

Docadmin: Created page with '== Summary == Migration is a process of data and metadata moving within one cluster (one namespace) as well as to/from external non-Lustre storage servers. HSM, free space bal...'