Docadmin at 21:18, 22 January 2010

2010-01-22T21:18:51Z

← Older revision		Revision as of 14:18, 22 January 2010
Line 1:		Line 1:
	'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.		'''''Note:''''' ''The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.''

	= Requirements =		= Requirements =

Docadmin: Protected "Architecture - New Metadata API" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

2010-01-22T18:42:18Z

Protected "Architecture - New Metadata API" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

← Older revision	Revision as of 11:42, 22 January 2010
(No difference)

Docadmin at 00:17, 19 January 2010

2010-01-19T00:17:51Z

← Older revision		Revision as of 17:17, 18 January 2010
Line 1:		Line 1:
			'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.

	= Requirements =		= Requirements =

Docadmin: /* Implementation steps */

2010-01-14T23:46:14Z

Implementation steps

← Older revision		Revision as of 16:46, 14 January 2010
Line 180:		Line 180:
	# restructure llite and MDC to new API		# restructure llite and MDC to new API
	# IFF we want MDWBC implemented this way, restructure MDT/MDD and implement ROSD/TOSD		# IFF we want MDWBC implemented this way, restructure MDT/MDD and implement ROSD/TOSD


	~~[[Category:Architecture\|New Metadata API]]~~

Docadmin: moved New Metadata API to Architecture - New Metadata API

2010-01-14T21:55:37Z

moved New Metadata API to Architecture - New Metadata API

← Older revision	Revision as of 14:55, 14 January 2010
(No difference)

Docadmin: 1 revision

2010-01-14T21:01:10Z

1 revision

New page

= Requirements =

# clear, simple, no tricks
# portable
# support for CMD
# support for WBC
# support for few llite-like layer at same time (lustre, pnfs, etc)

== Proposal ==

# MDAPI consists of primitive metadata operations like lookup, getattr by fid, readdir, create, unlink, open, close, mkdir, rmdir, rename
# MDD uses same API so that client can talk to MDD directly
# all network-related things are extracted to MDC-MDT (intents, recovery)
# open is part of in-core state needed only for remote clients, so part of MDT-MDC only
# there may be MDD running on server and MDD running on client at same time. MDD on server is better for contended resource (like /tmp directory), MDD on client is metadata writeback cache
# MDC should provide notion of context where results of compound operation can be stored, this context to be opened and closed by llite (probably ldlm could help as we're holding lock till end of operation)
# LMV is request forwarder only (do we still need striped directories?)
# transactions are exposed above MDC with CMD as single transaction can update few servers

{|
|-
|
[[Image:mdapi.png|center]]
|
# red lines - Metadata API
# blue lines - OSD API
# black lines - RPC
# Llite - OS specific part, takes care about cache, locking
# MDC - client component turning set of MDAPI calls into single RPC
# MDT - server component turning RPC with intent into set of MDAPI calls
# ROSD - Remote OSD, client sending OSD requests over network
# TOSD - Target OSD, server receiving OSD request from network
|}

= API =

'''context''' - a notion allowing to store results of do-ahead operation (intents)

md_init_ctxt() -- initialize context
md_fini_ctxt() -- release context
md_lookup() -- lookup given parent directory and name into fid (NULL)
md_getattr() -- fetch attributes for given fid
md_setattr() -- set attributes for given fid
md_create() -- create file/directory with given name in given parent
md_open() -- open file/directory by fid, returns open handle
md_close() -- close given open handle
md_readdir() -- return set of directory entries from given fid/hash/offset?
md_unlink() -- unlink name in given directory (may cause file removal)
md_symlink() -- create symbolic link
md_rename() -- rename ...
md_xattr_*() -- set of calls to manipulate extended attributes
md_statfs() -- return fs stats (all/avail blocks/files, etc)
md_enqueue() -- enqueue lock (can take intent from context)
md_decref_lock()-- release (not meaning cancel) lock
md_cancel() -- cancel lock

= Use Cases =

== final lookup with IT_OPEN + IT_CREATE ==

{|
! non-WBC !! WBC
|-
|
ll_lookup()
ctxt = md_init_ctxt(IT_OPEN | IT_CREAT, parent, name, ...)
lockh = md_enqueue(ctxt, <parent fid>, LCK_PR, ...);
mdc_enqueue(ctxt, ...)
mdt_enqeue()
lock(pfid, UPDATE, PR)
pfid = find_obj_by_fid()
fid = md_lookup(pfid, name)
if (fid == NULL)
tx = tx_create()
tx_declare_write(tx, last_rcvd_file)
md_declare_create(tx, pfid, name)
mdd_declare_create(tx, pfid, name)
osd_tx_declare_insert()
osd_tx_declare_new_obj()
md_declare_set_xattr(tx, fid, LOVEA)
mdd_declare_set_xattr()
osd_tx_declare_set_xattr()
md_tx_start(tx)
mdd_tx_start(tx)
osd_tx_start(tx)
fid = md_create(tx, pfid, name)
mdd_create(tx)
fid = osd_new_object()
osd_insert_index(pfid, name, fid)
md_set_xattr(tx, fid, LOVEA)
mdd_set_xattr(tx, fid, LOVEA)
osd_set_xattr(tx, fid, LOVEA)
mdt_update_last_rcvd()
md_tx_commit(tx)
mdd_tx_commit(tx)
osd_tx_commit(tx)
md_getattr(fid, &attr)
lock(fid, OPEN, PR)
unlock(pfid)
/* pack lock&attr in reply */
/* mdc unpack reply and store in ctxt */
md_lookup(ctxt, parent, name, &fid)
mdc_lookup(ctxt, parent, name, &fid)
/*
md_getattr(ctxt, fid, &attr)
mdc_getattr(ctxt, fid, &attr)
inode = iget(FID2INO(fid), ll_test_inode, ll_set_inode, &attr);
md_open(ctxt, fid, mode)
mdc_open(ctxt, fid, mode)
md_decref_lock(lockh);
md_fini_ctxt(ctxt);

|writing-mode="lr-tb"|
ll_lookup()
ctxt = md_init_ctxt(IT_OPEN | IT_CREAT, parent, name, ...)
lockh = md_enqueue(ctxt, <parent fid>, LCK_PR, ...);
md_lookup(ctxt, parent, name, &fid)
mdd_lookup(ctxt, parent, name, fid)
rosd_lookup(ctxt, parent, name, fid)
if (fid == NULL)
md_tx_create()
mdd_tx_create()
rosd_tx_create()
md_tx_declare_create()
mdd_tx_declare_create()
rosd_tx_declare_create()
md_tx_declare_set_xattr
mdd_tx_declare_set_xattr()
rosd_tx_declare_create()
md_tx_start()
mdd_tx_start()
rosd_tx_start()
md_create(parent, name)
mdd_create()
rosd_new_object()
rosd_insert_index(pfid, name, fid)
md_set_xattr(tx, fid, LOVEA)
mdd_set_xattr()
rosd_set_xattr()
md_tx_commit()
/* put details in the context */
md_getattr(ctxt, fid, &attr)
mdd_getattr(ctxt, fid, &attr)
rosd_getattr()
inode = iget(FID2INO(fid), ll_test_inode, ll_set_inode, &attr);
md_open(ctxt, fid, mode)
/* here we somehow pin object */
/* not sure is this responsibility of MDD or upper layer */
mdd_open(ctxt, fid, mode)
md_decref_lock(lockh);
md_fini_ctxt(ctxt);

|}

== intermediate lookup ==

== intermediate revalidate ==

== final revalidate ==

== mkdir ==

== unlink/rmdir ==

== rename ==

== readdir ==

== lock cancel ==

= Problems =

# without intents in VFS, it's possible that stat(2) has to issue 2 RPCs to MDS: one from ->lookup(), another from ->getattr(). is solution needed?

= Implementation steps =

# restructure llite and MDC to new API
# IFF we want MDWBC implemented this way, restructure MDT/MDD and implement ROSD/TOSD

[[Category:Architecture|New Metadata API]]

Architecture - New Metadata API - Revision history

Docadmin at 21:18, 22 January 2010

Docadmin: Protected "Architecture - New Metadata API" ([edit=sysop] (indefinite) [move=sysop] (indefinite))

Docadmin at 00:17, 19 January 2010

Docadmin: /* Implementation steps */

Docadmin: moved New Metadata API to Architecture - New Metadata API

Docadmin: 1 revision