http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&feed=atom&action=history
Architecture - ZFS large dnodes - Revision history
2024-03-29T09:37:16Z
Revision history for this page on the wiki
MediaWiki 1.35.5
http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&diff=10627&oldid=prev
Docadmin at 21:27, 22 January 2010
2010-01-22T21:27:14Z
<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 21:27, 22 January 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1" >Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information. </div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>'''''Note:<ins class="diffchange diffchange-inline">''</ins>''' ''The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.<ins class="diffchange diffchange-inline">'' </ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Definitions ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Definitions ==</div></td></tr>
</table>
Docadmin
http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&diff=10555&oldid=prev
Docadmin: Protected "Architecture - ZFS large dnodes" ([edit=sysop] (indefinite) [move=sysop] (indefinite))
2010-01-22T20:48:15Z
<p>Protected "<a href="/index.php?title=Architecture_-_ZFS_large_dnodes" title="Architecture - ZFS large dnodes">Architecture - ZFS large dnodes</a>" ([edit=sysop] (indefinite) [move=sysop] (indefinite))</p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<tr class="diff-title" lang="en">
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 20:48, 22 January 2010</td>
</tr><tr><td colspan="2" class="diff-notice" lang="en"><div class="mw-diff-empty">(No difference)</div>
</td></tr></table>
Docadmin
http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&diff=10216&oldid=prev
Docadmin: /* Implementation constraints */
2010-01-19T19:32:40Z
<p><span dir="auto"><span class="autocomment">Implementation constraints</span></span></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 19:32, 19 January 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l52" >Line 52:</td>
<td colspan="2" class="diff-lineno">Line 52:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>'zfs' tool must be able to specify dnode size for a filesystem at creation time</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>'zfs' tool must be able to specify dnode size for a filesystem at creation time</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>EA is stored in microzap (as described in [[Architecture - <del class="diffchange diffchange-inline">ZFS_Microzap</del>|ZFS_Microzap]]) to allow efficient access, retrieval, and flexibility.</div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>EA is stored in microzap (as described in [[Architecture - <ins class="diffchange diffchange-inline">ZFS_TinyZAP</ins>|ZFS_Microzap]]) to allow efficient access, retrieval, and flexibility.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Questions and Issues ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Questions and Issues ==</div></td></tr>
</table>
Docadmin
http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&diff=10172&oldid=prev
Docadmin at 00:24, 19 January 2010
2010-01-19T00:24:13Z
<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 00:24, 19 January 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1" >Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information. </ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Definitions ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Definitions ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
</table>
Docadmin
http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&diff=9967&oldid=prev
Docadmin: /* Implementation constraints */
2010-01-15T00:18:25Z
<p><span dir="auto"><span class="autocomment">Implementation constraints</span></span></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 00:18, 15 January 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l50" >Line 50:</td>
<td colspan="2" class="diff-lineno">Line 50:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>'zfs' tool must be able to specify dnode size for a filesystem at creation time</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>'zfs' tool must be able to specify dnode size for a filesystem at creation time</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>EA is stored in microzap (as described in [[ZFS_Microzap]]) to allow efficient access, retrieval, and flexibility.</div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>EA is stored in microzap (as described in [[<ins class="diffchange diffchange-inline">Architecture - ZFS_Microzap|</ins>ZFS_Microzap]]) to allow efficient access, retrieval, and flexibility.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Questions and Issues ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Questions and Issues ==</div></td></tr>
</table>
Docadmin
http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&diff=9966&oldid=prev
Docadmin: /* References */
2010-01-15T00:16:40Z
<p><span dir="auto"><span class="autocomment">References</span></span></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 00:16, 15 January 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l59" >Line 59:</td>
<td colspan="2" class="diff-lineno">Line 59:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>http://www.opensolaris.org/jive/thread.jspa?threadID=39817&tstart=15</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>http://www.opensolaris.org/jive/thread.jspa?threadID=39817&tstart=15</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">[[Category:Architecture|ZFS Large dnodes]]</del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">[[Category:QAS|ZFS Large dnodes]]</del></div></td><td colspan="2"> </td></tr>
</table>
Docadmin
http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&diff=9848&oldid=prev
Docadmin: moved ZFS large dnodes to Architecture - ZFS large dnodes
2010-01-14T21:34:32Z
<p>moved <a href="/index.php?title=ZFS_large_dnodes&action=edit&redlink=1" class="new" title="ZFS large dnodes (page does not exist)">ZFS large dnodes</a> to <a href="/index.php?title=Architecture_-_ZFS_large_dnodes" title="Architecture - ZFS large dnodes">Architecture - ZFS large dnodes</a></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<tr class="diff-title" lang="en">
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 21:34, 14 January 2010</td>
</tr><tr><td colspan="2" class="diff-notice" lang="en"><div class="mw-diff-empty">(No difference)</div>
</td></tr></table>
Docadmin
http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&diff=9760&oldid=prev
Docadmin: 1 revision
2010-01-14T21:01:07Z
<p>1 revision</p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<tr class="diff-title" lang="en">
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 21:01, 14 January 2010</td>
</tr><tr><td colspan="2" class="diff-notice" lang="en"><div class="mw-diff-empty">(No difference)</div>
</td></tr></table>
Docadmin
http://wiki.old.lustre.org/index.php?title=Architecture_-_ZFS_large_dnodes&diff=9759&oldid=prev
Adilger: /* References */
2007-10-19T23:59:44Z
<p><span dir="auto"><span class="autocomment">References</span></span></p>
<p><b>New page</b></p><div>== Definitions ==<br />
<br />
; '''dnode''' : DMU node, 512 bytes in original ZFS implementation and includes "bonus buffer" for users to store extra data<br />
; '''EA''' : extended attribute generally of a limited size (4kB or less), not to be confused with the ZFS 'xattr' which is more like a named byte stream that may be an arbitrary size<br />
; '''znode''' : ZPL-format dnode, uses bonus buffer to store POSIX data for ZFS<br />
== Use Cases ==<br />
<br />
{| border=1 cellspacing=0<br />
|-<br />
!id !! quality attribute !! summary<br />
|-<br />
|large_dnode || performance || increased dnode size to allow more data in the inode<br />
|}<br />
<br />
=== EA in large dnode ===<br />
<br />
{|border=1 cellspacing="0"<br />
|-align="left" <br />
|colspan=2|'''Scenario:''' || Storage of EA in dnode<br />
|-align="left" <br />
|colspan=2|'''Business Goals:''' || Fast access to Lustre EA values<br />
|-align="left" <br />
|colspan=2|'''Relevant QA's:'''|| Performance<br />
|-align="left"<br />
|rowspan="6" writing-mode="vertical"|'''details'''<br />
| '''Stimulus:'''|| EA needs to be stored in ZFS-format dnode<br />
|-align="left"<br />
|'''Stimulus source:'''|| Lustre OSD (MDT/OST) storing EA data to object<br />
|-align="left"<br />
|'''Environment:'''|| EA being stored on a specific znode within a transaction<br />
|-align="left"<br />
|'''Artifact:'''|| EA is stored in the znode<br />
|-align="left"<br />
|'''Response:'''|| time to store EA data<br />
|-align="left"<br />
|'''Response measure:'''|| time is not significantly more than znode update without EA, much less than storing data in ZFS xattr<br />
|-align="left"<br />
|colspan=2|'''Questions:'''|| <br />
|-align="left"<br />
|colspan=2|'''Issues:'''|| None.<br />
|-<br />
|}<br />
<br />
== Implementation constraints ==<br />
<br />
DMU must work with both larger and original size dnodes in the same pool. There is currently no requirement for dynamic dnode size within a single filesystem<br />
<br />
ZFS must be able to work on a filesystem with large dnodes, even if it cannot initially access the extra dnode space<br />
<br />
'zfs' tool must be able to specify dnode size for a filesystem at creation time<br />
<br />
EA is stored in microzap (as described in [[ZFS_Microzap]]) to allow efficient access, retrieval, and flexibility.<br />
<br />
== Questions and Issues ==<br />
<br />
How do we handle case where EAs overflow available space in dnode? Strong consideration should be given to allowing EAs (also?) be stored in an external block or in a protected EA xattr file.<br />
<br />
== References ==<br />
<br />
http://www.opensolaris.org/jive/thread.jspa?threadID=39817&tstart=15<br />
<br />
[[Category:Architecture|ZFS Large dnodes]]<br />
[[Category:QAS|ZFS Large dnodes]]</div>
Adilger