http://wiki.old.lustre.org/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET&feed=atom&action=historyArchitecture - Multiple Interfaces For LNET - Revision history2024-03-29T12:30:57ZRevision history for this page on the wikiMediaWiki 1.35.5http://wiki.old.lustre.org/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET&diff=10598&oldid=prevDocadmin at 21:18, 22 January 20102010-01-22T21:18:13Z<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 21:18, 22 January 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1" >Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information. </div></td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>'''''Note:<ins class="diffchange diffchange-inline">''</ins>''' ''The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.<ins class="diffchange diffchange-inline">'' </ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Use Cases ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Use Cases ==</div></td></tr>
</table>Docadminhttp://wiki.old.lustre.org/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET&diff=10524&oldid=prevDocadmin: Protected "Architecture - Multiple Interfaces For LNET" ([edit=sysop] (indefinite) [move=sysop] (indefinite))2010-01-22T18:36:09Z<p>Protected "<a href="/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET" title="Architecture - Multiple Interfaces For LNET">Architecture - Multiple Interfaces For LNET</a>" ([edit=sysop] (indefinite) [move=sysop] (indefinite))</p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<tr class="diff-title" lang="en">
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 18:36, 22 January 2010</td>
</tr><tr><td colspan="2" class="diff-notice" lang="en"><div class="mw-diff-empty">(No difference)</div>
</td></tr></table>Docadminhttp://wiki.old.lustre.org/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET&diff=10145&oldid=prevDocadmin at 00:17, 19 January 20102010-01-19T00:17:26Z<p></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 00:17, 19 January 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1" >Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">'''''Note:''''' The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information. </ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Use Cases ==</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Use Cases ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr>
</table>Docadminhttp://wiki.old.lustre.org/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET&diff=9931&oldid=prevDocadmin: /* ISSUES */2010-01-14T23:45:01Z<p><span dir="auto"><span class="autocomment">ISSUES</span></span></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 23:45, 14 January 2010</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l83" >Line 83:</td>
<td colspan="2" class="diff-lineno">Line 83:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Suppose servers are added with a previously unspecified network. In this case the MGS needs to learn this at addition time, in particular, the MGS would have to reparse its own /etc/moprobe.conf file or get information from the new servers.</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Suppose servers are added with a previously unspecified network. In this case the MGS needs to learn this at addition time, in particular, the MGS would have to reparse its own /etc/moprobe.conf file or get information from the new servers.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># This requirements discussion does not address the naming of nodes, which might be an additional useful requirement.</div></td><td class='diff-marker'> </td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># This requirements discussion does not address the naming of nodes, which might be an additional useful requirement.</div></td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">[[Category:Architecture|Multiple Network Interfaces]]</del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>−</td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">[[Category:QAS|Multiple Network Interfaces]]</del></div></td><td colspan="2"> </td></tr>
</table>Docadminhttp://wiki.old.lustre.org/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET&diff=9832&oldid=prevDocadmin: moved Multiple Interfaces For LNET to Architecture - Multiple Interfaces For LNET2010-01-14T21:16:52Z<p>moved <a href="/index.php?title=Multiple_Interfaces_For_LNET&action=edit&redlink=1" class="new" title="Multiple Interfaces For LNET (page does not exist)">Multiple Interfaces For LNET</a> to <a href="/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET" title="Architecture - Multiple Interfaces For LNET">Architecture - Multiple Interfaces For LNET</a></p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<tr class="diff-title" lang="en">
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 21:16, 14 January 2010</td>
</tr><tr><td colspan="2" class="diff-notice" lang="en"><div class="mw-diff-empty">(No difference)</div>
</td></tr></table>Docadminhttp://wiki.old.lustre.org/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET&diff=9730&oldid=prevDocadmin: 1 revision2010-01-14T21:01:06Z<p>1 revision</p>
<table class="diff diff-contentalign-left diff-editfont-monospace" data-mw="interface">
<tr class="diff-title" lang="en">
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="1" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 21:01, 14 January 2010</td>
</tr><tr><td colspan="2" class="diff-notice" lang="en"><div class="mw-diff-empty">(No difference)</div>
</td></tr></table>Docadminhttp://wiki.old.lustre.org/index.php?title=Architecture_-_Multiple_Interfaces_For_LNET&diff=9729&oldid=prevBraam: /* = Use Cases */2007-07-30T16:02:29Z<p><span dir="auto"><span class="autocomment">= Use Cases</span></span></p>
<p><b>New page</b></p><div>== Use Cases ==<br />
<br />
A node may have one or more link sets - these are a set of nids that<br />
will be used in an aggregated fashion.<br />
<br />
# one to many, many to one, many to many and rails situations need to be supported. Precisely link sets with K elements should be able to connect to link sets with L elements.<br />
## clients with one interface to servers with 2<br />
## vice versa<br />
## rail situations<br />
# a link set requires an aggregation descriptor:<br />
## bandwidth aggregation behavior<br />
## link level failover/failure recovery model<br />
### Some of these are optional or for future versions. <br />
### These descriptors need to go into /etc/modprobe.conf<br />
# The MGS will be reached through passing multiple remote addresses describing a failover link set<br />
# Aggregation is desirable for links on a single or on multiple LNETs<br />
# Utilities like lctl ping <nid> can send/packets to an individual nid of an interface and to an aggregated link set. (a link set probably needs to be named with a nid)<br />
# Clients will connect to servers by naming the server link set. This requirement is to allow clients outside a firewall to connect to a server behind a firewall where the server has non-reachable nids (like 192.168.1.*) which might have a different meaning near the client.<br />
# Lustre will see multiple nid's only for failover, i.e. no new connection behavior<br />
# The nids in a link set will be made available though the LNET management node (probably the MGS) to allow dynamic server addition.<br />
# Configuration will allow "real failover IP addresses" to be configured.<br />
# Desirable implementation constraint: a linkset will be a nid on a lustre LNET and the routing mechanisms will be used to reach the nid and implement aggregation behavior. LNET bandwidth sharing, failure handling etc.<br />
# It shall be possible to specify lustre configurations for simultaneous use of different linksets on the same server targets.<br />
# If modified nids are used, they shall be big enough to contains both linkset modifiers and IP-v6 addresses<br />
# modprobe.conf shall remain a clusterwide file<br />
<br />
== CONFIGURATION MANAGEMENT ==<br />
<br />
=== Lustre configuration adaptations ===<br />
<br />
A lustre configuration specification must be able to describ linksets<br />
for each node that shall be used during Lustre setup.<br />
<br />
The ip2net configuration directive is extremely similar to what we<br />
need here. <br />
<br />
options lnet 'ip2linkgroup="eth-oss-vib-mds 192.168.0.[2-20]@tcp0:ethall;<br />
eth-oss-vib-mds 132.6.1.[2,3]@vib0; vib-all *@vib0"'<br />
<br />
The ethall directive is a linkset nid modifier as defined in B below.<br />
<br />
Specifying this as a modprobe.conf parameter is very desirable,<br />
because every node would have linkgroup descriptors which it could use<br />
to establish routes to aggregated linksets.<br />
<br />
The mount command can give a parameter:<br />
<br />
mount -t lustre -o linkgroup=<name> <mgs-nids-seq>:fsname /mnt/pt<br />
<br />
The MGS will map a linkgroup name to a linkset nid (using one of the<br />
two alternatives below) for each server, to be used by nodes<br />
connecting to this. These linkset nids will be in the configuration<br />
log and can be interpreted by LNET.<br />
<br />
This allows, for example, the MDS to connect to OSS's over IB while<br />
clients connect to the OSS's over TCP.<br />
<br />
mount -t lustre -o linkgroup=vib-all /dev/mds-dev /mnt/pt<br />
mount -t lustre -o linkgroup=eth-oss-vib-mds <mgs-nids-seq>:fsname /mnt/pt<br />
<br />
=== linkgroup indicators in NIDs ===<br />
<br />
define linksets in /etc/modprobe.conf without a requirement to<br />
define a unique nid, e.g.:<br />
<br />
linkset= <linkset-name>[{<aggr params>}]( <iface list> ) <br />
<br />
linkset=eth-all{failover,noloadbalance}(eth0 eth1)<br />
<br />
extend the syntax of the nid from nid = <address>[@<network>] to<br />
<br />
nid = <address>[@<network>][:<linkset name>]<br />
<br />
Now: <br />
192.168.1.5@tcp0:eth0<br />
192.168.1.5@tcp0:eth-all<br />
become valid nids.<br />
<br />
<br />
<br />
== ISSUES ==<br />
<br />
# Suppose servers are added with a previously unspecified network. In this case the MGS needs to learn this at addition time, in particular, the MGS would have to reparse its own /etc/moprobe.conf file or get information from the new servers.<br />
# This requirements discussion does not address the naming of nodes, which might be an additional useful requirement.<br />
<br />
[[Category:Architecture|Multiple Network Interfaces]]<br />
[[Category:QAS|Multiple Network Interfaces]]</div>Braam