WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.

Difference between revisions of "Architecture - User Level Access"

From Obsolete Lustre Wiki
Jump to navigationJump to search
m (1 revision)
(No difference)

Revision as of 14:19, 14 January 2010

Summary

The LNET userspace API driver exports the LNET API to userspace so that userspace lustre servers and clients can use all existing kernel LNET infrastructure.

Requirements

LNET API
Userspace processes can use the LNET API transparently to communicate with other processes of all flavours.
Multi-user
LNET users are distinguished by LNET PID. This PID may be shared by several O/S processes which may in turn be multi-threaded. Processes sharing the same LNET PID may share ME, MD and EQ handles. Kernel threads all share the same PID.
PID assignment
PID is requested in LNetNIInit(). If this is LNET_PID_ANY, a new unique PID is created and given the real userid of the caller. If the requested PID exists already, the caller may be permitted to join it. If it does not exist, the caller may be permitted to create it.
Privilege
Privileged users may create or join any PID. Unprivileged users must set the "userflag" when requesting a PID and my only join an existing PID if the caller's real userid matches the PID's userid.
Isolation
Replies, acks and events for a given PID are isolated from other PIDs, including future incarnations of the same PID.
Cleanup
A PID is destroyed when all its processes have finalized or terminated. At this time, all outstanding communications are completed with failure in finite time and all associated state is freed.
Parameter checking
NO userspace LNET API parameters are trusted. Checking may occur at any time - e.g. invalid buffer addresses may be discovered when a communication completes rather than when it was initiated.
Memory
User pages bound to an MD are locked down until the MD has been destroyed.
EQ Callbacks
EQ callbacks in userspace are not supported. Userspace processes may dedicate one or more threads to process events eagerly.
Thread safety
All exported LNET API calls are thread-safe.
Zero-copy
Communications to/from userspace buffers are zero-copy where the kernel LND supports it.
Resource Limits
LNET kernel module parameters allow resource limits (e.g. extant PIDs, MEs, MDs, concurrent GETs and PUTs) to be set for two classes of user: privileged and unprivileged. These limits include global maxima to limit the total resource that either class of process may use in aggregate. Either class may be set to infinity or zero. The normal "getrlimit" resource limits will apply to pages associated with MDs.

References