Architecture - External File Locking

Note: The content on this page reflects the state of design of a Lustre feature at a particular point in time and may contain outdated information.

Definitions

POSIX file lock, lockf lock, fcntl lock

file range lock obtained through one of file range locking interfaces prescribed by POSIX and based on SysV lockf() and fcntl() interfaces.

flock lock

whole-file lock obtained through BSD flock interface.

group lock

whole-file lock obtained through Lustre-specific LL_IOC_GROUP_LOCK ioctl().

file lock

either of above locks.

Use Cases

id	quality attribute	summary
posix-and-flock	usability	Both POSIX and flock locking is supported.
EDEADLK	usability, availability	Faithful dead-lock detection.
lock-recovery	availability	The cluster on which migrations are active suffers a failure. File locks are recovered properly.
scalability	usability, performance	No service degradation in case of very wide striping and file lock contention.
unit-tests	testability	Unit tests for flock usage and dead-lock detection.

posix-and-flock

Scenario:		Both POSIX and flock locks are being taken over overlapping regions of the same file.
Business Goals:		Avoid interaction between different locking interfaces.
Relevant QA's:		Usability
details	Stimulus:	Two threads on the same client or threads from different clients obtain flock and lockf locks on the overlapping regions of the same file.
	Stimulus source:	User applications.
	Environment:	File being locked.
	Artifact:	Overlapping locks of different flavours.
	Response:	No interaction between locks.
	Response measure:	Clients do not block or dead-lock on such locks.
Questions:		This is traditional Linux behaviour. Do we want to reproduce it?
Issues:		None.

-EDEADLK

Scenario:		Client places blocking file lock request on server that would result in dead-lock if processed.
Business Goals:		Avoid dead-locks.
Relevant QA's:		Usability & Availability
details	Stimulus:	Client tries to acquire lock, resulting in possible dead-lock.
	Stimulus source:	User applications.
	Environment:	Locks.
	Artifact:	Waiting cycle among locks.
	Response:	Dead-lock is detected.
	Response measure:	Lock request is denied with -EDEADLK error.
Questions:		How precise dead-lock detection has to be, i.e., is server permitted to return -EDEADLK sporadically when no actual dead-lock is possible? How sound dead-lock detection has to be, i.e., is server permitted to occasionally miss dead-lock?
Issues:		Precise and sound dead-lock detection can be quite intricate. In addition, it basically mandates keeping track of all file locks in a single place within cluster, which conflicts with scalability requirement.

recovery

TBD.

scalability

Scenario:		Large number of locks is created. Locks are contended by clients.
Business Goals:		Ascertain graceful service degradation.
Relevant QA's:		Usability & Performance
details	Stimulus:	Clients obtains large number of locks, and place large number of lock requests waiting to be granted.
	Stimulus source:	User applications.
	Environment:	Servers handling locks.
	Artifact:	Large number of locks and lock requests.
	Response:	Algorithms handle this situation gracefully.
	Response measure:	No server overload.
Questions:
Issues:		Conflicts with dead-lock detection somewhat.

Implementation constraints

Use DLM to implement flock.

Questions and Issues

• any special handling for liblustre?
• do we want file locks to be kept on LRU list by clients?

References

• Implementation as bug 5506
• flock recovery as bug 11880
• flock and MPI-IO issues as bug 11660