|
|
| (39 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| == Google Doc Questions and Comments ==
| | <small>''(Updated: Mar 2010)''</small> |
| === Lustre Windows Client Questions ===
| | __TOC__ |
| | The Windows Native Client (WNC) project is to deliver a port of the Lustre File System client for the Windows operating system. This port is based on Lustre 2.x code. This project will not port the Lustre servers to Windows. It is a port of the Lustre client only. |
|
| |
|
| This page is to be populated by Lustre Group engineers with information, questions, highlights and risks associated with native Windows client porting efforts.
| | The Lustre driver for Windows will be based on the [http://www.microsoft.com/whdc/devtools/ifskit/default.mspx Windows Installable Filesystem (IFS) Kit]. |
|
| |
|
| === Branch details (Nikita) === | | == Software and Hardware Requirements == |
|
| |
|
| '''Information:''' | | ''Windows versions:'' Windows Server 2008/2008 R2<br> |
| | ''Lustre:'' HEAD code/version 2.x<br> |
| | ''Supported Networks:'' Ethernet and IB (TCP/IP only) |
|
| |
|
| branch for OSR: b_winnt_port
| | ==Currently supported features== |
| | These basic operations have been implemented: |
| | * Lustre client filesystem mount and dismount |
| | * File creation/open/close/rename/getattr/setattr/deletion |
| | * Directory enumeration |
| | * Directory change notification |
| | * Data reading/writing: cached i/o, direct i/o, mmap i/o |
| | * Byte range lock/flock |
|
| |
|
| it's based on b_client_io_layering, which, in turn, is based on HEAD
| | ==Future plans== |
| | * GUI tool for management of lustre mounts |
| | * Integration to Windows network drive assignment |
|
| |
|
| b_winnt_port uses HEAD lnet
| | ==Download and Support== |
| | | The Alpha/Beta release will only be available via the early access program. |
| this branch is in public CVS repository
| |
| | |
| '''Questions:'''
| |
| | |
| is separate lnet branch necessary?
| |
| | |
| repository access control and visibility?
| |
| | |
| === Compiling on Windows (Matt Wu) === | |
| | |
| '''Information:'''
| |
| | |
| gcc & vc incompatible issues: mingw for intermediate modules and wdk for libcfs/tdilnd/llite(final fs driver)
| |
| minimum modifications are needed. building process will do extra steps to process codes
| |
|
| |
| share codes to be made generic (llite/libcfs/tdilnd)
| |
| | |
| 1) structure dot initialization to be converted to code (around 500 LOC in all client modules, evaluated on HEAD)
| |
| | |
| 2) typeof,MACRO to be avoided, the same to other issues which can't easily be done by a program
| |
| | |
| 3) "long" and "long long" to be avoided
| |
| | |
| 4) automatic conversion of CDEBUG, constant suffix ...
| |
|
| |
| automatic builds for different targets
| |
| | |
| 1) on Windows: automatically generate windows style makefiles (DIRS/MAKEFILE/SOURCES) and mingw makefile
| |
| | |
| 2) on Linux: provide a mingw cross-compile environment for Linux people to assure not breaking Windows codes. Only adopted to common codes
| |
| obdclass: windows-specific codes to be moved to windows llite
| |
| | |
| '''Questions:'''
| |
|
| |
| Can we make b_winnt_port a complete windows branch with all above issues resolved ? It eases windows development, but increases complexity on syncing and landing
| |
| | |
| Windows doesn't permit low-level page/task manipulations, while current Linux codes deponds much on them, especially for CLIO routines. like current->journal_info, Windows has "Thread Local Storage" array in every thread for similar purpose, but we can not guarantee other programs' access to it. File system driver has another structure (context control block) to maintain the session related information.
| |
| | |
| === CLIO API (Nikita) ===
| |
| | |
| '''Information:'''
| |
| | |
| most up to date version of CLIO is in b_client_io_layering branch
| |
| | |
| detailed interface documentation is automatically generated from annotated source code by doxygen.
| |
| high level design: HLD/client-io-layering.lyx
| |
| | |
| detailed level design: DLD/client-io-layering-dld.lyx
| |
| | |
| a list of Quality Attribute Scenarios (architectural use cases): http://arch.lustre.org/index.php?title=QAS_Client_Cleanup
| |
| | |
| a to-do/known-issues list: http://spreadsheets.google.com/a/lustre.org/ccc?key=pIIxuIuTZWiZ2P5wvELYoRQ&hl=en /* Hmm... access permissions? */
| |
| | |
| top-level Bugzilla ticket: https://bugzilla.lustre.org/show_bug.cgi?id=14166
| |
| | |
| '''Questions:'''
| |
| | |
| scope of file handles. It looks like in Windows kernel handles for kernel objects (such as files and volumes) are attached to a particular process and can be used only in the context of this process. Is this correct? If so, care has to be taken to work around this issue in ptlrpc (asynchronous request handling), and dlm (lock ASTs). [matt: Windows kernel uses FILE/DEVICE_OBJECT directly rather than Handle. Handle could also be created in kenrel space and will stay valid under arbitrary thread contexts.]
| |
| | |
| OwPurgeCache() and OwFlushCache() works on the whole files only. We need an extent or page/block based interface to support shared-IO mode efficiently.
| |
| | |
| can we get an access to the source code of sample FSDK based file system drivers?
| |
| | |
| locking/sharing model: mandatory/advisory, requested by user-space/kernel?
| |
| | |
| FS_READ, FS_WRITE: synchronous? Should they return when IO operation completes?
| |
| | |
| What are the consistency expectations of FSDK? [matt: most of FSDK-style interfaces are synchronous. but FSDK lib can be asynchous and allow concurrent I/O. I'm not sure of FSDK internals. Need to confirm with OSR.]
| |
| | |
| FS_READ: read region is partially cached. What component handles local (intra-node) locking?
| |
| | |
| === MD API (Alex) ===
| |
| | |
| Semantic:
| |
| | |
| FSDK doesn't allow to unlink open files?
| |
| | |
| does access to file's attribute imply open?
| |
| | |
| does lookup through directory imply preceding open?
| |
| | |
| Caching:
| |
| what exactly does FSDK cache, how and when: parent/name->object, object attributes and directory entries
| |
| | |
| how can we control this? (except through OwPurgeCache() and OwFlushCache())
| |
| | |
| Intents:
| |
| how can we learn purpose of FS_LOOKUP/FS_LOOKUP_PATH: intermediate component, final component, file to be open, file to be created
| |
| | |
| is there a way to store context over single syscall in FSD?
| |
| | |
| should we use FS_LOOKUP_PATH to implement intents?
| |
| | |
| === Next Steps for OSR ===
| |
| | |
| '''QUESTION (PSB)''':
| |
| | |
| What development activities will OSR undertake while support Sun efforts are underway? Please identify a set of deliverable tasks and estimate timeframe.
| |
|
| |
| Matt:
| |
| Could OSR send us FSDK ? The FSDK examples can help us to well understand FSDK interfaces.
| |
| FSDK core library is distributed in binary. Would it be fine for us ?
| |
| | |
| Page purging stays a risk. Cache can't be purged for mmapped files. We either play with stale data or cancel lock revocation. Hope OSR can propose a solution.
| |
| | |
| FSDK doesn't provide cache manipulation support on file data/directory page cache to it's client fs. It's not a good idea to maintain memory pools in the client file system while FSDK/Cache manager maintain another copy.
| |
| | |
| a) No way to read data from cache in FSDK internal interface. The client file system can only talk to the underlying disk or network layer. Though we can get the FileObject from it's orignal Irp, it might conflict with FSDK (OSR doesn't recommend).
| |
| | |
| b) Cannot maintain directory page cache. Windows file system driver normally creates a stream file object for directory inode, the it can perform page / cache operations upon the stream file object. It needs FSDK to work together to handle these virtual stream file objects (to avoid possible conflicts).
| |
| | |
| === End of Google Doc questions ===
| |
| | |
| = Windows Project Information =
| |
|
| |
| == Milestones ==
| |
| - Working Linux Lustre cluster (vmware or otherwise) - Michael MacDonald to assist asap?
| |
| - Mounting on windows – requires Matt’s branch to work
| |
| - File Browsing
| |
| - File I/O
| |
| - Integrate with CFS testing
| |
| - Performance
| |
| | |
| == Methodology ==
| |
| | |
| OSR has a reasonable idea how to get skeleton code together for 1-3, but needs a review of lock revocation and metadata caches before finalizing 3, from someone like Oleg. I expect some assistance from Matt suggesting skeletons etc can help tremendously. Nikita should help architect a skeleton for the file I/O architecture.
| |
| | |
| == Contributors: ==
| |
| | |
| '''OSR'''
| |
| | |
| Dan Root – project manager
| |
| | |
| Tony Mason – design lead, working with Mark
| |
| | |
| Mark Cariddi - developer
| |
| | |
| '''Sun'''
| |
| | |
| Bryon Neitzel - Project Manager
| |
| | |
| Matt – owns libraries, compiler issues, LNET
| |
| | |
| Nikita – file I/O guidance
| |
| | |
| Alex - metadata API
| |
| | |
| Oleg ? – Metadata guidance
| |
| | |
| Eeb – control landing from the branch
| |
| | |
| Braam / Eeb – architecture, abstractions
| |
| | |
| == Risks ==
| |
| - all risks are fairly likely and highly disruptive
| |
| | |
| Loss of CFS attention – CFS project manager
| |
| Poorly defined details – CFS documentation team
| |
| Out of date libraries – Matt Wu
| |
| Compiler issues – Matt Wu
| |
| Code base continues to move – CFS development team
| |
| Getting it right vs working – Nikita / Oleg
| |
| Landing into 2.0 – Barton
| |
| | |
| == Implementation strategy ==
| |
| | |
| OSR will work on skeletons / prototypes and request review of these when ready.
| |
| | |
| == Project management ==
| |
| | |
| Frequent status reports with risk update.
| |
| OSR agreed to weekly status reports and bi-weekly conference calls.
| |
| | |
| == Budget ==
| |
| | |
| Peter Braam and Tony were not yet concerned about the budget situation, but the original estimates were too optimistic, in particular because CFS made no progress between October 2007 and April 2008.
| |
| | |
| == Meeting History ==
| |
| === Windows Client Meeting, Monday, 2008-04-21. ===
| |
| Eric Barton
| |
| Peter Bojanic
| |
| Nikita Danilov
| |
| Alex Tomas
| |
| Matt Wu
| |
| Bryon Neitzel
| |
|
| |
| A Google Document was created for questions from the development staff for Braam's meeting with OSR this week.
| |
| http://docs.google.com/a/lustre.org/Doc?id=d8jgmdp_20fpxwdgb. Alex and Nikita to document questions by Wednesday.
| |
|
| |
| A branch has been created for use by OSR. b_winnt_port branch of lustre-core module (sharing HEAD lnet so far)
| |
|
| |
| Comments from the Meeting:
| |
|
| |
| - eeb: need a clean interface that porting will use.
| |
| - eeb: need metadata interface that doesn't expose internal stuff to another platform
| |
|
| |
| - Matt to continue documentation, compile issues, and porting all client components that are common to other platforms (LOV, LDLM, OSC, MDC, LVM, PTLRPC, etc) Eric would like to be involved before too much design work is done.
| |
| - Matt has drafted a doc titled gcc_vc.pdf that describes the differences between the gcc and vc compilers, and some proposed solutions. This was sent to Dan Root, et al, on Jan 20. Eric to provide feedback.
| |
|
| |
| - Alex wants to learn more about Windows. understand simple use cases.
| |
|
| |
| - would OSR implement the file handle stuff? (do we need to export an MD FID lookup?)
| |
|
| |
| - Get de-brief from Braam on meeting, have him give direction to Alex on md api
| |
| - Alex to send a list of questions to braam
| |
|
| |
| - Put a question in doc about what OSR can be doing?
| |
|
| |
| - Need an OSR work plan as a result of Braam meeting, so the Lustre team knows all the dependencies and dates on items we're responsible for.
| |
|
| |
| - Do we need an LNET branch? Give them a snapshot of all of the sources. Otherwise we don't know what they're implementing to. Nikita says current client IO can't be used for implementation, as it's not far enough along yet.
| |
|
| |
| - Provide branch names: b_winnt _port, HEAD version of LNET. OSR will need their own private branch when they start to commit code. Need to sort out branch situation when OSR is ready to check in code. Their code cannot live in a pubic branch.
| |
