WARNING: This is the _old_ Lustre wiki, and it is in the process of being retired. The information found here is all likely to be out of date. Please search the new wiki for more up to date information.

Difference between revisions of "Windows Native Client"

From Obsolete Lustre Wiki
Jump to navigationJump to search
 
(7 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
+
<small>''(Updated: Mar 2010)''</small>
 
__TOC__
 
__TOC__
= Windows/Lustre Native Client Project Information =
+
The Windows Native Client (WNC) project is to deliver a port of the Lustre File System client for the Windows operating system. This port is based on Lustre 2.x code. This project will not port the Lustre servers to Windows. It is a port of the Lustre client only.
 
 
== Links ==
 
[[Windows Native Client Questions]]
 
 
 
[http://arch.lustre.org/index.php?title=Metadata_API Metadata API]
 
 
 
[[Windows_Native_Client_Build_Guide|Windows-Lustre Build Guide]]
 
 
 
== Milestones ==
 
- Working Linux Lustre cluster (vmware or otherwise) - Michael MacDonald to assist asap?
 
- Mounting on windows – requires Matt’s branch to work
 
- File Browsing
 
- File I/O
 
- Integrate with CFS testing
 
- Performance
 
 
 
== Methodology ==
 
 
 
OSR has a reasonable idea how to get skeleton code together for 1-3, but needs a review of lock revocation and metadata caches before finalizing 3, from someone like Oleg. I expect some assistance from Matt suggesting skeletons etc can help tremendously. Nikita should help architect a skeleton for the file I/O architecture.
 
 
 
== Contributors: ==
 
 
 
'''OSR'''
 
Dan Root – project manager
 
Tony Mason – design lead, working with Mark
 
Mark Cariddi - developer
 
 
 
'''Sun'''
 
Bryon Neitzel - Project Manager
 
Matt – owns libraries, compiler issues, LNET
 
Nikita – file I/O guidance
 
Alex - metadata API
 
Oleg ? – Metadata guidance
 
Eeb – control landing from the branch
 
Braam / Eeb – architecture, abstractions
 
 
 
== Risks  ==
 
- all risks are fairly likely and highly disruptive
 
 
 
Loss of CFS attention – CFS project manager
 
Poorly defined details – CFS documentation team
 
Out of date libraries – Matt Wu
 
Compiler issues – Matt Wu
 
Code base continues to move – CFS development team
 
Getting it right vs working – Nikita / Oleg
 
Landing into 2.0 – Barton
 
 
 
== Implementation strategy ==
 
 
 
OSR will work on skeletons / prototypes and request review of these when ready. 
 
 
 
== Project management  ==
 
 
 
Frequent status reports with risk update.
 
OSR agreed to weekly status reports and bi-weekly conference calls.
 
  
== Budget ==
+
The Lustre driver for Windows will be based on the [http://www.microsoft.com/whdc/devtools/ifskit/default.mspx Windows Installable Filesystem (IFS) Kit].
  
Peter Braam and Tony were not yet concerned about the budget situation, but the original estimates were too optimistic, in particular because CFS made no progress between October 2007 and April 2008.
+
== Software and Hardware Requirements ==
  
== Meeting History ==
+
''Windows versions:'' Windows Server 2008/2008 R2<br>
=== Windows Client Meeting, Monday, 2008-05-09 ===
+
''Lustre:'' HEAD code/version 2.x<br>
Eric Barton
+
''Supported Networks:'' Ethernet and IB (TCP/IP only)
Peter Bojanic
 
Nikita Danilov
 
Tony Mason
 
Bryon Neitzel
 
Matt Wu
 
Mark Cariddi
 
  
'''Agenda
+
==Currently supported features==
'''
+
These basic operations have been implemented:
1. Progress by Mark - need help building any Lustre component?  ldiskfs or LNET?
+
* Lustre client filesystem mount and dismount
2. What is next step for OSR after Lustre builds?  Recommendation is to use existing metadata APIs
+
* File creation/open/close/rename/getattr/setattr/deletion
3. Progress from Matt Wu on compilers and macro issues
+
* Directory enumeration
4. Status on MD apis from Alex
+
* Directory change notification
5. Review developer questions from Google Doc
+
* Data reading/writing: cached i/o, direct i/o, mmap i/o
 +
* Byte range lock/flock
  
'''Discussion'''
+
==Future plans==
1. Mark will get back to the build process next week.  Matt's note on how to build should help.
+
* GUI tool for management of lustre mounts
2. OSR will start to work on existing metadata api's.
+
* Integration to Windows network drive assignment
3. Matt will finish compiling task next week.  The data api has a low level description that has been published to the mailing list. 
 
4. The MD api hasn't been started.  The MD api definition should work for all porting targets.  Could OSR possibly help with the definition?
 
* OSR will need to ask questions regarding the semantic differences between Windows and Linux. 
 
* OSR would like to have something re: metadata in about a month.  Have been waiting for work product to show up.  Should OSR start with the data api since these are mostly done?  Need to start with metadata.  The biggest thing OSR needs is Matt's work, basic network infrastructure, ptlrpc, LOV, OSC, MDC, MGC, etc in order to actually run something.
 
* OSR could be scoping out design, writing initial code to talk to MGS to get file system info, etc, while waiting for more components to be complete.
 
* Just getting nw connection set up will be first step.
 
* Must allow nw address of the mgs to change.
 
* store nw addresses as strings, not binary.  format will change down the road.
 
* first milestone is to show directory browsing from Windows.
 
* When will the client be exposed to lock callbacks?  This isn't a concern for the first milestone.
 
* Need to find a balance between a fixed price contract and a cost contract.  Hard to estimate in absolute terms how long this will take.
 
* Tony thinks the amount of time won't vary too much, but the elapsed time will depend on how quickly Sun can deliver dependent components and api's
 
* Some re-work will be required when the final MD api is finished.
 
* An important thing is the locking interaction and the required number of rpc's to perform required Windows actions. 
 
*
 
5.
 
  
=== Windows Client Meeting, Monday, 2008-04-21. ===
+
==Download and Support==
Eric Barton
+
The Alpha/Beta release will only be available via the early access program.
Peter Bojanic
 
Nikita Danilov
 
Alex Tomas
 
Matt Wu
 
Bryon Neitzel
 
 
A Google Document was created for questions from the development staff for Braam's meeting with OSR this week.
 
http://docs.google.com/a/lustre.org/Doc?id=d8jgmdp_20fpxwdgb.  Alex and Nikita to document questions by Wednesday.
 
 
 
A branch has been created for use by OSR.  b_winnt_port branch of lustre-core module (sharing HEAD lnet so far)
 
 
Comments from the Meeting:
 
* eeb: need a clean interface that porting will use.
 
* eeb: need metadata interface that doesn't expose internal stuff to another platform
 
* Matt to continue documentation, compile issues, and porting all client components that are common to other platforms (LOV, LDLM, OSC, MDC, LVM, PTLRPC, etc)  Eric  would like to be involved before too much design work is done.
 
* Matt has drafted a doc titled gcc_vc.pdf that describes the differences between the gcc and vc compilers, and some proposed solutions.  This was sent to Dan Root, et al,  on Jan 20.  Eric to provide feedback.
 
* Alex wants to learn more about Windows. understand simple use cases.
 
* would OSR implement the file handle stuff?  (do we need to export an MD FID lookup?)
 
* Get de-brief from Braam on meeting, have him give direction to Alex on md api
 
* Alex to send a list of questions to braam
 
* Put a question in doc about what OSR can be doing?
 
* Need an OSR work plan as a result of Braam meeting, so the Lustre team knows all the dependencies and dates on items we're responsible for.
 
* Do we need an LNET branch?  Give them a snapshot of all of the sources.  Otherwise we don't know what they're implementing to.    Nikita says current client IO can't be used for implementation, as it's not far enough along yet.
 
* Provide branch names: b_winnt _port, HEAD version of LNET.  OSR will need their own private branch when they start to commit code.  Need to sort out branch situation when OSR is ready to check in code.  Their code cannot live in a pubic branch.
 

Latest revision as of 17:44, 9 March 2010

(Updated: Mar 2010)

The Windows Native Client (WNC) project is to deliver a port of the Lustre File System client for the Windows operating system. This port is based on Lustre 2.x code. This project will not port the Lustre servers to Windows. It is a port of the Lustre client only.

The Lustre driver for Windows will be based on the Windows Installable Filesystem (IFS) Kit.

Software and Hardware Requirements

Windows versions: Windows Server 2008/2008 R2
Lustre: HEAD code/version 2.x
Supported Networks: Ethernet and IB (TCP/IP only)

Currently supported features

These basic operations have been implemented:

  • Lustre client filesystem mount and dismount
  • File creation/open/close/rename/getattr/setattr/deletion
  • Directory enumeration
  • Directory change notification
  • Data reading/writing: cached i/o, direct i/o, mmap i/o
  • Byte range lock/flock

Future plans

  • GUI tool for management of lustre mounts
  • Integration to Windows network drive assignment

Download and Support

The Alpha/Beta release will only be available via the early access program.