SAFE NFS - open() capabilities and restrictions

nfs

#1

Mapping POSIX file handling (as used by FUSE, fs etc.) to SAFE NFS has some tricky bits. As a result I’m exploring the limits of the SAFE NFS API and my experiments with git have just pushed past what I assumed those limits might be, by appearing to be able to open files for read and write, and also being able to open the same file multiple times.

Some of this at least appears to work, well doesn’t throw errors in the SAFE API. But I’m not sure what is actually supported, and if there are things I should do in a certain way etc.

I would find this very hard to discern from the code myself, so would appreciate someone who knows expanding on this, and maybe the answers could be fed into the API docs.

So, is it ok to open a file for read and write for example? Consider, the NFS open() mode flags we have are:

NFS_FILE_MODE_READ
NFS_FILE_MODE_APPEND
NFS_FILE_MODE_OVERWRITE

The docs show:

open(file: [File](https://docs.maidsafe.net/safe_app_nodejs/#file), openMode: ([Number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number) | CONSTANTS.NFS_FILE_MODE_OVERWRITE | CONSTANTS.NFS_FILE_MODE_APPEND | CONSTANTS.NFS_FILE_MODE_READ)): [Promise](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise)<[File](https://docs.maidsafe.net/safe_app_nodejs/#file)>

But I don’t understand what the (Number | etc means because it seems different to the parameter description (next), so it would help to clarify what flag combinations are acceptable.

openMode ((Number | CONSTANTS.NFS_FILE_MODE_OVERWRITE | CONSTANTS.NFS_FILE_MODE_APPEND | CONSTANTS.NFS_FILE_MODE_READ) = CONSTANTS.NFS_FILE_MODE_OVERWRITE)

Unfortunately, POSIX options are subtly different, and rely on being able to specify the position of each read and write rather than just having this implied (as with NFS ‘overwrite’ (pos = 0) or ‘append’ (pos = EOF) modes). Here are the POSIX modes:

#define O_RDONLY  00
#define O_WRONLY  01
#define O_RDWR    02

Currently I’m getting away with this mapping:

O_RDONLY to NFS_FILE_MODE_READ
O_WRONLY to NFS_FILE_MODE_APPEND
O_RDWR   to NFS_FILE_MODE_APPEND|CONSTANTS.NFS_FILE_MODE_READ

Fingers crossed this seems to work fairly well with how FUSE does stuff on behalf of various applications, without too much hoop jumping in my code. Phew! :slight_smile:

But now I see git is pushing the boundaries, more threading (ok so far), and opening a file for O_RDWR, and then opening it again multiple times for O_RDONLY! God knows why, but it is doing this, and I’m passing most of that to SAFE NFS and it isn’t throwing errors. I’m not sure if it behaves as expected yet, but… what should I expect?

Some explanation of how SAFE NFS works would be useful, or more details of what is allowed / not allowed / any restrictions for:

  • opening a file for both NFS append and read
  • opening the same file multiple times simultaneously
  • opening a file for append, read, and opening it again for read etc

Thanks.

EDIT: Just remembered another question…

Another thing that happens is requesting file attributes after creation, normally just to check it is there which works ok. But git is also doing this after writing a load of stuff before closing the file, and I don’t save anything to the network until close().

So is there a way to get the actual bytes written (i.e. file size) from a File object before closing it? I need to check if I’m actually getting this atm, but if I am it is coming back as size zero. -will edit this when I’ve confirmed what the code is doing here.


#2

On your pre-edit stuff:

I’m not really sure, to be honest @happybeing.

I don’t think the JS api would have any problem with this per se with your suggestions (although… maybe the FFI lib might… that’s going a bit beyond my realm of ken).

@bochaco can you see any issues from the JS side?

And if FFI doesn’t have a problem with it, I’m not sure on the client lib side, what might be acceptable.

@nbaksalyar or @marcin could you perhaps shed a bit of light on what’s theoretically possible on the Rust side?

And we can see if you come up against anything specific @happybeing that might be a JS specific issue.

I’m also not au fait enough to answer solidly, but I suspect that’s not possible. (happy to be told I’m wrong though :stuck_out_tongue: ) (Though perhaps there’s a way to spoof it if needs be…?)