Just a simply query on what is the actual max file size supported by NFS emulation. I know that it is an abstraction upon MData, which has a max file size of 1MiB if I recall, but does NFS enable larger files than this limit?
In extension to that, if it can handle larger files, how have people found the performance to be with larger files?
Yes, because the files themselves are stored indirectly.
I.e. what is stored in MData itself is just a file descriptor structure, which you can find here. The data_map_name field points to a special structure, a data map, which can be thought of as a private encryption key and also a pointer to a data location. This data structure is created by our self-encryption library, which splits the file into chunks and stores them as separate Immutable Data objects.
These chunks themselves are limited to roughly 1 MiB in size for now.
The trick is, as the file data can be split into multiple Immutable Data objects (the logic for it is defined here), the file size becomes virtually unlimited. However, it will be limited by the size of the data map, which itself is stored in an Immutable Data object, hence the max. size of a data map is 1 MiB.
Edit: also worth noting that of course the data map itself can be split into chunks, of course, but that can quickly become impractical as far as the CPU usage and the network bandwidth are concerned.
Okay I see what you are saying. So if my understanding is correct, something like http://docs.maidsafe.net/safe_app_nodejs/#writer can write an almost arbitrary amount of data to the network? As the self_encryption library will chunk all the data in the background.