File sharing at the level of safe: URLs

API
21

Files are forever stored in the network and this is a costly feature for the network in the long term. It would be unfortunate to not use this feature by generating URL that aren’t forever valid.

First, a concrete application is the end of favorites list with links that have become invalid in them.

Then, there is the transparency of the network. I have argued a lot in the name of transparency to reimplement SD deletion to prevent recreation of a SD with same key, same value but a different content. It was hard to convince Maidsafe but finally it was accepted. Here I feel the same and I think that sending a link that you can invalidate later is not transparent.

Imagine that Maidsafe implements both kind of links to satisfy both of us. Which one do you think users will prefer? Permanent one or removable one?

Maybe some companies and governments could prefer removable ones because they want to be able to remove some information that have become compromising to them. But I am sure that consumers and citizen would prefer permanent ones, simply because this is one of the founding features of the network.

This is the reason I like file URLs implemented as base64 encoded datamaps. Like hashes in the bitcoin blockchain they cannot be compromised. They are bigger but they are still manageable and above all, they give access to the files, contrary to the hashes in the blockchain.

Another argument, but a technical one for @ben,

I suppose that @happybeing wants links that can be invalidated by a deliberate action of the sender and certainly not as a side effect of another unrelated command. I am not sure of the object whose address you think can be used as the URL, but I am afraid that a simple action as deleting or adding another file in the same directory can invalidate the URL.

4 Likes
22

@tfa I appreciate your thoughts and don’t have strong feelings either way at this time - I’d like to hear more views and explore the pros and cons of the different behaviors, but am aware that it is already the case that websites will be mutable, so I’m unclear of the value of just this one aspect being permanent.

But, shall we open a topic on the other forum for this - so we can get more voices in the discussion?

1 Like
23

Maybe it’s a bit early for that, but you can do this if you want.

The standard process for adding an evolution to the network seems to be: creation of a RFC, opening a new topic in this forum for discussion and lastly a link is created on the other forum (often in a MaidSafe Dev Update).

I think this process should be followed here and that a RFC should be written, preferably by MaidSafe team because there are technical constraints for the general concept of URLs to be implemented inside beaker browser but also outside it and even outside safe network (and, I don’t have time to do it).

The link to the RFC can then be added in this topic and a link to this topic added in the other forum.

2 Likes
24

Well,

so, the currently discussed New Authentication Flow RFC proposes in its appendix about Containers that NFS containers are

mapping to the serialised version of either XOR-Address in the network
[…]
The XOR-Address should point to another container following the same convention as its parents or to a serialised file struct as described before.

Nowhere is that this data at the address must continue to be SD-Data, it could very well also be a serialised File-Struct as ImmutableData. However, that would mean any change to a file (even its name) would have to create a new ImmutableData in the system and link it anew in the surrounding Container. We haven’t decided yet whether we’d like to favor SD-Data or ImmutableData for this in NFS.

However, if you wanted to share a permanent URI to that specific version, you could do that easily by just storing it as a ImmutableData. That would never change or be removed from the network. But that might not be what you want either.

Taking the example of the favorites from before, if you’d store a permanent URI (or, for that matter, you could also store the serialised data yourself and not need the URI nor an ImmutableData yourself), you’d never “know” about updates to it. In these cases a DNS-based “URL” is probably more useful, even if they sometimes go dead. That said, a browser could easily, for every page you are on, offer to store the retrieved field-struct. But then the internal links are most-likely still going to point to DNS-lookup locations so … It might not render when they break. Or you’d have to replace all those links (including in all CSS and such) with similar serialised-file-structs.

That’s entire possible though. So we could additionally have a safe:data-struct:BASE64(serialised(FileStruct)) that the browser could undertand. It will be much bigger than all other proposed URLs though as it would contain the entire content…

Coming back out of the rabbit hole. I think we should make a clear distinction between what is a permanent link (address to an ImmutableData) and a mutating link (address to a SD) and the system might not provide both without having to do further network puts. However, this can still all very easily be done within an NFS layer.

I beg to differ. I do believe you can write a perfectly acceptable RFC about this. There is no need to understand all the underlying parts, you can really just describe the motivation, the API and URL-format and the expectation of what should happen (even from a higher level view) when that is called. Like: “we should add another NFS api call to allow creation of URLs for sharing like getPublicLink(fileName: String, permanent: bool) -> Future<String>, which returns a URI of” … yadaya.

Once proposed and in discussion, if there are any technical reasons to change things or the underlying network can’t support certain things - I think I said it clearly enough, that this entirely possible - it can be changed. But with a formal RFC the team, especially the people working on the lower parts, have a much better idea what is proposed and can give their input. And once they agreed on it we could make that happen, or even leave it to community members, who’d like to get more familiar with the code base (I’m happy to mentor this)!

2 Likes
25

OK @ben it shifts from I don’t think I have the low level understanding to: I’ll see if I can have a go within a reasonable time. I’d like to see this happen so I am motivated. And thanks for offering to mentor, that would be great.

@tfa that’s fine with me, so I won’t open it up on the other forum at this point. If you want to elaborate further in this thread by all means do, or wait for an RFC if you prefer.

6 Likes
26

If I am correct the File structure contains metadata in addition to the DataMap I proposed, so this would fine with me.

But it doesn’t really contain the entire content because it’s size would be the cumulated size of chunk details, so approximatively (4 + 32 + 32 + 8) * 4 / 3 ≈ 100 bytes per MB of compressed content (plus a small offset).

1 Like
27

That is correct.

That is correct, too. I was merely comparing it to the size that the other URIs would have (scheme+32hex chars). Compared to that these URIs are bigger and would proportionally in size when the file size (or its metadata) grow. That could still all be fine though.

2 Likes
28

This topic culminated in the following RFC which I’d like to revive if possible as a proposed enhancement to the API (nodejs and DOM).

I’m not sure if @ben is still around, but @bochaco, maybe you can comment on whether this is on the horizon, superceded, forgotten (!), or if there is reason to start again rather than pick this up where it left off?

[Gabriel, I’m afraid this is what happens when I hit a development blocker and have time on my hands, sorry :stuck_out_tongue_winking_eye:]

4 Likes