Purpose of MD Serialize

mutabledata

#1

When the first example of C# email app came, you could see that serialized MD:s were added to the app container.
Back then I thought that it was a strange way of getting the MD address in serialized form, because I didn’t imagine that we would want to store an entire MD within another MD. At the time I did it the same way. And I have seen others do this as well (decorum code for example).

Quite recently I realised that the serialize API didn’t give the serialized address, but instead the entire MD. So in my most recent code I store the MD address and type tag, whenever I need to store a reference to an MD.

But now I wonder, what is the purpose of storing a serialized MD into another MD? The serialized one will be variable in size, could fill up the entire MD, and instead we could just store a constant size reference in form of xor address + type tag. The data is already in the network, why store it in the network again?

The only reason I would want to serialize an MD now, is if I want to store it outside of the network, but then I might use any serialization that I prefer. And so I also wonder if there really is a need for the serialization API?


#2

Maybe that MDs can change (mutate) and you want the current contents saved away or to be able to use the contents at a later time.

But I thought at the time it was a way for the APP to read and use the MD contents. I’m pretty sure this is from SD days when there were no fields and one had to read the whole SD


#3

That would be very weird @oetyng, can you please explain how you confirmed the entire MD with its entries is being serlialised for you?

I admit I was also confused about this at the very begining, but AFAIK the serialise function only serialises the MutableData information, a.k.a. MDataInfo: https://github.com/maidsafe/safe_client_libs/blob/master/safe_core/src/ffi/mod.rs#L46

So this is how I see this now that we are inroducing the XOR-URLs:

  • Public MutableData: the only benefit of using the serialise function would be to have in a single string both the xorname and type tag of the MD, thus if you go this way and store it in another MD it wouldn’t be storing the whole MD but just the xorname and type tag, the rest of the data in the MDataInfo will be empty as there is no encryption keys. You can of course do what you were saying and storing the xorname and type tag using your own defined serialisation, but it’s clear that goes against the whole idea of linked-data and RDF formats. Therefore, with the new experimental APIs, it would make more sense to use XOR-URLs for storing a reference to a MD, which it’s what linked-data it really is, just URLs.
  • Private MutableData: in this case it would still make sense to use the serialise function since it would create a string with not only the xorname and type tag but also the encryption keys that you’d need to then be able to decrypt its entries. I’d argue that even in this case XOR-URLs should be used for priv MD, but it’s not clear to me yet what’d be the best way to keep the encrpytion keys along with the XOR-URL, quick answer (thinking in terms of RDF) is by simply having other triples in the same graph, where the “object”'s are the encryption key and nonce. Or perhaps, encoding the ecryption key and nonce in the XOR-URL itself, but once you share it, the priv MD is not really private anymore but simply encrypted with shared encryption keys, it could still work.

#4

Ah, that’s right, the MDataInfo, not all of it with entries. (Which invalidates much of what I wrote).

The things you bring up in the context of xor-urls and rdf are very interesting, I hadn’t thought of them in this. If the xor-url contains it all, then that could replace the serialise function.

But as you mention it, I’m just thinking that it seems less intuitive to include the key and nonce in the url, since they are not needed for locating the md. So that would be giving the url a bit new responsibility and meaning. Maybe could be a good trade off anyway, just thinking out loud here. (I’d need to get more into rdf-in-mds to see how that would compare to the cost of keeping it in other triples.)

Thanks for clarifying, and shedding some light on what it could look like with the new parts as well.


#5

Now worries, I’m also very excited about the new parts and how we can start putting all these things together.

I fully agree with this, I believe the URL should contain only what’s needed to locate the content.