Changes to TAG_TYPE for MD containers referenced in _public, _documents etc

If I understand correctly, _public, _documents etc are intended only to hold entries who’s value is the address of a Mutable Data that is always going to be accessed by or compatible with ‘NFS’ emulation.

At the moment, those NFS containers are created with TAG_TYPE_WWW (15002) which implies that the tag type could be different depending on the service which created them.

That would be fine if there was a root container per service, because then you would know the tag type needed to retrieve the MD. But if entries in _public have MDs with different tag types, you won’t know which to use.

So either the value of an entry needs to be changed so that it has both the tag type and name needed to get the MD (e.g. using newPublic()) or we should standardise on a type which doesn’t suggest the service type.

In the latter case I suggest changing TAG_TYPE_WWW to TAG_TYPE_NFS (it can still use 15002 of course).

It might be preferable though to have different tag types for different services as well, in which case we could allow MD entries to always allow the option of metadata (by using a JSON object to hold the value of an entry rather than just a raw value). This would add a small overhead, but only needed for the standard containers and any services which want to use the standard containers.

For now though we must use TAG_TYPE_WWW (15002) for any NFS container if it is to be inserted into one of the standard containers.

This seems to be just the same issue we have/had with our _publicNames and the container for subdomains, if we go to full RDF we can cover all the scenarios you described, specially if we have the content addressable URLs support, i.e. you don’t need to specify the type tag but just the link to the MD/ID, and the mapped values don’t have to be NFS compatible but any type of data.

Can you elaborate what that means - what changes and how that will affect the API?

Particularly the NFS API… I’m writing code for this, public names, Web and other services etc. so wondering if that is likely to change much?

I’m also writing JavaScript classes to wrap these types and provide simplified ways of doing common operations so a heads up on any areas to leave until last would be helpful.

Even if it’s just speculation for now, any info could be helpful.

Thanks.

If we consider the current/existing APIs (I’m not considering the ones we created for the WebID/RDF POC) then they shouldn’t be affected, since they either:

  • give access to raw data regardless of its representation (this is the case of MD and ID APIs), or…

  • they provide an emulation in which case the API consumer shouldn’t really care about the data representation (this would be the case of NFS emulation API).

Now if we think of the data itself, we will need to make a decision if we are gonna make such a migration of the data representation onto RDF without supporting backward compatibility, which can be done by simply wiping out all the data and provide the new API implementations with the next alpha net (assuming all data will be wiped out in next alpha net, which I don’t know). I personally prefer this option since there won’t be really value in providing backward compatibility for data representation at this stage.

If I had to suggest you what approach to take for the utilities you are writing perhaps is simply make sure the API you are exposing provides a good abstraction to the consumer/user which won’t be affected by a change in the internal data representation we finally decide to go with, and make the implementation of such APIs to be adaptable as easy as possible. We started this week with some planning about this, but we’ll have to go thru several steps in analysing which RDF schemas we wanna use, serialisation format, etc. We are also thinking of providing utilities like the ones we already created for the POC, we just need to now sit down and carefully design the API since for the POC we just made sure we had the APIs we needed to showcase the WebID demo apps

2 Likes

Thanks, very helpful. I’m thinking then that anything using NFS emulation should survive, but anything creating services or public names, or adding shares to _public could break (for example if you change the nature of those MD keys or values).

Also anything using MutD/ImmD for its own purposes not affected.

Any updates on these areas would be appreciated :slight_smile:

Also, likely any existing data will be wiped as expected between testnets.

1 Like

Can you elaborate more about this, because this doesn’t correspond to my observations?

What I observe is that all NFS containers like the ones you mentioned ( _public,
_documents) are created before a service is added by user.

I did my observations with a modified safe_vault program that logs the number of MDs grouped by tag type:

  • When an account is created, 8 containers (type_tag=15000) and one account packet are created (type_tag=0).

  • When the user authorizes the browser and WHM, 1 additional container is created (a common one shared by both).

  • When the user creates a public id, 1 DNS entry is created (type_tag=15001)

At this point, no WWW services have been created (type_tag=15002). It is only when the user uploads a web site that one is created. Furthermore, no additional containers are created when doing so.

I suspect the problem you see is related to MD values, but I don’t know the general graph structure between MDs. Could you explain the links between MDs? (or someone from @maidsafe).

1 Like

I’m referring to the containers created for services rather than the root containers themselves (which are not NFS containers). So at the moment, if you want an NFS container to be accessible by looking up the entry in one of the root containers, you can only do so if you know the tag type in the MD (which is pointed to by the the value of a root container entry). Currently this is assumed to be TAGTYPE_WWW, whether or not it was created for a www service.

If I wanted to use a different tag type for a different kind of service (eg LDP), and store that in _public so the files could be seen by other apps, the Web Hosting Manager and any other app would not understand it, so I’m forced to use TAGTYPE_WWW.

Thanks for providing the additional details on root containers, you are revealing info I hadn’t gleaned myself.

Feel free to ask more if my answer isn’t clear - just off to watch a film right now.

1 Like

I suppose that @maidsafe did this because of the 1000 entries limit on MDs.

A user might want to manage many safe sites under his account. This limit is acceptable for the files of one safe site, but not for all his safe sites if the files were directly stored in the ‘_public’ container (which would be the natural place to share them between several services).

More generally, a user might want to manage several public names possibly each with several services and storing all their files in the same container would easily hit this limit. This is the reason why, I think, there is one distinct container per (public name, service) couple.

But, unless I am missing something, it should be theoretically possible to share the same container between several services, from the same public name or not. An app could be developed to do that.

Currently a service container is already shared: It is pointed to from both the ‘_public’ container and the services container corresponding to the public name. There could be additional pointers from the ‘_public’ container and from the same services container (for other services within the same public name) or even from other services container (for other public names).

This means that different paths would lead to the same container (because public name and service name are part of the path), but I don’t think this is a problem.

This is essentially what I’m arguing for (not the app, but a shared tag type for different services which use the same kind of container). So these should all be tagged as NFS if that’s what they are rather than www as they are now, because that suggests each service should use a different tag type which is not correct, and as I noted will break.

1 Like

OK, I understand: to allow several kinds of apps to share a common folder (which is what we both propose) a common generic type tag is needed and TAGTYPE_WWW is too specific. Renaming this constant into something more generic would be beneficial, like TAGTYPE_GENERIC_NFS for example (but service name for WHM can remain “www”).

I don’t think this naming problem challenges what I propose to share a folder between several services or/and public names.

1 Like