Dumb DOM API questions

This proposal transform the mutable data in a new kind of immutable data. That’s change one basic purpose of MD and loose privacy.
Hope could delete, at least, previous versions.

I see. But now we have a list of item that can only grow overtime. It’s not very practical as a list. For example, the MD for NFS will be filled with empty field pretty fast.
Sounds like the problem of data disappearing needs to be fixed elsewhere, if at all. Maybe apps that needs it should figure out a protocol using immutable data for this kind of problem. Idk.

Its just that the impact on the list of entries is too big IMHO.

I think the solution is two kinds of MD:

  • one which exhibits current behaviour, and is useful where keys rarely or are never needing to be removed
  • or where keys can be deleted

In the mean time it is possible to simulate the latter if you need it, at the cost of not being able to use the built-in NFS API ‘simulate as’ functionality.

You could do this by storing your index in a single MD key, but since the cost of updating this would likely be the same as the equivalent size immutable data, it might be more sensible to use ID for an index rather than MD. You would then have the option of also storing references to your ID as a history, giving rollback/archive functionality too.

I can see good reasons for building your own layer rather than using the built in features (such as emulate as NFS). For ‘serious’ applications, you could optimise the storage to minimise PUT costs by moving infrequently used data together, and frequently changed data together for example.

So we can do what we need with the existing MD & ID, which means we don’t have a good case for changing the API for this at least.

1 Like

That sound backward, use an immutable data when you need to remove items for a list and mutable data when you want a immutable list… It’s not an elegant intuitive solution and it makes the NFS api unusable in the long term as a file system, that should hint that something is off in the design.

If we need an immutable list of item, create a key value list within a ID where the value points to an MD that can be modified.

So I don’t see why it should be the default behavior of MD when it’s doable with an ID in the rare case that you’ll need it.

My 2c anyway, I understand I might be missing something.

Edit: wait, can’t we create a MD without the delete permission for undeletable list?

3 Likes

Just to be sure there is no misunderstanding: The use case for clearing entries instead of deleting them, is not for immutable data, but is for mutable data for which we want to know if it is has been modified. Hard delete implementation doesn’t allow this because an entry could be recreated with the same version as before but a different content.

I’m with you. Could we achieve the same result with a MD that has no owner and no delete permission? Or, not really sure if it’s possible but, a MD set in a way that the owner can update, insert but not delete?

3 Likes

You can also add: without update permissions permission. But yes, it should work. I think you have found the solution to allow both usage:

  • no possibility to delete an entry when there is a need to prove non alteration (but an entry can still be cleared with an update operation)

  • efficient hard delete otherwise.

4 Likes

Cool, glad I could help :slight_smile:

Edit: actually, @bochaco what do you think?

2 Likes

This would seem to work for MDs created for arbitrary purposes, but am I right in thinking it won’t help applications which want to use the standard containers (to store files for example)? Unless those are made to allow delete, which in not sure would be acceptable.

I’m not sure I understand, what would be a use case for wanting the root container to have un deletable file names? .( that’s what you mean right?)

I’m not sure, but I’m guessing that MaidSafe made these design choices for a reason. Maybe so you can implement a time-machine style rollback backup perhaps, but over to them! :slight_smile:

1 Like

I also don’t understand. What is proposed here is possible right now for any MD types, including standard containers: if at one time a MD has no delete permission and no modify permissions permission granted, then current entries cannot be modified covertly (meaning that they can be modified, but the entry version will be incremented with no possibility to cheat). If this is confirmed by @maidsafe, I think that delete operations can be implemented by a hard delete.

It is possible that current client tools don’t allow removing these permissions. But it’s only a matter of HMI to be developed.

My point is not that this couldn’t be applied to the root container. I agree with what you say there.

My point is that MaidSafe maybe arrived at this behaviour because they want the existing soft delete behaviour for those containers. In which case, any app that writes keys to them will not be able to make use of hard delete for those (root) containers. Hence, an app storing files within those root containers will not benefit from the option to hard delete file entries, since they won’t be allowed in the root containers. I don’t know that to be the case, so only speculating that this innovation won’t help that case, which is probably a very common one.

1 Like

Maybe a bug, feature or rookie coding error. All help appreciated!

I have some code which looks roughly as below. I iterate over the MD entries of the root container and use safeNfs.fetch() get info related to those entries where I’ve used NFS emulation to insert() immutable files. The function _makeFileInfo() which uses the fileHandle to get metadata for the file and returns a Promise. The iteration works and a correct value gets inserted at listing[ name ].

But the code does not produce the desired result after the iteration. After the ‘Iterations finished’ statement, listing is empty.

So it looks as if the .then of the forEach() iterator completes before the promises created inside the loop have completed (ie the call to fetch() shown, and the one returned by _makeFileInfo(). So I have two questions:

  • is this a bug in the forEach() implementation, or should the iterator complete regardless of the state of the Promise created by the fetch and _makeFileInfo()?
  • if it isn’t a bug in forEach() implementation, what would the correct structure be here (without using await if possible)?
window.safeMutableData.getEntries(self.mdRoot)
.then((entriesHandle) => window.safeMutableDataEntries.forEach(entriesHandle, (k, v) => {

  [snip]
  return window.safeNfs.fetch(self.nfsRoot, fullItemPath)
  .then((fileHandle) => self._makeFileInfo(fileHandle, fileInfo, fullItemPath)
  .then((fileInfo) => {
    RS.log('file version: ' + fileInfo.version);
    listing[name] = fileInfo;
  }));
}).then(_ => {
  RS.log('Iteration finished');
  RS.log('RESULT: listing contains ' + JSON.stringify( listing ) );
}, function (err){
  [snip]

For reference, here’s _makeFileInfo():

      _makeFileInfo: function (fileHandle, fileInfo, fullPath){
      return new Promise((resolve,reject) => {

        window.safeNfsFile.metadata(fileHandle)
        .then((fileMetadata) => {
            fileInfo.created = fileMetadata.created;
            fileInfo.modified = fileMetadata.modified;
            fileInfo.version = fileMetadata.version;
            fileInfo.dataMapName = fileMetadata.dataMapName; // mrhTODO Debug only!

            // Overwrite ETag using the file version (rather than the enry version)
            fileInfo.ETag = fullPath + '-v' + fileMetadata.version;
            resolve(fileInfo);
          }, function (err){
            RS.log('_makeFileInfo(' + fullPath + ') > safeNfsFile.metadata() FAILED: ' + err)
            reject(err);
          });
        });
    },

The function invoked in the forEach loop is not expecting a Promise to be returned and resolved therefore the loop finishes before the Promises returned in each iteration are resolved.
So that’s expected, you can see in the web hosting app how this type of things are done here: https://github.com/maidsafe/safe_examples/blob/master/web_hosting_manager/app/lib/api.js#L159

1 Like

Thanks again Gabriel. Just the help I needed :slight_smile:

1 Like

This thread was made for people like me!

Okay, so I see everyone throwing around MDs that can be shared. Hey, share with your friend, share with yourself, share with nobody. This all sounds fantastic! I’d love to share some MDs with myself across app sessions, more than just using the Own container. I absolutely see people saying this is a thing that happens.

I’m parsing through the breaker-plugin-safe-app 0.3.0 API. Perhaps I need to be pointed to a more relevant source?

ImmutableData has an XoR name. You can fetch() it. Sweet! I can share IDs! I can create an ID, I can get the XoR name for that ID, I can share an ID by sending someone the XoR name, and if someone gives me an XoR name I can fetch it!

I know I can create an MD (lots of ways, newPublic*, newPrivate*, toss in a bit of quickSetup for seasoning). I know I can fetch named containers with App.getContainer and App.getOwnContainer. Great! Okay. Now I’ve got this MD and I want to stash some kind of reference to it in … I dunno let’s say my Own container … so that I can access it in later sessions. Or I want to send it to Bob from Accounting.

Where and how does this actually occur? There is only one reference to an XoR name in that entire API relating to MD: safeApp.authoriseShareMd(). How do I get the XoR name for an MD that I’ve created? I assume once I’ve got the XoR name, I share the MD with others using authoriseShareMd()? Honestly the docs for that function say “authorise this application with specific mutation permissions for a MutableData” and not “fetch this MutableData from the Network, also btw we’ll use these permissions.” It sure doesn’t sound like it actually helps you get MD off the Network. Not that I know how to get an MD XoR.

From what I’ve seen of handles, those appear to be transient, app-specific, volatile references. I’m not going to create an MD, store its handle to my Own container, and make use of that handle later, right? Or is that the vital piece I’ve completely missed? If so, why is ID different?

2 Likes

Take a look at this: Accessing stored data without handle

2 Likes

Yup, that appears to be it. The API docs I’m looking at don’t explain either of those functions that well. Name (of name and tag) sounds like it could be any old thing you make up, as opposed to an XoR address situation. Then newPublic makes it sound like it always creates a new Public MD; I thought setting the name and tag was just a convenience thing, not a way to load data from the network.