Multisig Revamp for StructuredData/AppendableData


#1

Part 0 - Usability:

We will refer to Structured or Appendable Data as ShareableData for the purpose of this discussion as it concerns only the ownership aspects.

The present ownership evaluation by vaults is impractical for ownership sharing.

  1. Let S be the ShareableData created and thus owned by a. If a wants to share it with b, c, d and e so that it would have a total of 5 owners now, it needs to add their sign::PublicKey (henceforth sPK) and itself too to the current-owners field while also adding itself to previous-owners field too for vaults to grant ownership to all 5.

  2. If any of them wanted to modify S they would need to circulate it to al-least 3 others before the data was finally POST'd to the network else vaults would invalidate the update as an InvalidSuccessor. While this may be useful for certain cases, it is completely impractical for many others. E.g. if S was a shared-dir it is meaningless to expect majority to agree each time anyone wants to make changes. Also circulation would be out-of-band to the actual POST operation implying there is no certainty when it would be done (as it depends on a conscious decision of other parties).

  3. Once successfully shared, b, c, d and e could agree among themselves to remove a out of ownership. This again may not represent all the use cases as in many you would want the original owner/creator a to be incharge of ownership revocation.

Point (2) is the main one here. I’ll provide a baseline structure addressing it from where we can flag off our discussions. A weight-age system seems to be a nice starting point. This (as far as I am aware of) was originally hinted by @Fraser. Putting that idea along with a few others to address other points too together:

struct Owner {
    key: sPK,
    data_modification_weight: f32, // [0, 100] => 0% to 100%
    new_owner_add_weight: f32, // [0, 100]% - If this owner is allowed to grant ownership to others
    prev_owner_alter_weight: f32, // [0, 100]% - If this owner is allowed to revoke OR modify ownership of others 
}
    

ShareableData {
    //  .... other fields ...
    current_owners: HashSet<Owner>,
    prev_owners: HashSet<Owner>,
    signatures: HashSet<sign::Signature>,
}

The vault’s would check signatures would be:

  • If data field is modified: Total current owners in the copy of ShareableData present with the vault = n. This must be equal to either prev_owners in incoming data OR current_owners if the prev_owners is empty (Nothing different so far, just like the current implementation). Start checking signatures. As you validate singature of a particular owner, keep summing up data_modification_weight for that owner to the previous value.
  • Similarly if it is detected that there is an addition of new Owner check for corresponding weightage.
  • Finally if it is detected that there is an deletion of an existing Owner check for corresponding weightage.

Roughly:

let data_modified: bool = check_this();
let new_owners_added: bool = check_this(); // diff current_owners with previous_owners.
let prev_owners_removed: bool = check_this(); // diff current_owners with previous_owners.
let prev_owners_modified: bool = check_this(); // diff current_owners with previous_owners.
let prev_owners_altered = prev_owners_removed || prev_owners_modified;

let mut total_data_vote = 0;
let mut total_owner_add_vote = 0;
let mut total_owner_alter_vote = 0;

for owner in &stored_S.current_owners {
    if owner_has_signed_the_incoming_S {
         if data_modified && total_data_vote < 100 {
             total_data_vote += owner.data_modification_weight;
         }
         if new_owners_added && total_owner_add_vote < 100 {
             total_owner_add_vote += owner.new_owner_add_weight;
         }
         if prev_owners_altered && total_owner_alter_vote < 100 {
             total_owner_alter_vote += owner.prev_owner_alter_weight;
         }
     }
}
if data_modified && total_data_vote < 100 ||
   new_owners_added && total_owner_add_vote < 100 ||
   prev_owners_altered && total_owner_alter_vote < 100 {
     return Err(MutationError::InvalidSuccessor);
}
// ... Continue whatever it currently does

When you add an owner (POST operation, {For PUTs vaults don’t evaluate anything}), the following 3 fields:

new_owner_add_weight: f32,
prev_owner_alter_weight: f32,

must be 0. To make it non-0 for that new owner a fresh post must be required. This is to prevent misuse in which you can add new owner and form alliance with them to oust an existing one. Say a had 100% weight in eveything but b and c had 100 % for adding new owners but 40% each for removal. They could easily add a new owner d, give him > 20 % revocation power and oust a as b, c and d would be able to. To prevent this misuse, all weights apart from data-modification which is not as serious, should be 0 for new owner.

This design has great flexibility and answers many more problems than the current one.

  • For those who want to keep it simple and retain ownership would essentially grant 0 weight to the new owners they add except perhaps for the data field - e.g. private share by someone - this itself could be a major use case - One wouldn’t have to circulate data as each could have 100% weight for data-manipulation,
  • A particular forum owner a can add many people as moderators (who could add more moderators but not remove them) by giving them permission to add as 100% (unless a want to be involved too in decision making) and 0% as permission to remove etc.
  • For those who want more granularity, they can calculate the percentage accordingly.

I hope this provides some starting point from where we can take things further - chop things off (if we think it is complicated) or add to it or modify according to taste.

@nbaksalyar @madadam @Fraser @AndreasF @dirvine @Viv @vinipsmaker @Krishna @Qi_Ma (feel free to tag more if i missed someone).


#2

Part 1 - Performance:

Currently signatures is a separate field, and other fields are unsorted vectors. This leads to following inefficiencies:

  1. O(n^2) check for duplicates.

  2. O(n^2) check for verification. Each false hit entails a throw-away signature verification.

This can be optimised if we use a HashMap instead of a vector and integrate signatures field with owners field. The ShareableData would now take the form:

ShareableData {
    //  .... other fields ...
    current_owners: HashMap<Owner, Option<Signature>>,
    prev_owners: HashMap<Owner, Option<Signature>>,
}

Those opting to sign would put the signature mapped to their ownership instead of in a different field. Data to sign would be same as previously:

sign(other_fields + current_owners.keys() + previous_owners.keys());

This eliminates an O(n^2) here and turns this from O(n^2) to O(n).


#3

27 posts were split to a new topic: Transparency or opacity of SD modifications


Transparency or opacity of SD modifications
#5

I think I misread sorry


#7

To allow someone to read you just give the encryption key if it was encrypted. Others can be categorised as modification operation. I don’t know how useful is it to subdivide it further into write/modify/append etc. What you are suggesting can easily be achieved giving 100% ownership to everyone you want to share a particular facet with, so i am not sure i would call it imprecise (rather there is too much precision and that can be a problem with complexity). OTOH if we implement what you suggest then the opposite use-case of consensus-involved activity cannot be implemented which is equally important to support.


#10

I had misread the use of weights. You must have replied before I deleted that post. So sorry again for my misreading.


#12

Hey np at all man :smiley: - even if you misread it could also mean my explanation wasn’t precise, so i am all for anything i can write more to make things (from my perspective of things) clearer (unless i am sleepy, which i am now)


#13

Assuming someone only has half the weight to make a change, requiring others to also confirm the change, presumably a mutation error should not be returned in those circumstances? Will this message be different in the future and will other owners be notified in some way to confirm the change?

This may be going beyond what is being described here, but it would be good to know how these use cases will be handled.


#16

I can say two things on this right now:

  1. There is no push model yet, so vaults (at-least for now) don’t send notifications.

  2. Illegal modification attempt will generate a mutation error right now. The person with limited weight will have to circulate the ShareableData to enough ppl to get to 100%. This has to be done out-of-band. So say you make changes and know that if i agree too, it will be accepted (will reach >= 100%). You can serialise it and send it to me using the SAFE Email app we released. When i get it, deserialise it, read the changes and agree to the changes, i will add my signature too and then POST it. It will now be accepted by vaults.


#37

Thanks - it is great to hear that 2 will be (is already?) possible, even without push. I suspect a nice client could wrap this process, even before push is available.


#38

Yes, the code in safe_core should already be there. All there needs to be done is expose it via FFI (API interface) to Launcher - whenever it demands. Infact most of this interface should already be there too, let’s see:

  1. Create an SD.

  2. Serialise it.

  3. Use the SAFE-email-app (just as an e.g.) to send it to the other you think necessary.

  4. They deserialise this to get back the SD.

  5. Check they agree to the changes. Other API’s will be exposed here if this discussion is fruitful and new fields as suggested in this thread are accepted in some form - then expose some more API’s to verify the changes in those fields too, not just data. Probably person shooting the mail should hint what’s changed so ppl receiving know what to look for. If they don’t trust the guy then they can always get the latest from vaults to check the diff.

  6. Add their signature (FFI API currently missing for this - but it’s trivial as you can guess and will put in soon, when Launcher/frontend requires it).

  7. Repeat 2 - 6 if more signatures/permissions are required.

  8. POST the final SD - hopefully should be accepted by vaults.


#39

I’m broadly in favour of this. A couple of thoughts though:

  1. I’ve never been a fan of the prev_owners field. I think the only place where that’s needed is when a mutation is being validated, and that check is done by the data holders who already hold the previous version and hence already have the set of previous owners’ keys. This is probably a separate topic though.

  2. By requiring >= 100% weight to modify data, we’re requiring at least one signature. This is probably fine - just pointing it out.


#40

All the goat sacrifices i made seem to have worked :smiley: - your approval is a rare thing !

It’s during transfer/change of ownership i think. So if you want to transfer ownership to me your SD Post would look like:

current_owners: My-Onwer-Struct, // Or your's too if you wanted a shared ownership instead of just transferring.
previous_owners: Your-Onwer-Struct,

So in this case vaults verifiy the signature against prev_owners (instead of usual current_owners). When they store data at vaults though they blank out the pre_owners so that field in current implementation is always blank if you look into Valuts (at-least that is my knowledge of it).

What do you propose here (which should be able to do all that the current one does) ? A separate API for ownership transfer ? Difference in representation in stored vs serialised version ? Maybe a Type/API snippet might be helpful if you can provide one.

Just curious : Why the probably ? Are we missing something ?


#41

Just to be sure - you agree to both parts right ? (Part-0 and 1)


#42

Simply removing previous_owners from SD is all I’m recommending. I believe SD could do with a complete overhaul, so I’m not going to spend time here detailing what I think it should be - that’s a separate RFC in itself! :slight_smile: But if you imagine that the only change to SD’s API is that previous_owner_keys is removed from new().

To transfer ownership, Clients would continue to behave as they do now - construct a new SD from the existing one’s fields, swapping out the public keys for the new ones. They add signatures using their old keys.

The Vaults handling the request already have the existing version stored - they’re using this existing version to call validate_self_against_successor(). There’s no need for other to provide the previous set of public keys there - we even go as far as checking that they’re the same at the moment, even though we go on to validate the signatures (which would fail if they weren’t the right keys)!


#43

Yes - I agree to both parts!


#44

Good point - that makes me think that non-ownership-transferring POST updates could even blank out the current_owners. So if current_owners is blank, vaults continue to use the stored current_owners. If it has something then swap it (transfer of ownership kind of stuff). That would basically make majority of the POSTs not carry that field at all - some (depending on number of owners) reduction in B/W ? Will that have issues ?


#45

Not that I can see, but I’m usually wrong :smiley:


#46

Ah @Fraser just realised one thing: dropping previous_owners and the rest would invalidate Part - 1 which you had agreed to as well.


#47

Ah - right - good point. I guess I’d opt for dropping the previous_owners (reducing message size and saving space on disk) at the expense of the processing inefficiencies you pointed out. I guess we could always change to ordering the Vec of keys and Vec of signatures to match, so the only waste would be if we provide less signatures than keys we’d still be doing O(n) signature validations where n is the number of keys rather than the number of signatures.