JAMS Technical Discussion

I can, playlists for example.

I think the word ‘services’ is misleading in this respect - or rather it isn’t immediately obvious what a service means in a serverless system!

I think I of a SAFE service as a protocol which knows how to do things with a particular type of SAFE storage.

So www takes a URI, accesses a file via an MD that it expects to find has been created with entries in a certain format. It also knows to add ‘index.html’ to the URI if it ends in a ‘/’. It happens that the browser knows how to implement part of the service (retrieval), while the Web Hosting Manager implements storage and retrieval. They both have this behaviour hard coded, but using a services library makes it possible for any app to access www data through a standard RESTful interface. Web servers support this through simple GET/POST etc, WebDav was a standardised service for storage, and the Linked Data Platform (LDP) is yet another - all similar ways of putting and getting data from a Web server through slightly different protocols.

Unfortunately lots of functionality has tended to end up on servers, so these simple protocols have been replaced by complex libraries, and lots of code running on a server to implement them. With SAFE and SOLID, I think the idea is to get back to TimBL’s original vision, have simple RESTful protocols that just know how to store and get data in standard ways such as LDP, and let the client do the manipulation and processing of that data.

This is the separation between data processing (app) and data storage (platform) where the user can choose where to store their data, and which app to use with it.

A Playlist Service

So imagine a playlist service. This would expect some kind of label (really also a URI) and the service knows how these are stored also. So it can save one for you, and if you try and load one, it knows what to do with it - load it into JAMS for example, or another music player you have registered for that service. So if you and others build support for JAMS playlists into your apps, all you need to do is include the services library in your app. This may be best handled in a different way - playlist files on a standard storage device like LDP for example. I’m still trying to understand it myself TBH, but I think it is worth thinking about this so that different apps can more easily store and use data for use with other apps.

At least, this is how I envisage SAFE services working - I don’t think MaidSafe have described this anywhere yet so I’m going out on a limb a bit. We’ll find out soon enough, when I have the LDP service working we can see what they think of it :slight_smile:

I’d be interested to here from Maidsafe where I’m right and where I’m wrong in this, but obvs they are busy and I expect we’ll get into this more when I have some working code.

In the short term I think you and other app devs can ensure a degree of interoperability by sticking close to the SAFE NFS storage model, as used by the Web Hosting Manager, and we can build services on top of that where standard client side processing is useful, such as LDP.

4 Likes

I’ve thought of something similar before but was under the impression that would be a JAMS API. Where another SAFE app could do a JAMS API call for our player, and/or playlist, other possibilities, and it shows up (with some JAMS branding possibly :thinking:), kind of like you know when you see an imbedded YouTube video. I might be wrong on how this is done technically but try to peg down what could be useful with what makes it technically possible. Does this sound close @happybeing?

Also what you mentioned allows people to use any competitor to JAMS or just a slightly different service because it’s their data, which is cool too :sunglasses:

1 Like

I think both are possible - details of what to do in each case will be clearer in time.

I think services are quite a limited but clean, so easy to create or take advantage of. You’ll see what I mean when I document my stuff I hope. There are comments in the services Web API code right now. An English explanation would help a lot, but I think anybody that wanted to could read the comments and implement their own service now, also by looking at what I’ve done to implement LDP (even though it’s unfinished).

@happybeing

I like your enthusiasm and I see what you’re saying.

I’m not sure Playlists make sense for a service, at least not as JAMS currently has them implemented. It’s literally two parts, extremely trivial: a name, a list of context-less XORs.

The reason it is a “playlist” is because JAMS know the list of XORs refer to songs, and then it looks up each XOR in the Library to find metadata about the song: artist, song title, etc.

The Library metadata might actually be a much better example of the service you’re talking about. Given this XOR, what the heck song should I be expecting? Check the JAMS DB. It’d be a bit like CDDB, but based on XOR rather than music fingerprints.

@Nigel would like the user to be able to modify their song metadata and I can understand why. Perhaps the user wants to change the title to something shorter or more meaningful to that individual. That would imply each user has a local, mutable copy of the JAMS DB with their songs in it which they can override.

That would also imply that maintaining the JAMS DB is not communal (otherwise people would fight over a song title and have wiki wars); CDDB was not communal either. I’m not sure who has the onus of maintaining the DB, adding new entries, etc. I’m not sure what happens when the JAMS DB maintainer gets hit by a bus. I suppose someone else has to download as much of the JAMS DB metadata as possible and then upload their own copy so they can become the maintainer of a fork?

EDIT: After proof reading this, I wonder if @bochaco might have been suggesting this JAMS DB would exist as public MD URLs; perhaps that’s how the JAMS DB service would exist sans API.

2 Likes

And perhaps it could! The cost of updating the public JAMS DB would have to be paid by the JAMS apps developer rewards that are dolled out by the network, based off app usage/popularity, so the burden doesn’t fall on the user. This is what would basically enable a freemium model of Spotify on SAFE.

I see legal issues possibly brewing here though? It’s definitly something to consider. If we don’t do it then someone else will and maybe that would be for the best if it is a legal issue. If anyone wants to convince me otherwise I’m all in.

Yes enthusiasm trumps understanding and knowledge! (particularly of JAMS) :smile:

I’m not sure it’s really a good example either, but glad it’s helped explain my thoughts about services, so any responses to that - questions, issues, alternative etc are welcome, possibly in the services topic topic so not to derail talk about JAMS.

2 Likes

@Nigel The legal issues would be for hosting the ImmutableData songs. That’s effectively distribution of copyrighted material (which would be a problem the SAFE Team has to deal with … and should definitely be thinking about now before they become the next Napster or Pirate Bay). Hosting the metadata about this or that ImmutableData already on the network? Should be fine.That’s all the JAMS DB would do: return data in the JAMS DB telling you the artist, title, album, etc for ID songs already on the network.

1 Like

Good point. Whoever maintains the JAMS DB will have to pay to keep it up to date, so there does need to be some model that makes it efficient to maintain. I hadn’t considered that when discussing problems of community-maintained metadata DB, but it’s a great addition to the list of pro/con list.

Pro: single maintainer is economically efficient than smaller TXs by community contributors; Con: single maintainer has larger burden than TXs spread across community contributors.

I would basically think of it like this. A user uploads their music library and pays to put unique ImD’s and MD’s to the network, they can be prompted to voluntarily contribute to the JAMS DB then the JAMS app gets all relevant xor’s and meta data and stores publicly per @bochaco’s earlier suggestion. JAMS network developer rewards would pay for storing its own version of the users data. Assuming it all balances out cost wise.

The pros here are:
•users who upload to their own library will have to pay PUTS on those MD’s before being able to contribute to the JAMS DB so spamming it becomes a burden to them (and us if they do it anyways)
•JAMS DB has its own copy to sort through, correct, etc
•it’s voluntary for the user to participate
•there are songs added to an open source freemium streaming service that has music of all genres and cultures to choose from, that can scale *if things are balanced correctly.

Cons:
•have to think :thinking:

1 Like

I’d would like to only add, as a gral. comment, that I’m particularly looking at the aspect of separating what’s the app’s own data to properly function from what’s the user’s data being stored using the app, i.e. the music is the user’s data being stored, but the pointers to the music and how to structure them are just so they can be used by the app, which can differ from app to app.
Thus if the design makes sure that the user’s data is stored in locations that can be shared with other apps (they’ll need to be compatible at a certain degree) the user is never tight to an app (I’m sure we’ve all suffered from this). Moreover, I see the account’s containers as places to have these pointers to the data owned by the user rather than the place to store the data itself, just like the index of the user’s data on the network, whilst the data is on the network on MD’s and ID’s. I’m not fully sure, but my understanding is that this is also aligned with what’s proposed by Solid, @happybeing can probably help us understand if this is correct or inaccurate.

5 Likes

On another note (sorry for inserting in the middle of the ongoing discussion):

Play history.

For one, history is just always good with these sort of things. But most importantly: we can choose from any available recommendation algo, and feed our history to it.
Spotify’s Discover etc. used to be one of the best ways for me to find new music that I actually liked. Something is odd with the algos now though (or I am becoming increasingly hard to please…) because I think the recommendations are really bad. (I would like to pause the algo sometimes to try listen to weird stuff it suggests, so that it doesn’t reinforce that bad feed with more of it… It’s sticky!)
So the point is: we want to enable any vendor and algo to give recommendations based on our (and others’ !) play history.

3 Likes

Why does this ring a bell…?..I remember when I was a kid/teenager and the radio stations and music TV channels ranking meant something to me…until I started to learn how the music business works :disappointed:

2 Likes

Am I wrong in my assumption that if the MD is public the original owner of that MD can mutate it but affect what others would see if it was shared?

I was just thinking last night if the app kept handling the data as it does, it’s like the app doesn’t own it but it makes it basically unusable to the user anywhere else right? So, because of previously stated reasons I think JAMS could function better as originally intended as is BUT the same way we could make the users data public for a JAMS DB streaming service, we could also provide a “Move to new service” option. Obviously the users data would remain and could always still use JAMS but their data could be put in public containers to use with other services. I think the major con is it would probably cost the user in PUTS.

There has to be other ways too. When JAMS is open sourced then any app could ask to use the _music container of a user and just handle the data the same ways as JAMS correct? Just thinking out loud here

I wonder if this is an example where we could look to abstract the data structure. ‘Indexed list’ or something.

So while that’s pretty generic… it’s a list of XOR refs for datas (in this case songs). The data itself is generic enough that any other app could parse it / use it. *

It could be that part of index lists's data is some meta data like some kind of ‘type’ that could list ‘music’ or ‘playlist’ so this could be handled by whatever app. (or perhaps tags…)

Just because the data is only significant for now for JAMS doesn’t mean it couldn’t be useful in another app/context later. I think if we consider it enough we could have user data sets that are both useful for the app and potentially more things later.


(* I imagine a website plugin that adds a playlist to the page… just feed it the XOR ref for the indexed list and you’re off. The same ‘playlist’ could work for videos too, for example (though maybe needs that data as part of the indexed list idea).

4 Likes

This could be MySpace’s big comeback! :smile: I completely agree here. Iya really helpful having so many perspectives laid out here for Bryan and I to mull over, I really appreciate it, especially from those so closely involved with the project.

1 Like

haha. Happy to have such good apps inspiring such thinking!

Really is great to see, @Nigel, @BryanB!

Keep up the good work :tada:

1 Like

Correct, in the same way as when you change a file of a safesite, everyone sees the change automatically.

I’m not sure I understand what you mean, but I think I know were you are coming from. One important thing to remember is that the account’s Default Containers (i.e. _public, _publicNames, etc.) are just private MutableData, i.e. they are not stored in a specific or special place, they are just at a random location and encrypted with the account’s/user’s credentials. What makes them special, if you wish, it’s just that you have an index created by default (in the Root Container) when the account is created so you can access them thru their names. You can find more details about the Root and Default containers in this RFC: https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md

I think that’s what the type tag is/should be for, so you can have a convention of type tags for albums, another for playlist, etc.
Also, if you think about it, you could be publishing albums using their title/name in a similar way as how the public id’s Services container are stored. Thus if your album is called "Soda Stereo" and the convention for albums is to have a type tag 15008, the album’s MD could then be stored at address sha3("Soda Stereo") with type tag 15008, then I don’t even need to give you the XOR address to share it, I just text you with the album’s name. This will only work if that address happens to be free obviously, perhaps unlikely to happen but totally possible (you could have some squatting issue here as well I imagine).

3 Likes

This is an interesting thread and very useful in trying to figure out the best design patterns to use with the SAFE API and storage model.

A couple of thoughts:

This is tricky. One thought is where individuals share things the app creates a ‘salt’ for each user, and when sharing with trusted friends this ‘salt’ could be shared. So users of hash(title) the xor address to find things I share with you would be hash(title) ^ happybeing's salt

Clunky, the salts would be stored by your app, so if @bochaco texts me Hey happybeing, check out ‘Soda Stereo’ I paste ‘Soda Stereo’ into JAMS and when it comes up with not found it prompts Check friend uploads? and when I choose @bochaco from the list it can find the file by using his ‘salt’.

Ugly, so maybe not a great idea. Into thought number two…

To handle the problem of Gabriel sharing a playlist with me or anything else with editable data (music library index) there are two options, and apps can do either A or B, or offer this as a configuration setting, or prompt etc

A) Store a pointer to the Mutable Data. Pro: simple, cheap (1 PUT) Concensus: when Gabriel changes his MD I might lose stuff I wanted to keep (eg a particular music file) or worse, be deceived into downloading something harmful (malware, porn, an advert for Coka Cola etc)

B) Store a Copy of The Mutable Data, including a pointer back to the original). Pro: I have both the original and my own copy which I can edit Con: it could cost more (depends on the size of the index).

C) Store a Copy of the Mutable Data in an Immutable Data, and add a mutable data if I ever want an edited version (which could just store changes). Pro: maximum functionality - never lose the original, can see/track Gabriel’s changes, can have my own changes on top of the original or in top of Gabriel’s latest. Con: more complexity, cost depends because using Immutable Data might be cheaper than B) until you start editing at least.

5 Likes

Those options you described @happybeing is exactly what I had in my head, just give the option to the user receiving a shared album/playlist, because sometimes you are sharing an album with someone and he is not really interested in storing a copy/clone of it but just listen to it once, the same for those using a jamstube safesite :stuck_out_tongue: , the user could be just drinking some beer while listening to random music that was shared by people and friends, perhaps when the user likes a song/album/playlist (by clicking a like button/icon) at that moment the app can make a copy/clone for the user (option A or B).
I agree this is a nice topic with interesting thoughts!

5 Likes

Hmmm. I’m not sure the type tag is necessarily enough… Or I worry things could get too granular there (and we end up maintaining an index of type tags to find all relevant to your cause…)

A playlist could be music or video… or audiobooks… (or a specific subset of those).

A playlist type tag doesn’t necessarily help an app distinguish if it’s what it should be consuming. So then would we have a type tag for each? Or is an ‘indexed list’ the type tag (and so would be applicable to search indexes as well) and then some ‘tag’ setup on the data which could be used in terms of metadata searching.

4 Likes