Synergy - Project SOLID and SAFEnetwork

Always happy to assist with testing once itā€™s reached a stage where a non-dev could provide useful input.

2 Likes

Iā€™m a web developer. Can I be a part of this project?

1 Like

That would be great. Iā€™m currently working on getting a Solid app to work with SAFE.

Not much to show yet, but watch my forks of solid-plume and rdflib.js for starters.

What aspects would you like to help with, JavaScript, HTML/CSS, design etc?

4 Likes

Thank you! I am a coder, so HTML, CSS, and JavaScript for sure. I can translate mock-ups into web pages but I am not much of an artist, or designer.

I will study those repos and get myself up to speed ASAP.

2 Likes

Found this nice overview of Solid and the projects built around it ( in this case mainly crosscloud )

http://crosscloud.org/2016/www-mansour-pdf.pdf

4 Likes

UPDATE: I met Sir Tim Berners-Lee yesterday. We happened to be in the same area so he invited me to lunch so we could meet offline rather than just online as we have been recently - where he and others from the Solid community have been helping me get up to speed with their project and code.

Tim is the man who invented the Web and has been working on it for 25 years (remind you of anyone? :wink:), culminating in Project Solid (read/write Web, semantic data/documents with built in rules, forms etc)

It was fun, amazing for me to meet the man responsible for the birth of the Web, and to get to know him a bit personally. We talked about Solid and SAFE, but mostly about other things.

It was very helpful for me, to hear first hand what he regards as the stand out features of LinkedData. Iā€™ve read about it but had missed some key features. Some points about LinkedData/RDF that I noted:

  • itā€™s a graph not a tree (unlike XML). I knew that!
  • can include rules (for transformation, validation?). Not that!
  • can include forms (so a client can create a form for the data in a document). Or this!
  • you can subscribe to documents (but not containers, as yet)
  • although you can represent RDF in XML and Json but I think going that route you tend to lose some of the features of LinkedData
  • adoption of LinkedData and using it to include semantics in documents is inhibited by the extra work involved, but schema.org has helped a lot (due to Googleā€™s influence and the benefits in Internet search)

These features make all sorts of applications possible, including multiple views and styles of presentation of the data, as exemplified in the Tabulator (see end).

Iā€™m continuing with work on a PoC demo to show that Solid apps can work with SAFE and will post updates here on that, also as I learn more about Solid. I have only scratched the surface, but itā€™s more and more impressive the more I learn about it, just like SAFEnetwork! :slight_smile:

[ASIDE: If you want to dig deeper into LinkedData, there is a paper about the Tabulator (see github README.md) which I think is a good place to look.]

22 Likes

Very interesting, he knew the SAFE project from before?, can you share some thoughts from Tim about SAFE and this possible (or maybe not) integration between SAFE and SOLID?

We are looking at this from many angles in house. Mark is spearheading the intro and hopefully integration partnership or similar. It gives us a great angle for the client API, if we were able to marry these two projects with their client / linked data API and our back end network it could be a big winner. Itā€™s early days to say the least, but Mark has had the intro meeting, but it is an intro and not a project yet.

There will be more meetings to come, we will share what we can, but as always folk donā€™t like meetings being too open when they are not aware of it. so we will give Tim his privacy and respect that until we have something more solid ( :wink: ) to share.

Hope you understand.

15 Likes

I understand the interest but itā€™s not OK for me to share othersā€™ private comments, not that we went into that - just that if we had I would not publicise it unless I knew it was OK.

I will though keep posting updates of what Iā€™m up to! :slight_smile:

6 Likes

Do you have some links describing more of how this works?

I made some prototype a few years ago of adding functions (interfaces actually) to linked data. I didnā€™t really find any examples of anything similar at the time.

My idea was basically to add the whole program to the graph. For a client side javascript application this essentially boils down to adding an interface to every javascript function and store it to the graph. Javascript in itself doesnā€™t support interfaces, so the interfaces instead those interfaces are only in the linked data graph. An example is a javascript function that takes a username and email address and output a user object. In the graph Iā€™d then have something like a registerUser interface that has a link from the username entity and the email entity to a user entity. Another example is a javscript function that takes some datatype and transforms it to html. Once you have this you can manipulate the program with queries and do queries to find all existing transformations functions of datatypes you have in the graph. In theory that could make it possible to make a very powerful IDE, where a lot development could be done by querying existing functions for manipulating the data types you want to use in your application, find existing html views for the data types to quickly prototype etc.

Iā€™m wondering if they are working on something similar.

2 Likes

I donā€™t have more on LD rules yet, but can ask. Or by all means ask in the LinkedData gitter yourself.

I got the impression that a lot of what you describe above is already possible with LD.

I havenā€™t though read the stuff Iā€™ve linked to thoroughly myself so would like to do that now I realise thereā€™s more to it than I had appreciated. If you turn anything up please summarise or link on this topic and Iā€™ll do the same.

1 Like

@nice thanks for those links about crosscloud. I hadnā€™t seen this and I think it could be very helpful. :slight_smile:

1 Like

So I did a bit of Googling and found a recent project called the ā€œFunction Ontologyā€ that can be used to define functions in a linked data graph, this seems to be similar to what I made a simple version of in my prototype.

They have some examples of publishing and discovering javascript functions in this paper.

This could be a great thing to have with SAFE. You could have lots of JavaScript or Rust snippets stored as immutable data and use the Function Ontology to describe what the functions do and make them queryable with rdflib from Solid.

Combine this with a QA app similar to Stack Overflow, but where you were required to use the function ontology to markup your functions, you can have the basis for making natural language queries. Today in programming, poeple often use Google and then click on stack overflow, but have to manually take the code from there, but with functions marked up with the function ontology this process could to some extent be automated. You could write a natural language query, search the index from the QA app and then get a list of potential functions back. Especially in a more limited domain like SAFE apps this might work. People who publish answers to the app could also include things like a license for the function.

3 Likes

Tim pointed me at this. Not dug into it yet but seems to be a useful set of core applications or app features, and is part of mashlib:

Good technical overview of Solid/Crosscloud platform (client/server, user/application developer, features and example applications). Best intro yet IMO:

  • A Demonstration of the Solid Platform for Social Web Applications (PDF)

Notes to self:

  • LDP PATCH is required according to this, whereas I thought it was an optional server feature and can in any case be simulated in the client (within limitations of the client and connection).
  • LDP SPARQL and link following are optional server features, but clearly powerful, and so we might consider how SAFE API can provide for their implementation - either client side, or perhaps by the network in the future.
  • W3C Social Web Working Group user stories looks worth following up
  • Access Control Lists and WebID authentication are key parts of Solid platform and it would be worthwhile looking at how SAFE API could support similar or equivalent functionality in a compatible way (for example, with fine gained control over access of individual resources through an LDP compatible API).

For SAFE API support of Solid features, much can probably be achieved using a solid-safenetwork client application library, which could later be moved into the main SAFE app APIs, but certain features may require changes to SAFE network client libraries if they are to be supported.

To begin with, it would be very useful to know what the current SAFE network and APIs can and cannot accommodate, and how this would limit or block Solid app features when used with SAFEnetwork.

6 Likes

See proposal of related ideas in topic by @DavidMtl.

And how these relate to Solid + SAFE:

5 Likes

Hey @happybeing!

Iā€™ve fiiiiiiiiinally been getting to some more serious SOLID reading, and digging into your services lib a bit (nice work so far!). I have a few Qs I was hoping you might be able to clarify:

  • The aim for the lib is to hijack fetch and would be the REST API for SOLID, in order to map those to SAFE functionality (MDs)? Or are more things at work?

  • Youā€™re using rdf.js to manage the triples and perform SPARQL queries on returned data?

  • The advantage of rdf.js is the above exactly? Simpler queries of linked data right? Or is rdf.js more focussed on generating REST style calls?

Let me know if Iā€™m on/near/wayoff there, please :slight_smile:

1 Like

Thatā€™s pretty much it. Glad you are looking at this Josh as I would value any feedback, observations, ideas, advice etc

To confirm/clarify:

  • rdflib.js is forked and modified very slightly so I can intercept its fetch()
  • a Solid app expects to use HTTP to talk to a RESTful (Solid) server
  • a Solid app can use fetch(), or the rdflib.js Fetcher (which adds functionality)
  • if it uses other means (e.g XHmtlRequest) they must first be converted to fetch()
  • the Solid/SAFE Plume demo uses both window.fetch() and rdflib.js Fetcher
  • safenetwork-webapi:
    • implements LDP (a subset of Solid server)
    • lacks WebID auth or access control lists
    • intercepts any window.fetch() or Fetcher and handles any safe: URLs
    • is written with more than Solid/LDP in mind. My idea is that additional services could be supported by just adding a ServiceInterface class (file sharing, remoteStorage.io etc)

rdflib.js is extensive and I donā€™t know it very well. The Fetcher handles different data formats, and returns a ā€˜graphā€™ which can then be queried directly. Also, I think the Fetcher may be able to crawl multiple resources where one resource refers to another, including on other domains - a bit like a scraper where you set the depth. Iā€™m not sure about that, so take with a pinch of salt, but if so how cool is that!

The SAFE Plume demo uses Fetcher, but doesnā€™t do anything complex, so I think it could just as well only use fetch() but I may be underestimating it! :slight_smile:

3 Likes

Yeh nice. Sounds similar to how I was imagining search indexes to function (time and time ago when I was trying to whip that up back when Structured Data was a thing :smiley: ).


@happybeing, itā€™s a great idea for smoothing out things for SOLID devs to open up adoption etc :+1:

Only thing that pops up is that right now youā€™re working in one JS file. For end use probably it should be split up, such that you need only include the LDP portion and wont need the rest of the code services etc that are irrelevant for you.


Mostly for me it leads into questions about how we should be storing the data on SAFE, and if/how we can treat LDP as something first class perhapsā€¦ ? So in the end we could aim to be building apps directly on the network using this setup for data.

How are you storing/writing data on the network at the moment?
An array of objects on the MD?

{
    //... rest of MD stuff
    RDF : [ //... more triples
        {
            s : 'SAFE'
            p : 'is'
            o : 'rad'
        },
        {
            s : 'happybeing'
            p : 'made'
            o : 'this'
        }
    ]
}

Is there any more info that needs to be stored for Linked Data?


Some other thoughts:

  • I wonder how LD triples lend themselves to site indexing (although probably thereā€™d be a need for more detailed info than RDF triples provides?)

  • What would be the expected / potential structures of LD for a website or a user profile? (is that standardised anywhere?)

  • Could we query the network directly for triples? Is that even possible? Is that even desirable?

1 Like

Thanks for the feedback and support Josh, Iā€™m glad youā€™re interested in this and it would be great to have as much input from you as you want to / can give :slight_smile:

You ask important questions and ideas which are food for thought. Some answers & commentā€¦

The library is one file just for convenience atm but the code is structured so it can be split up easily, new services added, implementable replaced etc, even by non coders. So yes, thatā€™s to be done.

Iā€™ve also just fleshed my thoughts on how to standardise this for SAFE, by using RESTful file based services as you may have seen:

The LDP and Solid standards use a file / document metaphor, which are resources in RDF terminology. So this is how the storage is implemented, and the transactions boil down to CRUD operations on containers and files, but using RDF (usually Turtle for any payload and response body). Turtle is essentially a text format for Subject/Predicate/Object triples (e.g. Josh/works for/Maidsafe etc - BTW thatā€™s not Turtle, just a hint about what a triple is :wink:).

Linked Data is generally implemented in a Solid or LDP server as directories of files where each file represents an RDF resource that can be addressed with its own URI. Responses are therfore often parsed into a graph which is then queried and manipulated.

SPARQL queries return the triples which might be a subset of one resource or drawn from multiple resources, even spread across different servers.

So my implementation stores each RDF resource as a file (using SAFE NFS). Content negotiation (not yet implemented) allows the requester to receive results in different forms, depending on the representation that is stored (Turtle, N3, JSON-LD etc) and whatever conversions the server supports from each stored representation. In addition, rdflib.js Fetcher has several conversions built in too - so client side.

Note that a resource can be in any format, not just RDF, so plain text, .png, .mp3 etc, but using RDF leverages the semantic Web, SPARQL etc and if you upload a Word document to Timā€™s Solid server, a kitten dies.

Iā€™ll be showing the RESTful LDP interface at work, and the Turtle it returns for container listings, and that is used store blog posts in my demo. So if you want to see some really simple LD maybe take a look at that - itā€™s all there in the console output if you visit plumetest atm).

I donā€™t think thereā€™s any limit to what triples can represent, though what is the best implementation is a different matter. Some use file system back ends, some graph databases etc.

For leveraging the existing semantic Web (the data out there, applications, tools, libraries and skills), and to create an ecosystem where SAFE apps can share, reuse and reach across a sea of data, itā€™s the protocols Solid, LDP and the standards RDF/LinkedData, WebID etc that we should support. And things like the RESTful principles and the [Architecture of the World Wide Web] (Architecture of the World Wide Web, Volume One). Sitting in on the chats between Tim and his colleagues has taught me a lot about these things, and the value of building with them.

HTML has various ways of integrating with LinkedData and there is a standard way for users to publish a profile as part of a Web identity (a WebID URI), which is something a user owns and controls.

Now youā€™re thinking! Lots of ways to pay with this and I havenā€™t really got the basics yet. Step by step I plod. I look forward to working with folks to figure things like this out.

2 Likes