Synergy - Project SOLID and SAFEnetwork

solid
linkeddata
relatedprojects

#21

See proposal of related ideas in topic by @DavidMtl.

And how these relate to Solid + SAFE:


#22

Hey @happybeing!

I’ve fiiiiiiiiinally been getting to some more serious SOLID reading, and digging into your services lib a bit (nice work so far!). I have a few Qs I was hoping you might be able to clarify:

  • The aim for the lib is to hijack fetch and would be the REST API for SOLID, in order to map those to SAFE functionality (MDs)? Or are more things at work?

  • You’re using rdf.js to manage the triples and perform SPARQL queries on returned data?

  • The advantage of rdf.js is the above exactly? Simpler queries of linked data right? Or is rdf.js more focussed on generating REST style calls?

Let me know if I’m on/near/wayoff there, please :slight_smile:


#23

That’s pretty much it. Glad you are looking at this Josh as I would value any feedback, observations, ideas, advice etc

To confirm/clarify:

  • rdflib.js is forked and modified very slightly so I can intercept its fetch()
  • a Solid app expects to use HTTP to talk to a RESTful (Solid) server
  • a Solid app can use fetch(), or the rdflib.js Fetcher (which adds functionality)
  • if it uses other means (e.g XHmtlRequest) they must first be converted to fetch()
  • the Solid/SAFE Plume demo uses both window.fetch() and rdflib.js Fetcher
  • safenetwork-webapi:
    • implements LDP (a subset of Solid server)
    • lacks WebID auth or access control lists
    • intercepts any window.fetch() or Fetcher and handles any safe: URLs
    • is written with more than Solid/LDP in mind. My idea is that additional services could be supported by just adding a ServiceInterface class (file sharing, remoteStorage.io etc)

rdflib.js is extensive and I don’t know it very well. The Fetcher handles different data formats, and returns a ‘graph’ which can then be queried directly. Also, I think the Fetcher may be able to crawl multiple resources where one resource refers to another, including on other domains - a bit like a scraper where you set the depth. I’m not sure about that, so take with a pinch of salt, but if so how cool is that!

The SAFE Plume demo uses Fetcher, but doesn’t do anything complex, so I think it could just as well only use fetch() but I may be underestimating it! :slight_smile:


#24

Yeh nice. Sounds similar to how I was imagining search indexes to function (time and time ago when I was trying to whip that up back when Structured Data was a thing :smiley: ).


@happybeing, it’s a great idea for smoothing out things for SOLID devs to open up adoption etc :+1:

Only thing that pops up is that right now you’re working in one JS file. For end use probably it should be split up, such that you need only include the LDP portion and wont need the rest of the code services etc that are irrelevant for you.


Mostly for me it leads into questions about how we should be storing the data on SAFE, and if/how we can treat LDP as something first class perhaps… ? So in the end we could aim to be building apps directly on the network using this setup for data.

How are you storing/writing data on the network at the moment?
An array of objects on the MD?

{
    //... rest of MD stuff
    RDF : [ //... more triples
        {
            s : 'SAFE'
            p : 'is'
            o : 'rad'
        },
        {
            s : 'happybeing'
            p : 'made'
            o : 'this'
        }
    ]
}

Is there any more info that needs to be stored for Linked Data?


Some other thoughts:

  • I wonder how LD triples lend themselves to site indexing (although probably there’d be a need for more detailed info than RDF triples provides?)

  • What would be the expected / potential structures of LD for a website or a user profile? (is that standardised anywhere?)

  • Could we query the network directly for triples? Is that even possible? Is that even desirable?


#25

Thanks for the feedback and support Josh, I’m glad you’re interested in this and it would be great to have as much input from you as you want to / can give :slight_smile:

You ask important questions and ideas which are food for thought. Some answers & comment…

The library is one file just for convenience atm but the code is structured so it can be split up easily, new services added, implementable replaced etc, even by non coders. So yes, that’s to be done.

I’ve also just fleshed my thoughts on how to standardise this for SAFE, by using RESTful file based services as you may have seen:

The LDP and Solid standards use a file / document metaphor, which are resources in RDF terminology. So this is how the storage is implemented, and the transactions boil down to CRUD operations on containers and files, but using RDF (usually Turtle for any payload and response body). Turtle is essentially a text format for Subject/Predicate/Object triples (e.g. Josh/works for/Maidsafe etc - BTW that’s not Turtle, just a hint about what a triple is :wink:).

Linked Data is generally implemented in a Solid or LDP server as directories of files where each file represents an RDF resource that can be addressed with its own URI. Responses are therfore often parsed into a graph which is then queried and manipulated.

SPARQL queries return the triples which might be a subset of one resource or drawn from multiple resources, even spread across different servers.

So my implementation stores each RDF resource as a file (using SAFE NFS). Content negotiation (not yet implemented) allows the requester to receive results in different forms, depending on the representation that is stored (Turtle, N3, JSON-LD etc) and whatever conversions the server supports from each stored representation. In addition, rdflib.js Fetcher has several conversions built in too - so client side.

Note that a resource can be in any format, not just RDF, so plain text, .png, .mp3 etc, but using RDF leverages the semantic Web, SPARQL etc and if you upload a Word document to Tim’s Solid server, a kitten dies.

I’ll be showing the RESTful LDP interface at work, and the Turtle it returns for container listings, and that is used store blog posts in my demo. So if you want to see some really simple LD maybe take a look at that - it’s all there in the console output if you visit plumetest atm).

I don’t think there’s any limit to what triples can represent, though what is the best implementation is a different matter. Some use file system back ends, some graph databases etc.

For leveraging the existing semantic Web (the data out there, applications, tools, libraries and skills), and to create an ecosystem where SAFE apps can share, reuse and reach across a sea of data, it’s the protocols Solid, LDP and the standards RDF/LinkedData, WebID etc that we should support. And things like the RESTful principles and the [Architecture of the World Wide Web] (https://www.w3.org/TR/webarch/). Sitting in on the chats between Tim and his colleagues has taught me a lot about these things, and the value of building with them.

HTML has various ways of integrating with LinkedData and there is a standard way for users to publish a profile as part of a Web identity (a WebID URI), which is something a user owns and controls.

Now you’re thinking! Lots of ways to pay with this and I haven’t really got the basics yet. Step by step I plod. I look forward to working with folks to figure things like this out.


#26

Latest: video demo of a Solid app running on the SAFE Network and slides from a presentation explaining the benefits of building Solid apps on SAFE.

I spent a few months getting this far, but can’t take this much further on my own so if you like these ideas please consider getting involved - I have lots of ideas for things to do! :slight_smile:


Resource Description Framework implementations and 'Portable Data...': Some Links and learnings
#27

How do we move this forward?

Although far from understanding the concepts of the semantic web, the most obvious piece is that of resources being linked and accessed via URI which brings us back again to @bochaco’s to be able to access data on the network as a SAFE URI via XOR name, like safe://4291165faf85f9964d4c8f5d12e4b0dc31ffda4509d2049704c77a018f922c98 (a hexadecimal slice of a 32 byte XOR name).

Additionally, it’s necessary for a URI to contain the format and metadata of the data which it represents, which https://multiformats.io can help provide inspiration.


#28

I think it will be useful both to have a mechanism such as the one you refer to here, and the ability for RDF resources to be accessed by the browser as if they are a Web service. The former will allow linking directly to Immutable Data, and more general forms of file sharing without using a public ID, so will be very valuable. The latter may prove useful in itself, because a short readable URI is a better UI, and can be updated for example, and is available now.

In the DevCon demo you can see me doing the second kind, but I hacked it by making an LDP based on top a SAFE www device, rather than a separate new LDP SAFE service type - which is necessary if we want to generalise this. I made a proposal for how to do that here:

Where files are stored using NFS we can use the file extension to imply the ContentType (which is how a typical Solid server works), but if the SAFE NFS API is extended to allow this to be set as metadata, we could use that instead.

I’m not sure how important that is for ContentType because using the file extension works well enough, but I think it’s still going to be useful to allow NFS file metadata to be set, and I believe this is already planned.

At the same time I’d like to add the option to have MD entries for containers so they can also have metadata, and can also be explicitly created and deleted, because this improves compatibility with LDP and therefore Solid.

Thanks for your input and please keep digging in, it’s great to see. I am due to chat with @Viv and @Krishna about going forward so tagging them with your interest.


#29

This is an interesting site for looking at LD sets http://lod-cloud.net/


#30

Linked Data Templates / LinkedDataHub

A couple of links here relevant to the Semantic Web:


#31

SPARQL queries in the client! One to bookmark :slight_smile:

Comunica

Comunica is a highly modular and flexible meta query engine for the web

Overview and details:

https://comunica.github.io/comunica/

Direct link to Executing a SPARQL Query.


#32

I second this! This would be huge for media!