SAFE Browser Fetch API best practice

Aye that’s a fair point. In rust it makes sense in that ‘webby’ kind of way, but it could equally be get or something.

I’m also not super against a fetch wrapper, but if it’s coming in the browser it should probably overwrite the native fetch automatically.

I guess it’s just not something I see a lot of mileage in if the actual API is built properly. Right now the use case is only compatabilty, but that to me, means a custom wrapper shim-thing is the way to go.

If we’re missing things from the API itself, that’s another Q. (Are we missing things for a webfetch-shim? I’m not sure of all the things the fetch interface provides… or are we only missing safe: handling? [which is actually a permission issue if memory serves… so perhaps we’re just missing enabling that…])

1 Like

I’m not sure either. I think browser fetch has enough complexity we may well be missing something so it needs checking out. I’m still learning - only realised you can consume response.body as a whatwg stream this month!

2 Likes

@joshuef

I won’t be able to contribute an opinion until I see the whole picture after having a simple web app running.

One potential problem I foresee is web framework compatibility with SAFE if existing web API isn’t supported. For example, I suspect that the fetch service of Yew will not work by default. If correct, it won’t be as seamless of an on-boarding experience for developers to SAFE.

2 Likes

@joshuef

Logging r reveals

{PublishedImmutableData: {…}}

Are facilities available for determining the data type so that (r.headers.get('Content-Type') != 'application/wasm') can be satisfied? I see that PublishedImmutableData contains a media_type key with a null value.

Does SAFE Browser support both WebAssembly.instantiate and WebAssembly.instantiateStreaming?

@joshuef

If you do not expect much usage of fetch, what API should be used in JS to retrieve a WASM file from SAFE for running the web app?

This seems like a safe-api issue in that wasm isn’t/wasn’t detected at upload (or isn’t supported atm). I’ll make an issue to check into that.

You’re getting a return, much like the cat command, which is also using fetch, so you can access the raw data in the data field of your PublishedImmutableData. That should be get you going as things stand.

We havent actively crippled that functionality, so such things should be available… Though I’m not familiar with what they do under the hood so can’t say for sure that they will work. The browser is built on electron, which is chromium. So it should support all things as chrome does, aside from the notable aspect of blocking http requests (for security).

That said, looking at the name, I’d assume streaming won’t necessarily work at the mo if that’s needing HTTP headers, for data ranges… I’m not sure.


This is kind of the bigger topic we touch on above. Should our API simulate a HTTP request to this end? (In rust or browser or wherever). And if so, why?

It’s not so much that we don’t expect much use of fetch, these APIs are new, non finalized, and yeh, there’s this bigger Q of backwards compatability and if/where that should live in the stack…

Compatibilty with Yew/fetch in general may be a nice thing to have, sure. But from an functionality standpoint, running a server that maps requests to SAFE would enable yew to work against that server (from what i understand, if we’re just fetching things from servers here), that doesnt sound like core api functionality.

That something doesn’t work off the bat, isn’t a reason to force compatibility with an old protocol, potentially increasing the security surface and the like.

Yew there is working with HTTP requests, so yes, that’s not compatible. But depending on what Yew actually needs in terms of data, the ideal (IMO) would be for Yew itself to support safe: urls, and not need to go through any HTTP spoofing.

What do we need for that? Well, a good API really. So it comes down to a question of what’s missing. And then we get that in.

Right now you’re blazing a bit of a trail (i don’t think even @happybeing has diven into these APIs yet), so there’s almost certainly going to be improvements wanted.

To be clear though: none of this isn’t to say that we don’t need a fetch api, in some form. Again, a wrapper around the browser may be useful in the short term, or a wee server for pinging requests onto safe may be useful. But should those be core API functionalities? (And what sort of priority should we give them?)


At the moment, I think you can use safe.fetch as you are. And grab the data portion of your PublishedImmutableData, this should be a string of the data (or perhaps just the raw bytes if the type is unknown right now; not sure off the top of my head).

From your code above, this is what (i think you need for feeding in to the WebAssembly.instantiate func).

2 Likes

@joshuef

Per the bufferSource parameter documentation at WebAssembly.instantiate() - WebAssembly | MDN

A typed array or ArrayBuffer containing the binary code of the .wasm module you want to compile.

How can PublishedImmutableData.data be converted into a compatible type? The error

Uncaught (in promise) TypeError: WebAssembly.instantiate(): Argument 0 must be a buffer source

is generated when attempting to directly pass data.

result = WebAssembly.instantiate(safe_data.PublishedImmutableData.data, imports);

FWIW I’m using WebAssembly.instantiate() - WebAssembly | MDN because while WebAssembly.instantiateStreaming() - WebAssembly | MDN is recommended, it requires a Response object and as I understand it safe.fetch() doesn’t provide that.

The error

hellosafe.js Uncaught (in promise) CompileError: WebAssembly.instantiate(): expected magic word 00 61 73 6d, found 00 00 00 00 @+0

is generated by

const APP_ID = "safe.users.hellosafe";
const APP_NAME = "Hello SAFE";
const APP_VENDOR = "users";

let safe = new window.Safe();

let auth_credentials = safe.auth_app(APP_ID, APP_NAME, APP_VENDOR);

safe.connect("net.maidsafe.safe-nodejs", auth_credentials);

const fetchPromise = (url) => {
    return new Promise((resolve, reject) => {
        try {
            const result = safe.fetch(url);
            resolve(result)
        } catch (error) {
            reject(error)
        }
    });
}

let safe_data = safe.fetch(module);

if (safe_data.PublishedImmutableData) {
    let typedarray = Int32Array.from(safe_data.PublishedImmutableData.data);

    console.log(arrayBufferToHex(typedarray));

    result = WebAssembly.instantiate(typedarray, imports);
}

yet using https://github.com/LinusU/array-buffer-to-hex/blob/master/index.js with typedarray outputs

0061736d...

as expected.

I think this should do it.


@happybeing, did you ever get fetch to work natively with safe: APIs? I remember you were experimenting and needed safe: marked as ‘secure’ in electron. Though I’m not sure of the outcome of that…

I’m wondering how easy/simple it would be to get fetch going in browser for normal response type APIs.

We already have a server doing this for requests from tabs. So perhaps makes sense to expose this as fetch (assuming that’s possible; if the above technique didn’t pan out).


@bochaco I’m not sure how you feel about a ‘web’ style end point in general in the api? (returning http responses)? It may be necessary short term…

I haven’t been back to try that. I guess there’s an issue open still explaining what the next step is assuming we’re using an election version that now supports it.

1 Like

Okay grand. In that case I’ll try and get a look at this the next week. It may end up being a simple fix that’s quite helpful :+1:

1 Like

@joshuef

Buffer is not available in browser environments AFAIK.

1 Like

Yup.

I’ll get an alpha build fired up which has fetch going :+1:

1 Like

@joshuef

https://github.com/maidsafe/safe_browser/releases/tag/v0.16.0-alpha.2 doesn’t have a macOS build available.

@joshuef

I’ve built the release and with the Fetch API enabled the JS and WASM files load and run as expected.

Thank you for providing a solution.

2 Likes

Those builds are up now @anon78698497: https://github.com/maidsafe/safe_browser/releases/tag/v0.16.0-alpha.2

Although good to see you’re already flying with that :metal:

1 Like

I’m sorry I just see this. I guess if that’s something we can expose from the browser should be good enough, not sure if we want it in the nodejs api anymore though, if it’s for compatibility with other frameworks i think they’d still need some adaptation as they won’t like safe: protocol in URLs anyway?? I don’t have like a strong opinion but if we can avoid it I’d vote for that, maintaining an http format is not something that we should probably go for, but again, i’m not strong on any opinion, just depends on cost/benefit.

I was meaning specifically from safe-api, for non browser things (yew in the above example) that may want it.

If the API provides a compatible fetch it would be a shame not to expose it in the browser. It would be very useful there and make migration and adoption easier, and allow people to build compatibility libraries that make putting existing code much easier.

@happybeing, right now there is no rust based http api, that’s what I’m pondering here. If that’s worth it in the core api layer or no.

FWIW, i’m leaning to the same conclusion as @bochaco above. It’s not really core safe api functionality.


On another note, the browser can now fetch safe: urls via the standard browser fetch. So that probably makes your life a lot easier for some of the work you were doing previous :+1: (and it was via the extra electron perm you were asking about a while back. So: nice one!)

1 Like