SAFE Browser Fetch API best practice

anon78698497 · January 24, 2020, 12:47am

What is the recommended solution to the error

Fetch API cannot load safe://hnyynysg796jaj1dui3ge37diqbmzr9z4ykaojerf3mkanoj6hjc6w3g44bnc/hellosafe.wasm. URL scheme "safe" is not supported.

and where in the docs could I have found the answer?

joshuef · January 24, 2020, 8:07am

You are using the native browser fetch API?

You’ll want to use a safe app’s fetch method for this. We don’t overwrite the native fetch functionality as yet.

anon78698497 · January 24, 2020, 3:01pm

@joshuef

Where in the docs can I learn about that?

happybeing · January 24, 2020, 4:08pm

Not overly helpful sorry, but try https://github.com/maidsafe/safe-api/search?q=fetch&unscoped_q=fetch

anon78698497 · January 24, 2020, 5:32pm

@happybeing

Those results are Rust whereas what I seek is the JavaScript API for use in SAFE Browser.

bochaco · January 24, 2020, 7:10pm

Try with this: https://safenetforum.org/t/safe-network-dev-update-january-23-2020/30937/23 , then safe.fetch("safe://...");

anon78698497 · January 24, 2020, 9:56pm

@bochaco

Using

let safe = new window.Safe();
const response = safe.fetch(module);

I’m able to progress to a new error

hellosafe.js Uncaught (in promise) Error: internal error in Neon module: Failed to fetch content from '/hellosafe.wasm': ContentNotFound("Content not found at /hellosafe.wasm")

FWIW the error is the same when removing the leading “/” from the path.

Per the docs, changing the address of the resource being fetched to its XOR-URL solved the error.

Whereas fetch apparently only accepts XOR-URL address references, DOM element src attributes accept both XOR-URL and path address references. Is that correct and the intended design?

The new error is

Uncaught (in promise) Error: internal error in Neon module: Failed to fetch content from 'safe://hbyyyynitex9ewwyg6anjmufsxrob9dg9ygq9ym37bfg3x6oqegegkmue6': ConnectionError("Application is not connected to the network")

Adapting code from the example, what would you suggest in lieu of specifically line 19

github.com

bochaco/electron-neon-safe/blob/master/renderer.js#L19


const APP_VENDOR = "MaidSafe.net Ltd";


const { Safe, XorUrlEncoder, SafeDataType, SafeContentType } = require('safe-nodejs');


let safe = new Safe();


console.log("Authorising application...");
let auth_credentials = safe.auth_app(APP_ID, APP_NAME, APP_VENDOR);


console.log("Connecting to the Network...");
safe.connect("net.maidsafe.safe-nodejs", auth_credentials);


// XorUrlEncoder
let random = Math.floor(Math.random() * Math.floor(1000));
let nrs_url = `safe://mypubname-${random}`;
console.log("Let's parse from NRS URL:", nrs_url);
let xorname = Uint8Array.from([1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2]);
let xorurl_encoder = new XorUrlEncoder(xorname.buffer, 10000, SafeDataType.PublishedSeqAppendOnlyData, SafeContentType.FilesContainer, "/folder", ["sub1", "sub2"], 5);
console.log("XOR-URL generated:", xorurl_encoder);
console.log("Encoding version:", xorurl_encoder.encoding_version());
console.log("Xorname:", xorurl_encoder.xorname());

as that causes SAFE Browser to hang. (when not immediately crashing due to a separate bug)

anon78698497 · January 24, 2020, 10:47pm

Correction, it’s eventually timing out

Uncaught (in promise) Error: internal error in Neon module: Failed to authorise application: AuthdClientError("Failed to read response: connection closed: timed out")

Allowing the request in the CLI with auth allow $RequestId solves the error.

The new error

hellosafe.js Uncaught (in promise) TypeError: response.then is not a function

is generated from within the code block of the fetch. If you would like to see the code I can include it here or FWIW I’m adapting GitHub - yewstack/yew-wasm-pack-minimal: A minimal template for starting a Yew project using wasm-bindgen and wasm-pack and the fetch is in bundle.js.

bochaco · January 25, 2020, 12:30am

If the response is what you assign the result of safe.fetch to, then the error (response.then is not a function) seems to be that some logic of that bundle is expecting it to be a Promise but it’s not, none of the safe-nodejs APIs exposed in the browser return a Promise yet (they eventually will).

The safe.fetch is to retrieve content/files from the SAFE network so you need their XOR-URL or NRS-URL.

anon78698497 · January 25, 2020, 1:14am

Being new to SAFE and having not worked with JavaScript for years I would appreciate any help solving the error

hellosafe.js Uncaught (in promise) TypeError: response.then is not a function

caused by the line .then(r => {

in the fetch code

   if ((typeof URL === 'function' && module instanceof URL) || typeof module === 'string' || (typeof Request === 'function' && module instanceof Request)) {
        const APP_ID = "safe.users.hellosafe";
        const APP_NAME = "Hello SAFE";
        const APP_VENDOR = "users";

        let safe = new window.Safe();

        let auth_credentials = safe.auth_app(APP_ID, APP_NAME, APP_VENDOR);

        safe.connect("net.maidsafe.safe-nodejs", auth_credentials);

        const response = safe.fetch(module);

        if (typeof WebAssembly.instantiateStreaming === 'function') {
            result = WebAssembly.instantiateStreaming(response, imports)
                .catch(e => {
                    return response
                        .then(r => {
                            if (r.headers.get('Content-Type') != 'application/wasm') {
                                console.warn("`WebAssembly.instantiateStreaming` failed because your server does not serve wasm with `application/wasm` MIME type. Falling back to `WebAssembly.instantiate` which is slower. Original error:\n", e);
                                return r.arrayBuffer();
                            } else {
                                throw e;
                            }
                        })
                        .then(bytes => WebAssembly.instantiate(bytes, imports));
                });
        } else {
            result = response
                .then(r => r.arrayBuffer())
                .then(bytes => WebAssembly.instantiate(bytes, imports));
        }
    } else {

        result = WebAssembly.instantiate(module, imports)
            .then(result => {
                if (result instanceof WebAssembly.Instance) {
                    return { instance: result, module };
                } else {
                    return result;
                }
            });
    }

joshuef · January 27, 2020, 5:37pm

You’ll want to make a wrapper function for safe.fetch that’ll return a promise.

This should do it (untested, but should give you the idea):


const fetchPromise = ( url ) => {
// return a browser native Promise, which has`.then()` functionality
    return new Promise( ( resolve, reject ) => {
        try{
            const result = safe.fetch( url );
            resolve( result )
        }catch( error )
        {
            reject( error )
        }
    } );
}

// you can then use this func inplace of your `const reponse = safe.fetch...`
const response = fetchPromise(module);

Hope that helps!

anon78698497 · January 27, 2020, 8:00pm

@joshuef

.fetch(url); generates the error

Uncaught (in promise) Error: internal error in Neon module: Failed to fetch content from 'safe://hbyyyynitex9ewwyg6anjmufsxrob9dg9ygq9ym37bfg3x6oqegegkmue6': NetDataError("Failed to GET Published ImmutableData: Error(CoreError(Symmetric decryption error - CoreError::SelfEncryption -> Decryption(\"BlockModeError\")))")

bochaco · January 27, 2020, 8:13pm

That’s likely to you are now trying to fetch it with latest CLI/Browser but you uploaded that file with previous version of CLI/Browser? assuming to a local vault?

anon78698497 · January 27, 2020, 8:18pm

@bochaco

Yes latest CLI and Browser. I ran ./safe files put ~/tmp/hellosafe/dist/ --recursive after the updates and used the returned FilesContainer XOR-URL however it’s possible there’s some latent state. Yes local vault.

bochaco · January 27, 2020, 8:22pm

There were some breaking changes in latest release of safe_client_libs, which causes old data most of the time incompatible with new lib.
I’d suggest you clean up your local vault storage, restart the vault, and upload the data again. You should be able to clean your local vault’s data by removing this folder completely: ~/.local/share/safe_vault

anon78698497 · January 27, 2020, 9:10pm

@bochaco

No change.

~/.local/share/safe_vault doesn’t exist on my system.

I stopped the vault and CLI, removed ~/Library/ApplicationSupport/net.MaidSafe.safe_vault/, started the vault, started the CLI, auth create-acc --test-coins; auth login --self-auth, uploaded the files, started SAFE Browser.

What else should I do?

bochaco · January 27, 2020, 9:21pm

Ah, I’m sorry, yes, I think that’s the correct path for Mac
Well, if you uploaded the files again, you should be able to fetch them without those issues you were seeing before with fetch.

You can also try with the file I uploaded before to the shared vault: safe://hbhybydrr6htazoz7e7iho6nn9rooshfbiayo6bsm6xzjnksoktf1h6qii, and see if you get that error, you should be able to fetch it with latest CLI/Browser:

$ safe cat 'safe://hbhybydrr6htazoz7e7iho6nn9rooshfbiayo6bsm6xzjnksoktf1h6qii'`
<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8" />
    <title>Hello SAFE</title>
    <script src="/artifacts/bundle.js" defer></script>
</head>
<body>
</body>
</html>

anon78698497 · January 27, 2020, 10:40pm

@bochaco

After removing all SAFE related preference and data dirs and reinitializing, the error is resolved.

The new error generated by .get('Content-Type') != 'application/wasm') { is

hellosafe.js Uncaught (in promise) TypeError: Cannot read property 'get' of undefined

joshuef · January 28, 2020, 8:08am

Mhmmm. Seems like it’s expecting the full return of a fetch interface (with response headers). Currently our safe.fetch does not work this way.

It perhaps could… though should it? For simplicity in browser it may be desirable for a Fetch compatible api, but what at the rust api level? At the mo we’re returning thus: https://github.com/maidsafe/safe-api/blob/master/safe-api/src/api/fetch.rs#L96

I kind of think wrapping that for the browser is a separate concern. I know @happybeing has worked on a js lib previous for overriding the browser’s fetch function, which may be a useful guide.

I’m not 100% sold that the browser should be doing this by default. Why simulate HTTP requests when that’s fundamentally not what’s going on (and there exist direct APIs for retrieving things for the network…)

@anon78698497 its looking like your issue above is that this code is depending on the browser fetch API, w/ .get and r.arrayBuffer. In the end to run WebAssembly.instantiate(bytes, imports)); You should be able to get the same result from the SafeData returned from safe.fetch (https://github.com/maidsafe/safe-nodejs/blob/master/native/src/lib.rs#L332).

happybeing · January 28, 2020, 12:25pm

I don’t think it’s a big deal. The benefit is that fetch is a standard, and handles lots of use cases (streams for example) in already well understood competent ways. And that existing design patterns, code, and libraries will sit easily on top of it.

But somebody else could make a small module to implement fetch inside SAFE browser on top of the existing API as you note. I guess doing it as part of the SAFE API might help ensure that the underlying APIs can be made to support equivalent functionality and don’t leave too much work or overhead for people wanting a fetch interface.

Somebody must have thought providing fetch was a good idea not long ago, because it is a relatively recent addition.

One thing though, if it isn’t to be like the standard fetch, I think it should though be renamed to avoid leading people into Dev traps.