SAFE Drive - help with testing

Problem Existed Between Keyboard And Chair :slight_smile:

should have known I couldnt wite to ~/SAFE
public

3 Likes

Hey there folks, I am willing to test the Windows version, whenever it becomes ready.

2 Likes

Iā€™m going through the tutorial that it says I should complete before moving on with the instructions on this page and am running into an issue there.

Thought it would make sense to create a topic for it instead of posting just here to keep this thread on topic. Here is that topic: No Authenticator pop-up in node.js "SAFE Desktop App Tutorial" tutorial

1 Like

Iā€™ve come a bit further now, but am running into an error I cannot figure out.

The error comes when I try to do npm install in the safenetwork-fuse folder. So Iā€™m at " b) Clone safenetwork-fuse:" on this page: https://github.com/theWebalyst/safenetwork-fuse/

this is what my PowerShell says:

PS E:\Project\program\javascript\safenetwork-fuse> $env:NODE_ENV = "dev"
PS E:\Project\program\javascript\safenetwork-fuse> npm install

> fuse-bindings@2.11.2 install E:\Project\program\javascript\safenetwork-fuse\node_modules\fuse-bindings
> node-gyp-build


E:\Project\program\javascript\safenetwork-fuse\node_modules\fuse-bindings>if not defined npm_config_node_gyp (node "D:\Program\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "D:\Program\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild )
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
  fuse-bindings.cc
  abstractions.cc
  win_delay_load_hook.cc
e:\project\program\javascript\safenetwork-fuse\node_modules\fuse-bindings\fuse-bindings.cc(12): fatal error C1083: Cann
ot open include file: 'fuse.h': No such file or directory [E:\Project\program\javascript\safenetwork-fuse\node_modules\
fuse-bindings\build\fuse_bindings.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (D:\Program\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:262:23)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "D:\\Program\\nodejs\\node.exe" "D:\\Program\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd E:\Project\program\javascript\safenetwork-fuse\node_modules\fuse-bindings
gyp ERR! node -v v8.12.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! fuse-bindings@2.11.2 install: `node-gyp-build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the fuse-bindings@2.11.2 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Lukas\AppData\Roaming\npm-cache\_logs\2018-09-28T09_04_04_419Z-debug.log
PS E:\Project\program\javascript\safenetwork-fuse>

This looks like an important part of the error message to me:

e:\project\program\javascript\safenetwork-fuse\node_modules\fuse-bindings\fuse-bindings.cc(12): fatal error C1083: Cannot open include file: 'fuse.h': No such file or directory [E:\Project\program\javascript\safenetwork-fuse\node_modules\fuse-bindings\build\fuse_bindings.vcxproj]

That file does indeed not exist, specifically: ..\safenetwork-fuse\node_modules\fuse_bindings\ does not exist. However, I donā€™t know if that is as a result of some previous step not succeeding or if that is the actual failure point.

Here is the error log it gave me, in case thatā€™s useful: https://paste.sh/dDZ7u4Ma#YvO5WQuyxdF7to3ja-QpqX_-

Hmm, fuse-bindings is present in the package.json dependencies, but try again after installing it manually:

npm install fuse-bindings

@bzee is on Windows and has done this, so maybe he can see what the issue is?

1 Like

Yes, I had this when I hadnā€™t installed FUSE (GitHub - mafintosh/fuse-bindings: Notice: We published the successor module to this here https://github.com/fuse-friends/fuse-native). Which you can install using Dokany on Windows.

During the installation of Dokany I think you have to select installing the dev headers.

2 Likes

OK @lukas, try that. It crossed my mind - if this is the issue you are doing a great job debugging the README.md already :slight_smile:

Maybe we should add a heading above ā€œGet the Sourceā€ called ā€œInstall the Dependenciesā€ which links back to the earlier instructions? Or something like that.

Good stuff, and thanks @bzee

@lukas please make a note of anything like this:

During the installation of Dokany I think you have to select installing the dev headers.

1 Like

Great, thanks!

Iā€™ll take another look, hopefully within a few days! :slight_smile:

Linking back up to the dependency instructions sounds like a good idea.

1 Like

I ended up looking at this today and got a bit further once more, thanks for the help!

The problem I have now is with running the app after building it.

I have these files here:

    Directory:
    E:\Project\program\javascript\safenetwork-fuse\dist\mock\node_modules\@maidsafe\safe-node-app\src\native\mock


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       2018-09-28     16:10       26540463 safe_app.dll
-a----       2018-09-28     16:10        5795840 system_uri.dll

And I run pkg . -t host -o ./dist/mock/mount-safe in the safenetwork-fuse directory to create the .exe file.

The problem is that the resulting exe doesnā€™t want to run. I get the following errors in PowerShell:

PS E:\Project\program\javascript\safenetwork-fuse> .\dist\mock\mount-safe.exe
pkg/prelude/bootstrap.js:1176
      throw error;
      ^

Error: No native build was found for runtime=node abi=57 platform=win32 arch=x64
    at Function.load.path (E:\snapshot\safenetwork-fuse\node_modules\node-gyp-build\index.js:31:9)
    at load (E:\snapshot\safenetwork-fuse\node_modules\node-gyp-build\index.js:13:23)
    at Object.<anonymous> (E:\snapshot\safenetwork-fuse\node_modules\fuse-bindings\index.js:1:99)
    at Module._compile (pkg/prelude/bootstrap.js:1252:22)
    at Object.Module._extensions..js (module.js:661:10)
    at Module.load (module.js:563:32)
    at tryModuleLoad (module.js:503:12)
    at Function.Module._load (module.js:495:3)
    at Module.require (module.js:594:17)
    at Module.require (pkg/prelude/bootstrap.js:1157:31)

This happens even though I have it set to ā€˜hostā€™ as the target, so I donā€™t know whatā€™s going on.

Any ideas?

This is great @lukas. If you look in package.json at the build commands you will see some of them copy some files next to the executable. See this part:

mkdir -p ./dist/prod/node_modules/@maidsafe/safe-node-app/src/native/prod/ && cp -R node_modules/@maidsafe/safe-node-app/src/native/prod/*.* ./dist/prod/node_modules/@maidsafe/safe-node-app/src/native/prod/

You will I think need to do that too, but unfortunately I donā€™t think those commands work on Windows. One of the things Iā€™d like is to convert those build commands to node.js scripts so they work on all platforms, but I donā€™t know how to do that yet. Probably we can find another project that does something similar, or can find out some other way.

Butā€¦ thereā€™s another bunch of .node files which need to be copied into the same directory as the executable, and this is done manually at the moment.

I think these are:
binding.node ffi_bindings.node node-57.node weakref.node

I think the error you quote is looking for node-57.node (see this part of the error: runtime=node abi=57 platform=win32 arch=x64)

Automating this is part of issue #8, though that part is not well specified atm:

Note: some extra files are copied by the build scripts into ./dist/prod and some are still being copied manually (e.g. .node binaries), so this also needs to be automated.

You will need to grab the Windows versions of those files as they are native binaries. They will be somewhere under node_modules, but in different spots. I donā€™t have a note of where each one is so you could search for them.

I think you are close to testing the actual code which is brilliant!

2 Likes

Thank you for the help! Once Iā€™m done Iā€™ll do a pass on smoothing this path out for anyone who comes after.

Hopefully Iā€™ll have time to work in this on Sunday.

3 Likes

You are already doing that, but yes, keep notes and letā€™s iron out those kinks. Really good work, and enjoy your other time too of course. No pressure here, just appreciation :slight_smile:

3 Likes

I have the .dll files in the specified directory.

I found and copied weakref.node, binding.node, ffi_bindings.node and node-57.node to the same directory as the .exe file.

However I still have the same error message. I think the problem is that the node-57.node file is not built for Windows.

I tried building it manually with prebuild --target 57 --path node_modules\fuse-bindings\ and putting the file in the resulting .tar.gz in the .exe directory, but it didnā€™t work. Tried renaming it to node-57.node, but it didnā€™t work.

I donā€™t know if Iā€™m on the right track here, any ideas? :slight_smile:

EDIT BY: @happybeing: I wonder if @bzee has any ideas?

1 Like

I think the issue is not the .node files. That solution was helpful when distributing the executable to other machines to make it work ā€” donā€™t think it applies here.

This exact issue is what I encountered a month back when helping @happybeing and I did not get it to work. However, for testing purposes you could use node bin instead of trying to package it and running the packageā€¦ node bin runs fine, so the issue must be related to packaging it.

2 Likes

Itā€™s fairly easy using fs-extra:

const child_process = require('child_process');
const fs = require('fs-extra');

const path_prod = './dist/prod/node_modules/@maidsafe/safe-node-app/src/native/prod/';

// mkdir -p
fs.ensureDirSync(path_prod);

// cp -r
fs.copySync('./node_modules/@maidsafe/safe-node-app/src/native/prod/', path_prod);

// Run pkg and direct output to current script
child_process.execSync('pkg . -t host -o ./dist/prod/mount-safe', { stdio: [ 0, 1, 2 ] });

Just put in a file like build.js and node build.js will do the rest. Could also be put in package.json so npm run build will run the above script.

2 Likes

Status update

A small update for superfans of SAFE Drive. Note, as of 11th October these features have been merged into master. One notable missing feature is mkdir/rmdir, which I am working on next.

I have file create, update, delete and rename working so after mounting your SAFE Drive, from the shell you can do stuff like:

$ cd ~/SAFE
$ ls
_public _publicNames
$ ls _public
rsapps
$ ls _public/rsapps
www-root
$ cd _public/rsapps/www-root
$ ls
index.html 
$ echo 'Hello world' > test.txt
$ cat test.txt 
Hello world
$ cp test.txt test2.txt
$ ls 
index.html test.txt test2.txt
$ cat test2.txt
Hello world
$ rm test.txt
$ ls
index.html test2.txt

I recommend not opening your SAFE drive in a file manager. This will almost certainly hang, or at least behave as if it has as soon as you open a folder or two. Really it is just making an endless series of calls to get status on everything in sight which will slow everything down even on a powerful computer.

Also, it will probably fail if you try saving from an application like an editor. (See update on this at the end of this post).

Files should load into applications OK, but saving will both be slow and then fail if the app tries to save safely (using file rename, because that is not implemented yet). And of course the applicationā€™s file open dialog is a file explorer, so best to open using the command line (eg gedit ~/SAFE/_public/rsapps/www-root/index.html).

You can create and access files on your SAFE public storage, but many things will just not work.

So this is only for the very curious, or the person who wants to help find bugs. And certainly not for trusting with any valuable data!

Please let me know if you have a play. There isnā€™t a release build of this so you will have to grab the master dev-savefile branch in both safenetworkjs and safenetwork-fuse repos, and in the latter run with node run bin.js - see the repo for instructions, or ask if you need help.

Windows

@bzee has some success on Windows today, so watch this space. I think all he did was change the drive setting in bin.js from I:\ to M:\ so anyone in Windows could try this after installing the dependencies (just Dokany I think - again, see the repo for instructions).

Mac OSX

SAFE Drive is asking the browser for authorisation but not receiving the acceptance. We have though called @joshuef back from Japan to look into this :wink:, so hopeful Mac testing will also be possible soon.

Update: today I implemented rename, so you can also use ā€˜mvā€™ to rename files or move them to an existing directory. Unfortunately gedit file editor still doesnā€™t work, but vi at least is happy to edit files. Note that it does not support renaming of directories because it would be difficult to implement for little benefit, because the way SAFE NFS works, renaming directories could create problems by quickly filling up a container with deleted entries, and prevent you from adding new files.

4 Likes

I get an error when I run node bin:

PS E:\Project\program\javascript\safenetwork-fuse> node bin
module.js:682
  return process.dlopen(module, path._makeLong(filename));
                 ^

Error: The specified module could not be found.
\\?\E:\Project\program\javascript\safenetwork-fuse\node_modules\fuse-bindings\build\Release\fuse_bindings.node
    at Object.Module._extensions..node (module.js:682:18)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at load (E:\Project\program\javascript\safenetwork-fuse\node_modules\node-gyp-build\index.js:13:10)
    at Object.<anonymous> (E:\Project\program\javascript\safenetwork-fuse\node_modules\fuse-bindings\index.js:1:99)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
PS E:\Project\program\javascript\safenetwork-fuse>

Did I miss some step? I went through all steps from the beginning again but canā€™t get it to work.

1 Like

Did you perhaps cut the .node files instead of copy when you put them in the same directory as the .exe?

That file must have been generated after npm install. Try removing ./node_modules/fuse-bindings/ and run npm install again.

2 Likes

Update:

Today I implemented file rename, so you can now use ā€˜mvā€™ to rename files or move them to an existing directory. Unfortunately gedit file editor still doesnā€™t work, but vi at least is happy to edit files.

Note that it does not support renaming of directories because it would be difficult to implement for little benefit. This is because the way SAFE NFS works means that renaming directories could create problems by quickly filling up a container with deleted entries, and prevent you from adding any new files.

These changes have now been merge to master.

2 Likes

I successfully ran it with node bin. I think the issue was that I had forgotten to do npm link safenetworkjs after doing npm install :sweat_smile:. Thank you for the help!

Iā€™m not able to make it work though it seems.

Admin PowerShell

If I run it in an admin PowerShell I get this message when running it:

PS E:\Project\program\javascript\safenetwork-fuse> $env:NODE_ENV = "dev"
PS E:\Project\program\javascript\safenetwork-fuse> node bin
Server path not specified, so defaulting to ipc.config.socketRoot + ipc.config.appspace + ipc.config.id /tmp/app.6032
starting server on  /tmp/app.6032
starting TLS server false
starting server as Unix || Windows Socket

This instance should quit. Ciao!

Then when I accept the authentication in Peruse, a node.exe console window pops up that prints out the following message forever:

######
error:  { Error: connect EPERM \\.\pipe\tmp-app.9160
    at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1191:14)
  errno: 'EPERM',
  code: 'EPERM',
  syscall: 'connect',
  address: '\\\\.\\pipe\\tmp-app.9160' }
connection closed 9160 /tmp/app.9160 Infinity tries remaining of Infinity
requested connection to  9160 /tmp/app.9160
Connecting client on Unix Socket : /tmp/app.9160

Non admin PowerShell

If I run it in a non-admin PowerShell I get the following output in my PowerShell:

PS E:\Project\program\javascript\safenetwork-fuse> $env:NODE_ENV = "dev"
PS E:\Project\program\javascript\safenetwork-fuse> node bin
Server path not specified, so defaulting to ipc.config.socketRoot + ipc.config.appspace + ipc.config.id /tmp/app.2948
starting server on  /tmp/app.2948
starting TLS server false
starting server as Unix || Windows Socket

This instance should quit. Ciao!

Then when I accept the authentication in Peruse, it flashes what looks like the node.exe console window and prints out this in the PowerShell:

## socket connection to server detected ##
received event of :  auth-uri { id: '2948-cli',
  message: 'safe-c2fmzw5ldhdvcmstznvzzq:AQAAAMQlnhEAAAAAAAAAACAAAAAAAAAAhC7wEqSTZyC05pDPGHyY-i6-5hZOiAxeRHcOiQPAEXkgAAAAAAAAAKsm-zHuZlZXSS5aKe3wCZqL64dNyAFMhByXm6ZrWdn3IAAAAAAAAACAKkrtcXzv6JF7FAujWoiXpTj9ncmsC4Dvn8K2VZYL4kAAAAAAAAAANoZ-fcwbTAPOpn5BgiFZOTEfuq65r_DTFVj3-uRPGdCAKkrtcXzv6JF7FAujWoiXpTj9ncmsC4Dvn8K2VZYL4iAAAAAAAAAAr7f2K2jms5Pq-88-PAIWfvp9P1SChKkn0kGX4_rZMnUgAAAAAAAAAIopW2v7ctf_IcOOxmzacoEcXQSKFVfWWBinbsXAyo-9AAAAAAAAAAAAAAAAAAAAAAuRF5vh3DD-SEIl36YY8EQdRkgQgwrecIJ-XI7QF23_mDoAAAAAAAAYAAAAAAAAAPkqIEUwHWLDw3MrqS5f9XmUaegqSCGnrgIAAAAAAAAADAAAAAAAAABfcHVibGljTmFtZXNLqcoJ3JfrFgmRlhiuvTyl12z08o9asVMsGAVp_k62sZg6AAAAAAAAASAAAAAAAAAA6OXDor9jkx0OPFT7QJ4IAXSC38Q24GhniqRddA_2S2cYAAAAAAAAAFgBuukcPrsHWQe_e5wbsYQtVZMZf0ixcgAEAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAHAAAAAAAAAF9wdWJsaWP3Mg2wMuUZ_dJwkjPcUwsggqMwM9d2-U-5EAWHlz0CF5g6AAAAAAAAAAAEAAAAAAAAAAAAAAABAAAAAgAAAAMAAAA' }
socket disconnected 2948-cli

And a Node.js process lies taking up a bunch of CPU and memory in the Task Manager until I close the PowerShell window. Ctrl+C doesnā€™t do anything in the PowerShell to stop the process:
image


Iā€™ve tried changing the drive letter in the bin.js file, as I saw somewhere that that made it work for someone, but no difference.

Iā€™m on really deep water now in terms of knowing what Iā€™m doing, so I hope Iā€™m not being too much of an annoyance with my need of help with getting this to work!

1 Like