SAFE Drive - OSX


#29

Thanks Josh. I’m still using v0.8.1 SAFE API so might that be the cause?

The auth IPC code is in safenetworkjs/src/bootstrap.js

I can’t claim any credit/blame :wink: it all belongs to @bzee. BTW the auth code works on Linux, and as of last night Windows.


#30

Okay cool. I’ve been triggering the pid/uri command manually with some nonsense to test (/Users/josh/.nvm/versions/node/v8.11.2/bin/node /Users/josh/Projects/safe/forks/safenetwork-fuse/bin --pid 1323131 --uri safe-auth:asdasdad and i’m seeing the following error:


######
error:  { Error: connect ENOENT /tmp/app.1323131
    at Object._errnoException (util.js:992:11)
    at _exceptionWithHostPort (util.js:1014:20)
    at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1186:14)
  code: 'ENOENT',
  errno: 'ENOENT',
  syscall: 'connect',
  address: '/tmp/app.1323131' }
connection closed 1323131 /tmp/app.1323131 Infinity tries remaining of Infinity

which looks like the ipc server isn’t reachable :frowning:

I don’t have time to dig more just now but I’ll fire in again the morning :+1:


#31

That’s to be expected when using nonsense parameters. :wink: The pid will determine which IPC ‘server’ (or rather socket) to send to. ENOENT means the socket file does not exist.

Still, the node-ipc module might be the culprit here, though it should support OSX without any trouble.


#32

fair point @bzee :smiley:

using the pid given by the process yields nothing (beyond some logs i added). So not really sure what could be going on there.

I’ll dig a bit deeper the morn (on japan time now, so soon to bed). If you, @bzee or @happybeing have any more ideas of what to scope out let me know.


#33

It’s an interesting exercise to think about possible issues without debugging! :slight_smile:

I assume the /tmp/app.<pid> file is created and the IPC server is effectively listening. Then the problem must be with the client, so I think you’re looking in the right direction @joshuef.

Perhaps try delaying the spawned process to see the actual output of the IPC client (not exiting immediately here: https://github.com/theWebalyst/safenetworkjs/blob/dev-savefile/src/bootstrap.js#L86). On Windows I usually see a glimpse of the Node script starting in a CMD and then disappearing — do you see that too on OSX? I’m curious to know whether the scripts spawns and if so what the output is.


#34

Haha, yeh. Tell me about it. That’s my windows debugging life. (aka appveyor hell :stuck_out_tongue: )


Anyway. Wee update: I can confirm after some deeper digging, that it’s not node-ipc or the sciprt itself. I got calls to the script to be picked up after logging the auth uri from the browser. (Which crashed the fuse script but… that’s another story. It did start doing things. :tada: )

So it seems there’s an issue with how the app is registering itself with the system. On osx, at least with fully fledged apps, to open them up via registered URI, a bundle id is needed. I’m not sure of the case for this w/r/t scripts/IPC etc. But it’s likely something around this. So I’ll be continuing my exploration form here tomorrow!


#35

Coming back to this.

I think this is quite an interesting problem, we’ll be needing to tackle for scripts in general.

OSX needs an Info.plist for an application bundle to register custom URL handlers (touched on over here: https://stackoverflow.com/questions/18534591/how-to-register-a-url-protocol-handler-in-node-js)

While with FUSE the final packaged app could have this… The bigger Q is ‘do we want scripts to be able to auth automagically’ (A: yes!). *

So how could we do that?

I think that we might be needing to pass another option at app instantiation… perhaps responseScript or some such (which would still just pass along the auth response URI instead of needing a custom protocol to be registered…)? @nbaksalyar @marcin not sure what yous would make of this (nor really if there’s some security implication vs requiring a custom protocol to be registered?).

Other options would be providing a UI to get a response from the authenticator (which could then be requested by the app, or set via ENV or some such). Less ideal UX imo, but another option…


  • For now a bonus log is on master for anyone wanting to get scripts going, with a yarn prod-dev from the safe_browser repo, you’ll be able to see/pull out the response url and manually trigger the script. Not ideal, but a workaround for now…

#36

I’m not sure what ‘bonus log’ means, but I think it might be best to await the solution rather than tweak SAFE Drive for this workaround, as to work it needs a custom build of the browser (if I understand correctly).

Instead, for SAFE Drive we could look at bundling an Info.plist if that is as straightforward as it sounds. @Stout77, would you like to have a crack at this, starting with the link Josh provided and asking for help here as needed?

Thanks for looking at this @joshuef. Sounds like we can make progress now, and work with any broader solution once that has been designed.


#37

I’ll be away for work a couple of days but I’ll give it a go later in the week/weekend; looks like I’ll have to do a bit of studying for this…


#38

Heh, by ‘bonus log’, I just meant another console.log which outputs the uri that would be triggering auth.

For sure it’s not something to rely on, but if you were wanting to progress onto other OSX issues, this would unblock you.


Bundling info.plist as I understand requires an app bundle output as part of the packaging process. I don’t think you can simply add an info.plist and it all goes, sadly :frowning:.


#39

No hurry, just good you fancy looking into it so update us / ask for help as and when.

I haven’t looked and will leave it to you look at the link and consider this… ask Josh for clarification on this etc:

Bundling info.plist as I understand requires an app bundle output as part of the packaging process. I don’t think you can simply add an info.plist and it all goes, sadly

Then we can discuss ways to move this forward, and if you are the person doing so you get to choose :slight_smile:

Thanks again @joshuef I expect we’ll be back for more!


#40

Considering we don’t have an app bundle, I wouldn’t know how to proceed with the info.plist option.
So I went ahead and compiled safebrowser as per https://github.com/maidsafe/safe_browser

steps:

  • git clone https://github.com/maidsafe/safe_browser.git
  • cd safe_browser
  • NODE_ENV=dev yarn
  • yarn rebuild

And then

  • yarn put-live-net-files-for-osx
  • yarn prod-dev

this launches the safe browser with additional infos in the console;
then is a separate terminal, in safenetwork-fuse/ :

DEBUG=safe-fuse*,safenetworkjs* node bin.js

app seems to authenticate (as before), output in the console is:


[15:04:09.264] Parsing safe uri safe-auth:AAAAADe8XF0AAAAAEAAAAAAAAABzYWZlbmV0d29yay1mdXNlABEAAAAAAAAAU0FGRSBOZXR3b3JrIEZVU0ULAAAAAAAAAHRoZVdlYmFseXN0AAIAAAAAAAAADAAAAAAAAABfcHVibGljTmFtZXMEAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAHAAAAAAAAAF9wdWJsaWMEAAAAAAAAAAAAAAABAAAAAgAAAAMAAAA
renderer.js:34 [15:04:09.264] Handling safe-auth: url
renderer.js:34 [15:04:09.298] Authenticator.js decoding request safe-auth:AAAAADe8XF0AAAAAEAAAAAAAAABzYWZlbmV0d29yay1mdXNlABEAAAAAAAAAU0FGRSBOZXR3b3JrIEZVU0ULAAAAAAAAAHRoZVdlYmFseXN0AAIAAAAAAAAADAAAAAAAAABfcHVibGljTmFtZXMEAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAHAAAAAAAAAF9wdWJsaWMEAAAAAAAAAAAAAAABAAAAAgAAAAMAAAA
renderer.js:34 [15:04:09.298] Authenticator.js decoded authReq result:  { authReq: 
   { app: 
      { id: 'safenetwork-fuse',
        name: 'SAFE Network FUSE',
        scope: null,
        vendor: 'theWebalyst' },
     app_container: false,
     containers: [ [Object], [Object] ],
     containers_cap: 2,
     containers_len: 2 },
  reqId: 1566358583 }
renderer.js:34 [15:04:10.833] IPC.js: another response being parsed.: { error: null,
  res: 
   { authReq: 
      { app: [Object],
        app_container: false,
        containers: [Array],
        containers_cap: 2,
        containers_len: 2 },
     isAuthorised: true,
     reqId: 1566358583 },
  type: 'DESKTOP',
  uri: 'safe-auth:AAAAADe8XF0AAAAAEAAAAAAAAABzYWZlbmV0d29yay1mdXNlABEAAAAAAAAAU0FGRSBOZXR3b3JrIEZVU0ULAAAAAAAAAHRoZVdlYmFseXN0AAIAAAAAAAAADAAAAAAAAABfcHVibGljTmFtZXMEAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAHAAAAAAAAAF9wdWJsaWMEAAAAAAAAAAAAAAABAAAAAgAAAAMAAAA' }
renderer.js:34 [15:04:10.838] Listener for addAuthNotification
renderer.js:34 [15:04:18.769] Listener for addAuthNotification
renderer.js:34 [15:04:18.770] success happeninng
renderer.js:34 [15:04:18.770] IPC.js: Sending auth response true { authReq: 
   { app: 
      { id: 'safenetwork-fuse',
        name: 'SAFE Network FUSE',
        scope: null,
        vendor: 'theWebalyst' },
     app_container: false,
     containers: [ [Object], [Object] ],
     containers_cap: 2,
     containers_len: 2 },
  isAuthorised: true,
  reqId: 1566358583 }
renderer.js:34 [15:04:18.771] IPC.js: onAuthDecision running... { authReq: 
   { app: 
      { id: 'safenetwork-fuse',
        name: 'SAFE Network FUSE',
        scope: null,
        vendor: 'theWebalyst' },
     app_container: false,
     containers: [ [Object], [Object] ],
     containers_cap: 2,
     containers_len: 2 },
  isAuthorised: true,
  reqId: 1566358583 } true
renderer.js:34 [15:04:18.772] authenticator.js: encoding auth response { authReq: 
   { app: 
      { id: 'safenetwork-fuse',
        name: 'SAFE Network FUSE',
        scope: null,
        vendor: 'theWebalyst' },
     app_container: false,
     containers: [ [Object], [Object] ],
     containers_cap: 2,
     containers_len: 2 },
  isAuthorised: true,
  reqId: 1566358583 } true
renderer.js:34 [15:04:21.755] authenticator.js: auth decision CB { 'ref.buffer': <Buffer@0x7f9e5e668140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00> } true
renderer.js:34 [15:04:21.755] IPC.js: Successfully encoded auth response. Here is the res: safe-c2fmzw5ldhdvcmstznvzzq:AQAAADe8XF0AAAAAAAAAACAAAAAAAAAAvNGL8EPumG_8-fh7N71kEFDBhFgU1K7VZEkTpbjfR4cgAAAAAAAAAPqUsEkj_a4Q-C9QBvZ83KrVNgIOJj44jKLy6p5gQD2OIAAAAAAAAAAYkWGu-PrusS4czY1etN0t89vn-9MQebVuhLVsry0WeUAAAAAAAAAAxjhOHPA2JV3dhd7Cjmo9g0eUXm2ZPrwxnbDrhAP2OU8YkWGu-PrusS4czY1etN0t89vn-9MQebVuhLVsry0WeSAAAAAAAAAAzD3p-dWvwBOurrEIVxuJYlUZLdCcrbVbbXw_pYtLCgQgAAAAAAAAAEESegDka9V8qRj2ukqTcCIt-Q7GMVafBGrPJJ3qEKB3GQAAAAAAAAAQAAAAAAAAADE3OC42Mi43Ni44OjU0ODMTAAAAAAAAADEzOC42OC4xODUuMjE4OjU0ODMSAAAAAAAAADEzOC42OC4xODEuNTc6NTQ4MxIAAAAAAAAAMTM4LjY4LjE4MS42MDo1NDgzEgAAAAAAAAAxMzguNjguMTgxLjg2OjU0ODMSAAAAAAAAADEzOC42OC4xODEuODc6NTQ4MxMAAAAAAAAAMTM4LjY4LjE4MS4xNjg6NTQ4MxMAAAAAAAAAMTM4LjY4LjE4MS4xNzY6NTQ4MxMAAAAAAAAAMTM4LjY4LjE4MS4xNzk6NTQ4MxMAAAAAAAAAMTM4LjY4LjE4MS4xODA6NTQ4MxMAAAAAAAAAMTM4LjY4LjE4MS4xODI6NTQ4MxMAAAAAAAAAMTM4LjY4LjE4MS4yNDI6NTQ4MxMAAAAAAAAAMTM4LjY4LjE4MS4yNDM6NTQ4MxMAAAAAAAAAMTM4LjY4LjE4MS4yNDk6NTQ4MxIAAAAAAAAAMTM4LjY4LjE4OS4xNDo1NDgzEgAAAAAAAAAxMzguNjguMTg5LjE1OjU0ODMSAAAAAAAAADEzOC42OC4xODkuMTc6NTQ4MxIAAAAAAAAAMTM4LjY4LjE4OS4xODo1NDgzEgAAAAAAAAAxMzguNjguMTg5LjE5OjU0ODMSAAAAAAAAADEzOC42OC4xODkuMzE6NTQ4MxIAAAAAAAAAMTM4LjY4LjE4OS4zNDo1NDgzEgAAAAAAAAAxMzguNjguMTg5LjM2OjU0ODMSAAAAAAAAADEzOC42OC4xODkuMzg6NTQ4MxIAAAAAAAAAMTM4LjY4LjE4OS4zOTo1NDgzEQAAAAAAAAA0Ni4xMDEuNS4xNzk6NTQ4MwFrFQAAAAAAAQcAAAAAAAAAYWxwaGFfMgCz8NjeMbEKmTWtb8uLJoeMYfyBnrNhUN_ABYWoeItdupg6AAAAAAAAGAAAAAAAAAD0pVob39N3GBb7iJiV6BOFUpD0LONqdpoCAAAAAAAAAAcAAAAAAAAAX3B1YmxpY9hglYN4Hj7lqZ8-kWWDmq1urzbkvOsoRBkWkYYvzsZUmDoAAAAAAAAAAAQAAAAAAAAAAAAAAAEAAAACAAAAAwAAAAwAAAAAAAAAX3B1YmxpY05hbWVzfOTSfZgtcNqrLLOybLdgmBAFIZ8Xe8n90GHHzYNwIYeYOgAAAAAAAAEgAAAAAAAAAPWq9dsM3oiXyOTbYonFEpD124PiDO19tcd3lWzkPVbGGAAAAAAAAADipHarejesdDDlVHyWhbgQna6vdNtYuIwABAAAAAAAAAAAAAAAAQAAAAIAAAADAAAA
renderer.js:34 [15:04:23.332] Remote Calling:  setAppListUpdateListener
renderer.js:34 [15:04:23.334] Handling remote call in extension { id: '0.3oh7xq7nxee',
  isListener: true,
  name: 'setAppListUpdateListener' }

@joshuef, is this what you meant? How can I manually trigger the script from here?


#41

Sorry for the delayed response @Stout77, I’ve been away the last week and a bit.

The encoded reponse is the uri on the line line starting renderer.js:34 [15:04:21.755] IPC.js: Successfully encoded auth response. .

So you need to fire up the script (once the inital server is running), with the uri and PID as a flag. This will pass siccessfully trigger the IPC receipt in the main safe-drive script and you can progress from there :+1:

eg:

node ./safenetwork-fuse/bin --uri safe-c2...<that whole uri> --pid <the PID output from the server

EDIT: See also the ‘workaround’ in the following topic [@happybeing]:


#42

Hey, thank you @joshuef !!
@happybeing, doing the above allows the script to continue, with the following output in the console:


**safenetworkjs:web** SafenetworkApi.initialise() +7m

FUSE library version: 2.9.7

nullpath_ok: 0

nopath: 0

utime_omit_ok: 0

unique: 2, opcode: INIT (26), nodeid: 0, insize: 56, pid: 81832

INIT: 7.19

flags=0xf8000008

max_readahead=0x00100000

INIT: 7.19

flags=0x00000010

max_readahead=0x00100000

max_write=0x02000000

max_background=0

congestion_threshold=0

unique: 2, success, outsize: 40

unique: 2, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 0

statfs /

**safe-fuse:vfs:index** Fuse.mount() at /Users/Home/SAFE +0ms

**safe-fuse:bin** Mounted SAFE filesystem on /Users/Home/SAFE +7m

**safe-fuse:ops** statfs('/') +0ms

**safe-fuse:vfs:index** getHandler(/) +3ms

**safe-fuse:vfs:root** getHandlerFor(/) - safePath: /, mountPath: / +0ms

**safe-fuse:vfs:root** RootHandler for / mounted at / statfs('/') +0ms

(node:81801) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'gt' of undefined

at safeVfs.getHandler.statfs.then (/Users/Home/safenetwork-fuse/src/fuse-operations/statfs.js:26:35)

at &lt;anonymous&gt;

at process._tickCallback (internal/process/next_tick.js:189:7)

(node:81801) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)

(node:81801) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

closer but not there yet by the look of it…

ls ~/SAFE

returns:

ls: SAFE: Device not configured

and so does:

ls ~/SAFE/_public


#43

OK, we’re virtually there! It’s tripping over statfs(). A lot has changed recently, so do you mind pulling those changes and trying again?

Just do git pull in each of safenetwork-fuse and safenetworkjs

Assuming you’ve not edited any files that will just work.

Thanks @joshuef, this is looking good :slight_smile:


#44

OK, much longer response this time, I’ll paste the first part:

**safenetworkjs:web** SafenetworkApi.initialise() +48s

**safe-fuse:vfs:index** SafeVfs.mountContainer({ safePath: '/' }) +0ms

FUSE library version: 2.9.7

nullpath_ok: 0

nopath: 0

utime_omit_ok: 0

unique: 2, opcode: INIT (26), nodeid: 0, insize: 56, pid: 82039

INIT: 7.19

flags=0xf8000008

max_readahead=0x00100000

**safe-fuse:stub** TODO: implement fuse operation: init() +0ms

INIT: 7.19

flags=0x00000010

max_readahead=0x00100000

max_write=0x02000000

max_background=0

congestion_threshold=0

unique: 2, success, outsize: 40

unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 81

getattr /

**safe-fuse:vfs:index** Fuse.mount() at /Users/Home/SAFE +37ms

**safe-fuse:vfs:index** SafeVfs.mountContainer({ safePath: '_public' }) +0ms

**safe-fuse:vfs:index** SafeVfs.mountContainer({ safePath: '_publicNames' }) +0ms

**safe-fuse:vfs:root** RootHandler.initContainer({ safePath: '_public', safeUri: undefined }) failed to create SAFE container +0ms

**safe-fuse:vfs:root** RootHandler.initContainer({ safePath: '_publicNames', safeUri: undefined }) failed to create SAFE container +1ms

**safe-fuse:vfs:index** Error: mount rejected: failed to initialise container

**safe-fuse:vfs:index** at SafeVfs.mountContainer (/Users/Home/safenetwork-fuse/src/safe-vfs/index.js:473:15)

**safe-fuse:vfs:index** at &lt;anonymous&gt;

**safe-fuse:vfs:index** at process._tickCallback (internal/process/next_tick.js:189:7) +1ms

**safe-fuse:vfs:index** Error: mount rejected: failed to initialise container

**safe-fuse:vfs:index** at SafeVfs.mountContainer (/Users/Home/safenetwork-fuse/src/safe-vfs/index.js:473:15)

**safe-fuse:vfs:index** at &lt;anonymous&gt;

**safe-fuse:vfs:index** at process._tickCallback (internal/process/next_tick.js:189:7) +0ms

**safe-fuse:bin** Mounted SAFE filesystem on /Users/Home/SAFE +48s

**safe-fuse:ops** getattr('/') +0ms

**safe-fuse:vfs-cache** VfsCacheMap.getattr(/) +0ms

**safe-fuse:vfs:index** getHandler(/) +1ms

**safe-fuse:vfs:root** getHandlerFor(/) - containerRef: { safePath: '/', safeUri: undefined }, mountPath: / +1ms

**safe-fuse:vfs:root** RootContainer.itemAttributesResultRef('', undefined) +1ms

**safe-fuse:vfs:root** TypeError: Cannot read property 'defaultContainer' of undefined

**safe-fuse:vfs:root** at RootContainer.itemAttributesResultRef (/Users/Home/safenetwork-fuse/src/safe-vfs/root.js:476:51)

**safe-fuse:vfs:root** at VfsCacheMap.getattr (/Users/Home/safenetwork-fuse/src/safe-vfs/vfs-cache.js:246:38)

**safe-fuse:vfs:root** at &lt;anonymous&gt;

**safe-fuse:vfs:root** at process._tickCallback (internal/process/next_tick.js:189:7) +0ms

**safe-fuse:vfs-cache** TypeError: Cannot read property 'result' of undefined

**safe-fuse:vfs-cache** at VfsCacheMap.getattr (/Users/Home/safenetwork-fuse/src/safe-vfs/vfs-cache.js:249:67)

**safe-fuse:vfs-cache** at &lt;anonymous&gt;

**safe-fuse:vfs-cache** at process._tickCallback (internal/process/next_tick.js:189:7) +1ms

unique: 2, error: -2 (No such file or directory), outsize: 16

unique: 3, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 0

statfs /

**safe-fuse:ops** statfs('/') +0ms

**safe-fuse:vfs:index** getHandler(/) +4ms

**safe-fuse:vfs:root** getHandlerFor(/) - containerRef: { safePath: '/', safeUri: undefined }, mountPath: / +3ms

**safe-fuse:vfs:root** RootHandler for { safePath: '/', safeUri: undefined } mounted at / statfs('/') +0ms

**safe-fuse:ops** { bsize: 1000000,

**safe-fuse:ops** frsize: 1000000,

**safe-fuse:ops** blocks: 1000000,

**safe-fuse:ops** bfree: 1000000,

**safe-fuse:ops** bavail: 1000000,

**safe-fuse:ops** files: 1000000,

**safe-fuse:ops** ffree: 1000000,

**safe-fuse:ops** favail: 1000000,

**safe-fuse:ops** fsid: 1000000,

**safe-fuse:ops** flag: 1000000,

**safe-fuse:ops** namemax: 1000000 } +0ms

but same message following ls:

ls: SAFE: Device not configured

sorry, I wish I could handle this myself without keep posting…:sweat_smile:


#45

No worries, you’ve helped a lot getting us to this point.

It’s not obvious what’s causing the issue here though. I will add to the debug output, but obviously that’s a slow process for each step.

If you feel up to it, I can help you get the debugger running, but if you’ve not done that before that will be a big hill to climb. The advantage would be that you could set breakpoints and step through the code to see exactly severe it fails more quickly.

Don’t feel obliged, this has been a great help and we can still make progress with you posting debug output, albeit slowly!


#46

@stout77 here are some instructions to get you going with the debugger. Don’t worry if they don’t make sense at any point, it helps me to realise what will need explaining better, so please ask if you can’t figure anything out.

To start you will need Chrome browser (or Chromium which I use), and then I suggest you try the instructions at https://github.com/theWebalyst/safenetwork-fuse#development

You don’t need to build because you can just run the source code. If you stick with the live network for now, you will already have the browser, and won’t need to go through the Maidsafe tutorial.

So skip the parts about the tutorial and building for mock.

Make sure you have done all the steps under Get the Source (particularly the npm link bits) and then see the following section on the debugger: https://github.com/theWebalyst/safenetwork-fuse#debugging-with-chromechromium

If you get the debugger going, F8 will start the program running, or you can step through line by line with F10 and F11. F10 steps through the displayed code line by line, and F11 is the same, except it will step into any function calls. Shift-F11 runs until you exit the current function. Those are the most useful commands, but you will need to learn how to set breakpoints (click on the left edge of a line once to set, again to unset). A blue mark on the left shows where a breakpoint is set.

It’s probably worth searching for a “chrome debugger tutorial” once you get to this stage.

But to start with, just hit F8 and see what happens! :slight_smile:

Good luck, and thanks again.


closed #47

This topic was automatically closed after 60 days. New replies are no longer allowed.


opened #48