SAFE Drive - help with testing

Apps
133

Woah! This is awesome, I didn’t dare try it. Thanks John, OMG this is brilliant. I can’t believe it, it feels too good to be true.

Next is to retest the apps I know didn’t work before: when editing files they’ve already saved to SAFE Drive both LibreOffice Writer and gedit (Gnome editor) were failing to save changes. `vim’ was OK though. Not tonight though, tonight is for celebrating :champagne:

We can try publishing a git repo now too. I’m busy tomorrow, putting boat into dry dock in the morning, and don’t know if I’ll be able to work on board yet. Should be, but we’ll see.

I’m really chuffed John, thanks for. checking it out.

7 Likes
134

I’m also busy tomorrow but might be able to find a couple of hours in the evening. If so I’ll try those things. Let me know if there is anything else needs testing out.

4 Likes
135

Thanks John, if you can do this that would be great and by all means just play until you break it. Different apps will do things in different ways so just trying things and making some notes about what worked or not would help.

I’ll publish git repo and we’ll see if you and others can clone it, make changes, and push them to your own repo. If that works, we have the basis for a decentralised github :slight_smile:

4 Likes
136

Now this I too understand. :wink:

But thanks a bunch for your hard work ashore, sailor! :anchor: I do appreciate it!

3 Likes
137

Thanks, appreciation helps too. I have been working non stop on this, but aboard :speedboat: not ashore! :wink:

2 Likes
138

I can create and save pretty much any sort of file into ~/SAFE/_public/tests/data1 which I think was created by the test script. So png (Gimp), text files (vi nano mousepad) ods, odf, odp files (Libreoffice) all work. However If I create a new directory ~/SAFE/_public/tests/data2 I get a ‘Remote I/O Error’ whatever I try to save or copy there. Same with other pre-existing and new folders from another domain. I can’t save files into those either (although not all give the I/O error).

It seems only data1 can be written to at the moment.

Hope this helps.

2 Likes
139

Yes it helps!

Saving to ~/SAFE/_public/tests/data2 won’t work because there is no container there, and I don’t create one automatically. I’m still not sure that’s a good idea, but can’t think of an alternative because the UI is so limited.

So anything under ~/SAFE/_public/tests/data1 should work.

Also anything under an existing path where you already have uploaded a file. So if you have ~/SAFE/_public/anypublicname/www-root you should be able to save under that, but not at ~/SAFE/_public/anypublicname because the container is actually at ~/SAFE/_public/anypublicname/www-root and the directories above it don’t really exist. This is confusing and there’s no way for me to indicate this or explain this to the user, which is why it’s tricky to figure out how best to do it.

From what you said it sounds like you can’t create a file under ~/SAFE/_public/anypublicname/www-root. Is that correct, or were you trying a directory above that level?

Testing different apps and file types is great, thanks. Different sized files, different ways of doing things all help.

One thing I found was not working with LibreOffice Writer and gedit was this:

  • start app, create file, save to SAFE Drive
  • close the app and restart it
  • load the file you saved on SAFE Drive, modify it and save the changes
  • close and restart the app, reload the file and check the changes were saved.

Basically it helps if anyone is willing, to try as many different apps, or ways of copying, moving, saving, editing etc as possible. And check it has actually done what you expect, even if there is no sign of an error.

This is a lot of tedious work, so I’m not expecting anyone, especially you John, to do this, but it does help a lot.

Files are precious, and we need SAFE Drive to be rock solid or it is no use at all.

Once again John, thanks a lot for your help. I really appreciate it.

I’ve been getting tired with working on this non-stop so I took a break yesterday. It’s the first time it felt OK to step back from the code, feeling that I’ve got a good solution, and understand how it works now (it is complex in places), and how to track down bugs fairly efficiently.

You know that scene from The Matrix where suddenly Neo can read the code. Well I’m now like that with the console output from all my debug statements!

I could do with a bigger break, but there’s a lot still to do, and I know there will be bugs to discover and fix, so I will be pressing on, today or before long.

4 Likes
140

I would gladly attempt to help test the SAFEdrive, I have both a windows and (buggy) ubuntu installation, a bad connection, both a laptop and desktop and of course a good amount of persistence.

I do however lack most of the knowledge required, so I’ll attempt to compile and install it using the info on the github, in the meantime if anyone has any pointers for someone with minimal command skills, please do share them.

edit: Ok after an hour or so of fidgeting with gibberish I seem to have gotten it working, compiled from git and now see the _public and _publicNames folder in the drive, also noticed that it used a handfull of puts, but I can’t find any files in the folders, I assumed that this would show you all files, not just ones from the app, was I mistaken or did I make a mistake?

3 Likes
141

Great stuff @isntism, persistence is invaluable with software. The following instructions are all for Linux, and have been tried on Ubuntu and Debian, and to a lesser extent on other distros. If you can help with Windows and MacOS, see below.

For you and anyone who fancies a go with Linux, I recommend:

  • follow the instructions on the github repo to get the basics set up (installing dependencies, using npm link with SafenetworkJs etc), but to start at least, don’t worry about doing a build. If you don’t get on with that, explain where you got to, what you’ve tried, and ask for help!
  • get the development branch in safenetwork-fuse and safenetworkjs by doing in each of those directories:
git pull
git checkout development
git checkout .

That will ensure you have the latest code.

Then:

  • start Peruse browser
  • login to SAFE Network
  • in a new terminal console in safenetwork-fuse run:
    SAFENETWORKJS_TESTS=testing DEBUG=safenetworkjs*,safe-fuse* node bin.js

(Note: that is for the live network. If you want to save PUTs and use mock, you should run the Dev build of the browser, and include NODE_ENV=dev in the above command.)

At this point you should see some gobbledegook printed to the terminal console, and the browser should jump to the front, requesting authorisation. After you’ve given that, in a new terminal, try:

ls ~/SAFE

And if you see this, give yourself a pat on the back:

_public _publicNames

Success!

If you get to this point, please report back here and mention your setup and anything you think could be done to improve the instructions on github or here. And then play, and again please report back, or post questions and so on.

Bugs n Stuff

If you think you’ve found a bug you can post about it here and/or create an issue on github giving details of your setup, what you did, and what did or didn’t happen. As much detail as possible helps a lot.

For anyone who wants to try, its easy to capture debug output, just add this to the end of the command: 2>&1 | tee test.log and output will be saved to test.log which you can upload or post extracts from.

Windows & MacOS

The above instructions are all for Linux, but we have already made some progress with Windows thanks to @lukas and @benno, and with MacOS thanks to @Stout77. If anyone wants to continue those efforts which are tantalisingly close to working, please post on the following topics:

Debugging

This is complicated code so I don’t expect anyone to take it on, but it can be interesting to some and I do welcome anyone who wants to help with it. Or maybe you just fancy trying the debugger. This is fun (ok, I find it fun) and is well worth trying out if you are interested in how code works or why it does something in a certain way. To use the debugger:

  • install Chromium (or Chrome) Browser
  • start Chromium (or Chrome)
  • get rid of the rude login dialog (I never use it)
  • paste the following into the address bar: chrome://inspect/#devices
  • click Open dedicated DevTools for Node
  • start SAFE Drive as above, but changing node bin.js to node --inspect-brk bin.js

When you start SAFE Drive now, the debugger should open and you will need to press F8 to start the program. You can avoid that step by using --inspect instead of inspect-brk but breaking at the start gives you a chance to set a breakpoint before running the program. If you find the debugger confusing, post a request on this topic.

Oops It Stopped Working

After something fails things may stop working and you will keep getting errors. If that happens try this before running the SAFE Drive command again: pkill node; sudo umount ~/SAFE

Thanks anyone for helping. Everyone can join in if you are willing to have a go. I’m happy to help you whatever level you feel you are at and see if we can get you going. Others are welcome to join in and help people get going of course.

Notes:

SAFENETWORKJS_TESTS=testing causes SAFE Drive to automatically create an NFS container (~/SAFE/_public/tests/data1). This is used by the test script which you can run yourself, although be aware that it uses up to about 400 PUTs per run! So you’ll only be able to run it a maximum of twice on any account. You can run it on mock of course, but will still have to keep creating new accounts while using it (or keep a copy of the MockVault file for an empty account and keep copying that back to /tmp, which is what I do).

DEBUG=safenetworkjs*,safe-fuse* produces output to the terminal that will show what its doing, where it hangs etc. and can be modified to extend and filter what is printed according to the require('debug') statement in different source files. I tend to debug more with the console output than the debugger at the moment because there’s so much useful info printed to show what’s going on.

3 Likes
142

Ah right, the development branch, that was not mentioned in the github page, but as for using 400 puts, I may have mistyped it since it only used 4, though the extra folders did show up, ah well, its working.

Now to try and find ways to break it, shouldn’t be too hard at this stage.

edit:
moved a txt file over, took a bit of time but it showed up, opened it without trouble and closed it again, since the file is slightly over 1KB it used two puts, I was under the impression it was a MB, but that may be wrong, excellent work so far though!

3 Likes
143

I’ll try to test it this evening. Which version of npm to use, or do you think that doesn’t matter?

1 Like
144

Good question. I’m using npm v6.4.1 and node v8.11.3 but I don’t know what is or isn’t good. I’ve been tracking what Maidsafe are using but may be out of step.

1 Like
145

Great that you got going so quickly @isntism. You were too fast for me to mention the development branch sorry :slight_smile: But yes, please use developmentand NOT master which is way, way behind.

That sounds about right because a small file will require one PUT to update the directory, and another to put the data (which will all fit in the data map). @marcin wrote a good post which summarises what causes a PUT or a GET which helped me optimise the code. Here’s what he says about small files, but the post contains a lot more detail which is very useful for devs:

Note that if you copy a file that is already on SAFE it should take only one PUT not matter how big it is because I just insert a pointer to its data map in the directory. :slight_smile:

4 Likes
146

Ok, it seems that a lot of programs create a temporary file then delete the previous one and rename it, which does not seem to be working right now since I get “Error renaming temporary file: Operation not supported” when replacing it from my filebrowser, now you probably already know that and how to fix it, but I felt the need to say so just in case.

Is it possible to see other people’s public files yet, or even that git project you tested on, or is that still a wip?

1 Like
147

Can you say which apps you tried, what you did, what happened?

I haven’t re-tested this but previously found that some worked and some didn’t. File renaming should work though, so you may be onto something useful there. More info please :slight_smile:

Everything I described works, er might work :wink:, so long as you are on developement branch. I haven’t tried publishing a git repo live myself, but John ran the test script on the live network and that tries everything that is needed AKAIK, so it should work.

1 Like
148

That perticular error was thrown by ubuntu’s own fileroller as I copied a txt file and told it to overwrite the same-named file, I also tried saving an edited version using the default app for said txt file, which did cost 4 puts but didn’t actually change the file, so I ended up assuming that the same thing would happen for my other programs. (bionic beaver btw, in case that may end up being usefull)

Should I quickly fire ubuntu back up and try and run a debug version before going to bed, or can it wait till tomorrow?

2 Likes
149

No hurry, but as much detail as you can will help me find these apps and try them myself. I’m not sure what you mean by “ubuntu’s own fileroller” (if you mean the file manager, I think that can vary with the Ubun version and desktop) and don’t know what “bionic beaver” does for example. I could search but it may save time if you can give a bit more detail.

So when you get time if you can describe the steps that are needed to cause a problem, and what that was it can save me time trying to replicate it that would be great. But it’s fine if not, every bit of feedback helps point me in the right direction.

Also the version of Ubuntu and which desktop you have might help here if you know them.

I assume you are doing everything on the live network, which is great because most of my testing is done on mock.

Thanks a million for stepping in and helping out :slight_smile:

EDIT : I just saw “bionic beaver” flying past on twitter so that mystery has been resolved :wink:

2 Likes
150

I’m at the point of ls ~/SAFE :slight_smile:

I’m testing on Linux Mint 18.3. I had initially some compile and run errors. That seemed to be solved after I removed the old ~/.npm, yarn and ~/.cache dirs and/or the clean installation of npm v6.4.1 and node v8.11.3, like you described…

Remarks until now:

  • it seems npm v6.4.1 + node v8.11.3 is not a default combination. If you install node(+npm): then it is 8.11.1 (which comes with npm v5.6.0) or the most recent stable: node v10.13.0 and npm v6.4.1. Maybe this doesn’t matter that much. I’ll try to test tomorrow if the default node/npm version combination also works. But it could be the cause of my initial problems to get it working.
  • Do you also get a lot of warnings at the ‘npm install’ command in safenetwork.js like this one:?
    ./src/binding.cc:643:8: warning: ‘Nan::Maybe<bool> Nan::ForceSet(v8::Local<v8::Object>, v8::Local<v8::Value>, v8::Local<v8::Value>, v8::PropertyAttribute)’ is deprecated [-Wdeprecated-declarations]
  • The ‘npm link’ for safenetwork.js needs to be executed as root, at least on my system. This is necessary to have create perm. for symlink /usr/local/lib/node_modules/safenetworkjs. → Maybe an idea to say ‘sudo npm link’ in the github readme.md?
  • It seems to work, but I find it strange, that according to github of safenetwork-fuse, you have to do
    NODE_ENV=dev npm install
    But then connecting not to the mock, but to the alpha 2 network, it seems

→ Is the ‘NODE_ENV=dev’ necessary in this case?
I also get a lot of ‘deprecated’ warnings during this npm install command.

  • When I execute the find cmd in the ~/SAFE dir, the find (and fuse) hangs. I think on the mutable data entry, created by safe://to-do.
4 Likes
151

@draw Re:
npm versions: that’s useful to know. Thanks for reporting back. I don’t recall why I have the versions I do, but I guess I updated one at some point and am stuck until I update again.

npm install warnings: I don’t recall exactly, but yes I probably get those too. I just haven’t done npm install for a while. I tend to ignore npm warnings (but not errors) until something doesn’t work and then I go back and look again.

npm link: I would say needing sudo is not great, not terrible either, but a system configuration issue. I don’t need that on Ubuntu so suspect Mint isn’t setting permissions up in line with expectations.

NODE_ENV=dev npm install: is a Maidsafe feature which will be (may be already) deprecated. All it does is make sure the debug SAFE libraries are installed. Without it you may only get the production libraries and would not be able to use the mock network. I’m still using the old 0.8 APIs so it is probably still required for now.

Is the ‘NODE_ENV=dev’ necessary in this case? When running SAFE Drive NODE_ENV=dev selects mock network (dev SAFE APIs), so you use it when you want mock rather than live network.

When I execute the find cmd in the ~/SAFE dir, the find (and fuse) hangs.

Bingo! I hadn’t thought to try find. Nice catch. Looks like something to add to the test script. I can do this, or if you fancy working with mock and updating the script to fall over on this, be my guest. Just let me know so we don’t both do it! If you would like to add an issue for this that would be good too.

Thank you for diving in Ward, I really appreciate it.

2 Likes
152

About the sudo: that was more a remark to add the sudo to the github readme.md instructions, to make it clearer.
I’m not immediately planning to look too deep into the code/change the test script. But should I do that, I’ll let you know. If I have more time and desire, getting Python api working seems more interesting (technically) to me. Not a big fan of Javascript.
Does your solution use ‘caching’ atm? I do mean first saving it locally, and then syncing with the Safe Network at the background?
Because if I look at the Dropbox solution on Linux: that Dropbox folder is not a mount. What seems logical to me: you first write the changes to local disk and then sync with ‘the cloud’ afterwards.

2 Likes