@Mindphreaker, a quick question: you didn’t experience any of the problems I listed in the post above? Thanks
I never experienced such problems but to be honest, I never let the network run for a longer period of time (> 1 day). Maybe the network gets unstable after some time, but then it should occur in Alpha 2 too I guess. Maybe someone from maidsafe can tell what these errors could mean. From the nature of the error messages it seems, that there might be an issue with churned nodes, section splits, etc.
Hey @oetyng - I’m gonna try and get a look at this soon to see if I can figure out what happened. I know it’s a long shot, but you don’t happen to have any of the old logfiles from when you saw the failures do you?
@oetyng I’ve had a look at the problems you saw and reviewed the code related to them, but I can’t see any obvious issues there. There’s probably not much more I can do unless you still have the logfiles and know what version of the code you were running?
If you can still replicate the problems then we could try getting on a hangout or something to maybe speed things up a bit?
Hi @Fraser,thanks a lot for having a look and sorry for the late reply!
I will sit down with this in the weekend and try to get a good systematic approach with the setup to reproduce this and record everything. Then if I can gather the necessary information we could go through it, would be great!
So, hang on and I’ll get back and thanks again.
I cannot currently reproduce this.
There has been environment changes*, but cannot say if they influenced the outcome.
I’ve been running 3 nodes, with the same config as in Mindphreakers repo.
Did try with Windows Defender turned on and off (it has been interfering with VisualStudio MS Tests among things) but made no difference.
I did however come into some other problems after starting up more nodes (total of 9). But I will unfortunately not be able to make a full account of it this weekend. But stay tuned, I’ll get back to it.
* Environment changes
I upgraded Windows 10 to Fall Creators Update 1709, installed new .Net Framework (4.7.1) and added a preview version of VisualStudio 2017 (running in parallel though, so not the one used in these tests), plus installed microservice framework ServiceFabric runtime on the machine + SDK and tools for VS. That’s pretty much it. (And the top two changes naturally would be the most likely to have anything to do with this. I have no clue how.)
I wonder if anyone else has experienced this issue before…
It appears that I have to run exactly the
min_section_size number of nodes for the network to function. So for example, if I set the
2 then ran four vaults (creating what I thought would be two complete sections) the network ceases to function. If I then stop two of the vaults it starts to work again… Is there a minimum number of sections you have to have running in order for the network to function (if you want multiple sections)? Or is there a requirement that the sections shouldn’t all be ran on a single computer?
Thanks in advance!
The network should work with
min_section_size set to
2, although I wouldn’t expect the network to split until there are at least 10 vaults running.
We have a
SPLIT_BUFFER set to 3, which means that for a section to split, it would need to be able to split into two new sections so that each would have at least
min_section_size + SPLIT_BUFFER vaults.
When I say the network should work - I mean that setting the size as low as 2 will be OK assuming you have very low or no churn. What are the symptoms for you when you mention the network ceases to function?
No, just one should be fine.
It should also be fine to run on one machine assuming you have set up the config files as per the details in the OP.
Thank you very much for taking the time to reply!
Thank you for the insight, I had no idea that is how it worked! Once I added the tenth vault the network split successfully.
As soon as I introduce more than
min_section_size nodes onto the network then successful network connectivity stops. I have included a screen shot just to help illustrate my point. Here I have the same ten vaults as above running with
min_section_size set to 2. I am trying to register an account but the SAFE Browser will just sit like this for a few minutes and then throw a
Core error: Blocking operation was cancelled. I have tried a few different
min_section_size values but this behavior appears to always exists when having a greater number of vaults than
Do you have any idea what is going wrong?
I’m guessing you haven’t provided the same Routing config to the browser? The browser is probably connecting and expecting the normal 8 responses to requests, but the network is only providing 2 (or whatever you set the
min_section_size to for the vaults)
If you copy the
safe_vault.routing.config file to the folder where your browser binary is, then rename it to match the binary (probably to
safe-browser.routing.config) that should hopefully fix this for you
You don’t need to copy the
safe_vault.vault.config by the way, just the routing one.
Error using Peruse dev browser v0.6.0
That got things working! Thank you so much!
My knowledge of routing/crust is limited but I am gonna guess this is what is going on… I am guessing that by default, the SAFE Browser uses the number of vaults it can detect as the
minimum_section_size. If I run the
minimum_section_size equal to the number of active vaults everything works perfectly fine. Above this value it breaks… So it is almost like (without proper configuration) SAFE Browser sits and waits for say 4 acks (4 is the number of vaults running) but doesn’t receive them because the
min_section_size is set to 2 so only 2 acks are sent. Is this on the money?
Do I need to configure a Node.js/Electron app I am building to have a
abcde.routing.config too? My application exhibits the same behavior as the the browser before I included the
routing.config file. Tried it and sure enough, yes is the answer. Everything seams to be working now.
One thing I am noticing… The network really doesn’t seam to like me adding more vaults after I have written ANY data to the network (including an account). If I open two vaults, create an account, then add a third it stops responding to requests. If I start three vaults then create an account it works perfectly fine. Is this expected behavior or is something else wrong? Vaults leaving and joining seams to be a bit unstable is what I am experiencing. Maybe this is just where Maidsafe are in development though!
You shouldn’t need any config on your application side, the information to connect to the network is already provided in the auth URI returned by the browser (unless of course your app is actually a vault app which I guess it’s not).
Thus as long as your app is being authorised by the browser that is connected to your local network, and the response is successfully received by your app, your app should be able to connect to your local network without any config file.
EDIT: I just double checked, and the
min_section_size value is not passed encoded in the URI to the app (only the IP’s and the network name), thus for your case @DaBrown95 you do need the crust and routing config files as you said. Thanks a lot @Fraser for pointing this out to me. I get that if the local network was setup with
min_section_size set to 8 in the vaults, then the app will be able to connect without the config, is it right @Fraser? perhaps we can consider adding this value in the uri so it becomes network agnostic?
Yup - that should work.
That’s not expected, but we’ve not done
much any testing with
min_section_size set to
2 since a network like that would be very prone to losing data during churn. I’ll try and replicate this issue over the weekend, but amongst other things, this critical event might get in the way
I have not started my own local network yet so this is just my intuition talking. Maybe this is not exactly the procedure you are following, but launching only 2 or 3 vaults seems way too small for a stable consensus network. I would think that you would want at least 16, 32, 64, or even 128 vaults (Can anyone share how many there are on Alpha-2?) before adding accounts and storing data in a stable manner. The default min_section_size=8 would seem to be indicative of a stable number, and I would think your going to want at least 4 section prefixes 00,10,01,11. So if you only have 2 computers to work with try starting up 16 or 32 vaults on each before adding any data. Maybe @Fraser can clarify this for us? It might be a good experiment for you to see how many instances you can load on a single server before performance suffers… that’s one test I’ve been meaning to try.
This is pretty intersting to hear! I had no idea that the URI could be used in this manner. I for one feel that including that piece of information in the URI could be useful. Always nice to keep things more generic. Although I do realise that for the vast majority of cases it would be ‘wasted’ bits. Thanks as always @bochaco for your insight!
Yeah possibly me having the section size this low is very unrealistic for the network, it is simply because I am worried about bottlenecking the network by running too many vaults on my machine! What have you, or others, found the maximum number of vaults you can run on a single machine? Maybe I am babying my machine a little too much…
Good stuff, Always have to make time for the truly important things in life!
I have always wondered this myself!
You are probably onto something, I am going to try this soon.
I am going to be doing this a lot over the next few weeks so I will report back my findings.
We can guess a lower bound by looking at “SAFE Browser.crust.config”: it has 25 contacts.
This isn’t necessarily the total number because Maidsafe may expose only of subset of the vaults in the config file.
Congratulations: SCOTLAND 22-13 ENGLAND