Possible routing bug in test 16 network

Core Libraries Routing
1

This post didn’t get any answers from Maidsafe. I duplicate it in this forum with the hope to revive the question.

I think the symptoms are indicative of a bug: a node has a normally a unique birth date and later a unique death date and the probability that later a node is created with the same node id is quasi null. But in test 16 this happened on at least 39 nodes. The phenomenon seems localized in time and in space, here are the main examples:

  • On 2017-04-06 in section “000” (“143156…”, “14dbba…”, “15b812…”)

  • On 2017-04-08 in section “110” (“c008ea…”, “c26ebf…”, “c273ee…”, “c2b336…”, “c3287b…”, “c3e5cd…”, “c3f46e…”, “e33b56…”)

  • On 2017-04-09 in section in “0011” (“3022d9…”, “3101b5…”, “3271a1…”, “32cc41…”, “336045…”, “3453d0…”, “34badc…”, “3512ee…”, “353d82…”, “35e5f4…”, “36a94f…”, “37bdf9…”, “4bf2e9…”)

  • On 2017-04-11/12 in section “010” (“5086b9…”, “515b32…”, “519d35…”, “51be37…”, “522189…”, “5305f5…”, “53d215…”, “53d3fd…”, “5410da…”, “5413fd…”, “567bb3…”, “56d73d…”)

I have the feeling that sometimes a section is in a bad state and when one of its nodes disconnects, its disappearance is not immediate as it should be, but is intermittent until it is definitively dead. This can last from several minutes to several hours. The worst case is node id “36a94f…” which was reborn 24 times! (on 2017-04-09: first death at 21:57:37 and last death at 23:30:25).

If the root cause cannot be determined and corrected, then a workaround could be to store in each node a btreeset of node ids that have disappeared from current section and to refuse any later connection of these node ids.

Here is a json text containing the nodes in test 16 that had several deaths and rebirths (precision of dates is about 1 minute and a half):

{
  "nodes":
  {
    
    "143156..":
    {
      "birth":[{"p":"000","d":"2017-04-06 21-35-31"},{"p":"000","d":"2017-04-06 21-54-59"}],
      "death":[{"p":"000","d":"2017-04-06 21-53-58"},{"p":"000","d":"2017-04-06 22-58-22"}]
    },
    "14dbba..":
    {
      "birth":[{"p":"000","d":"2017-04-06 22-42-18"},{"p":"000","d":"2017-04-06 22-47-08"},{"p":"000","d":"2017-04-06 22-50-57"},{"p":"000","d":"2017-04-06 22-53-00"}],
      "death":[{"p":"000","d":"2017-04-06 22-45-10"},{"p":"000","d":"2017-04-06 22-49-37"},{"p":"000","d":"2017-04-06 22-51-58"},{"p":"000","d":"2017-04-06 22-54-02"}]
    },
    "15b812..":
    {
      "birth":[{"p":"000","d":"2017-04-06 22-50-57"},{"p":"000","d":"2017-04-06 22-53-00"}],
      "death":[{"p":"000","d":"2017-04-06 22-51-58"},{"p":"000","d":"2017-04-06 22-54-02"}]
    },
    "1f2df8..":
    {
      "birth":[{"p":"000","d":"2017-04-06 21-12-14"},{"p":"000","d":"2017-04-12 00-52-19"}],
      "death":[{"p":"000","d":"2017-04-12 00-51-17"},{"p":"000","d":"2017-04-12 01-21-04"}]
    },
    "26c398..":
    {
      "birth":[{"p":"001","d":"2017-04-08 18-29-24"},{"p":"001","d":"2017-04-08 18-35-07"}],
      "death":[{"p":"001","d":"2017-04-08 18-34-03"},{"p":"001","d":"2017-04-08 18-37-15"}]
    },
    "3022d9..":
    {
      "birth":[{"p":"001","d":"2017-04-09 21-13-16"},{"p":"0011","d":"2017-04-09 21-58-39"},{"p":"0011","d":"2017-04-09 22-08-49"},{"p":"0011","d":"2017-04-09 22-10-52"},{"p":"0011","d":"2017-04-09 22-12-54"},{"p":"0011","d":"2017-04-09 22-14-56"},{"p":"0011","d":"2017-04-09 22-20-02"},{"p":"0011","d":"2017-04-09 22-24-07"},{"p":"0011","d":"2017-04-09 22-26-09"},{"p":"0011","d":"2017-04-09 22-30-14"},{"p":"0011","d":"2017-04-09 22-35-21"},{"p":"0011","d":"2017-04-09 22-41-28"},{"p":"0011","d":"2017-04-09 22-47-35"},{"p":"0011","d":"2017-04-09 22-49-37"}],
      "death":[{"p":"0011","d":"2017-04-09 21-56-36"},{"p":"0011","d":"2017-04-09 22-03-44"},{"p":"0011","d":"2017-04-09 22-09-50"},{"p":"0011","d":"2017-04-09 22-11-52"},{"p":"0011","d":"2017-04-09 22-13-54"},{"p":"0011","d":"2017-04-09 22-19-00"},{"p":"0011","d":"2017-04-09 22-21-02"},{"p":"0011","d":"2017-04-09 22-25-07"},{"p":"0011","d":"2017-04-09 22-29-13"},{"p":"0011","d":"2017-04-09 22-31-15"},{"p":"0011","d":"2017-04-09 22-36-22"},{"p":"0011","d":"2017-04-09 22-43-30"},{"p":"0011","d":"2017-04-09 22-48-36"},{"p":"0011","d":"2017-04-09 22-50-38"}]
    },
    "3101b5..":
    {
      "birth":[{"p":"0011","d":"2017-04-09 22-15-57"},{"p":"0011","d":"2017-04-09 22-17-59"},{"p":"0011","d":"2017-04-09 22-27-10"},{"p":"0011","d":"2017-04-09 22-33-17"},{"p":"0011","d":"2017-04-09 22-35-21"},{"p":"0011","d":"2017-04-09 22-42-29"},{"p":"0011","d":"2017-04-09 22-50-38"},{"p":"0011","d":"2017-04-09 22-55-44"},{"p":"0011","d":"2017-04-09 22-59-49"},{"p":"0011","d":"2017-04-09 23-04-56"},{"p":"0011","d":"2017-04-09 23-12-03"},{"p":"0011","d":"2017-04-09 23-18-09"},{"p":"0011","d":"2017-04-09 23-21-14"},{"p":"0011","d":"2017-04-09 23-24-18"},{"p":"0011","d":"2017-04-09 23-26-20"}],
      "death":[{"p":"0011","d":"2017-04-09 22-16-58"},{"p":"0011","d":"2017-04-09 22-26-09"},{"p":"0011","d":"2017-04-09 22-28-11"},{"p":"0011","d":"2017-04-09 22-34-19"},{"p":"0011","d":"2017-04-09 22-36-22"},{"p":"0011","d":"2017-04-09 22-44-31"},{"p":"0011","d":"2017-04-09 22-54-43"},{"p":"0011","d":"2017-04-09 22-56-45"},{"p":"0011","d":"2017-04-09 23-00-50"},{"p":"0011","d":"2017-04-09 23-06-58"},{"p":"0011","d":"2017-04-09 23-14-05"},{"p":"0011","d":"2017-04-09 23-19-11"},{"p":"0011","d":"2017-04-09 23-23-16"},{"p":"0011","d":"2017-04-09 23-25-19"},{"p":"0011","d":"2017-04-09 23-27-21"}]
    },
    "3271a1..":
    {
      "birth":[{"p":"0011","d":"2017-04-09 22-26-09"},{"p":"0011","d":"2017-04-09 22-31-15"},{"p":"0011","d":"2017-04-09 22-33-17"},{"p":"0011","d":"2017-04-09 22-39-26"},{"p":"0011","d":"2017-04-09 22-43-30"},{"p":"0011","d":"2017-04-09 22-45-32"},{"p":"0011","d":"2017-04-09 22-49-37"},{"p":"0011","d":"2017-04-09 22-51-39"},{"p":"0011","d":"2017-04-09 22-54-43"},{"p":"0011","d":"2017-04-09 22-57-46"},{"p":"0011","d":"2017-04-09 23-01-51"},{"p":"0011","d":"2017-04-09 23-06-58"},{"p":"0011","d":"2017-04-09 23-10-01"},{"p":"0011","d":"2017-04-09 23-15-06"},{"p":"0011","d":"2017-04-09 23-18-09"},{"p":"0011","d":"2017-04-09 23-26-20"}],
      "death":[{"p":"0011","d":"2017-04-09 22-27-10"},{"p":"0011","d":"2017-04-09 22-32-16"},{"p":"0011","d":"2017-04-09 22-37-23"},{"p":"0011","d":"2017-04-09 22-40-27"},{"p":"0011","d":"2017-04-09 22-44-31"},{"p":"0011","d":"2017-04-09 22-46-34"},{"p":"0011","d":"2017-04-09 22-50-38"},{"p":"0011","d":"2017-04-09 22-52-41"},{"p":"0011","d":"2017-04-09 22-55-44"},{"p":"0011","d":"2017-04-09 22-58-48"},{"p":"0011","d":"2017-04-09 23-05-57"},{"p":"0011","d":"2017-04-09 23-07-58"},{"p":"0011","d":"2017-04-09 23-11-02"},{"p":"0011","d":"2017-04-09 23-16-07"},{"p":"0011","d":"2017-04-09 23-22-15"},{"p":"0011","d":"2017-04-09 23-27-21"}]
    },
    "32cc41..":
    {
      "birth":[{"p":"001","d":"2017-04-09 20-59-41"},{"p":"0011","d":"2017-04-09 21-59-40"},{"p":"0011","d":"2017-04-09 22-02-43"},{"p":"0011","d":"2017-04-09 22-07-48"},{"p":"0011","d":"2017-04-09 22-09-50"},{"p":"0011","d":"2017-04-09 22-15-57"},{"p":"0011","d":"2017-04-09 22-17-59"},{"p":"0011","d":"2017-04-09 22-20-02"},{"p":"0011","d":"2017-04-09 22-24-07"},{"p":"0011","d":"2017-04-09 22-28-11"},{"p":"0011","d":"2017-04-09 22-33-17"},{"p":"0011","d":"2017-04-09 22-35-21"},{"p":"0011","d":"2017-04-09 22-50-38"},{"p":"0011","d":"2017-04-09 23-12-03"},{"p":"0011","d":"2017-04-09 23-14-05"},{"p":"0011","d":"2017-04-09 23-18-09"},{"p":"0011","d":"2017-04-09 23-20-12"},{"p":"0011","d":"2017-04-09 23-23-16"}],
      "death":[{"p":"0011","d":"2017-04-09 21-57-37"},{"p":"0011","d":"2017-04-09 22-00-40"},{"p":"0011","d":"2017-04-09 22-03-44"},{"p":"0011","d":"2017-04-09 22-08-49"},{"p":"0011","d":"2017-04-09 22-13-54"},{"p":"0011","d":"2017-04-09 22-16-58"},{"p":"0011","d":"2017-04-09 22-19-00"},{"p":"0011","d":"2017-04-09 22-21-02"},{"p":"0011","d":"2017-04-09 22-25-07"},{"p":"0011","d":"2017-04-09 22-30-14"},{"p":"0011","d":"2017-04-09 22-34-19"},{"p":"0011","d":"2017-04-09 22-38-25"},{"p":"0011","d":"2017-04-09 22-51-39"},{"p":"0011","d":"2017-04-09 23-13-04"},{"p":"0011","d":"2017-04-09 23-17-08"},{"p":"0011","d":"2017-04-09 23-19-11"},{"p":"0011","d":"2017-04-09 23-22-15"},{"p":"0011","d":"2017-04-09 23-30-25"}]
    },
    "336045..":
    {
      "birth":[{"p":"0011","d":"2017-04-09 21-47-24"},{"p":"0011","d":"2017-04-09 21-59-40"},{"p":"0011","d":"2017-04-09 22-10-52"},{"p":"0011","d":"2017-04-09 22-12-54"},{"p":"0011","d":"2017-04-09 22-14-56"},{"p":"0011","d":"2017-04-09 22-17-59"},{"p":"0011","d":"2017-04-09 22-20-02"},{"p":"0011","d":"2017-04-09 22-24-07"},{"p":"0011","d":"2017-04-09 22-28-11"},{"p":"0011","d":"2017-04-09 22-32-16"},{"p":"0011","d":"2017-04-09 22-40-27"},{"p":"0011","d":"2017-04-09 22-44-31"},{"p":"0011","d":"2017-04-09 22-47-35"},{"p":"0011","d":"2017-04-09 22-50-38"},{"p":"0011","d":"2017-04-09 22-53-41"},{"p":"0011","d":"2017-04-09 22-58-48"},{"p":"0011","d":"2017-04-09 23-04-56"},{"p":"0011","d":"2017-04-09 23-07-58"},{"p":"0011","d":"2017-04-09 23-11-02"},{"p":"0011","d":"2017-04-09 23-13-04"},{"p":"0011","d":"2017-04-09 23-19-11"},{"p":"0011","d":"2017-04-09 23-24-18"},{"p":"0011","d":"2017-04-09 23-28-23"}],
      "death":[{"p":"0011","d":"2017-04-09 21-56-36"},{"p":"0011","d":"2017-04-09 22-03-44"},{"p":"0011","d":"2017-04-09 22-11-52"},{"p":"0011","d":"2017-04-09 22-13-54"},{"p":"0011","d":"2017-04-09 22-16-58"},{"p":"0011","d":"2017-04-09 22-19-00"},{"p":"0011","d":"2017-04-09 22-21-02"},{"p":"0011","d":"2017-04-09 22-27-10"},{"p":"0011","d":"2017-04-09 22-29-13"},{"p":"0011","d":"2017-04-09 22-38-25"},{"p":"0011","d":"2017-04-09 22-41-28"},{"p":"0011","d":"2017-04-09 22-46-34"},{"p":"0011","d":"2017-04-09 22-48-36"},{"p":"0011","d":"2017-04-09 22-51-39"},{"p":"0011","d":"2017-04-09 22-57-46"},{"p":"0011","d":"2017-04-09 23-00-50"},{"p":"0011","d":"2017-04-09 23-06-58"},{"p":"0011","d":"2017-04-09 23-09-00"},{"p":"0011","d":"2017-04-09 23-12-03"},{"p":"0011","d":"2017-04-09 23-16-07"},{"p":"0011","d":"2017-04-09 23-22-15"},{"p":"0011","d":"2017-04-09 23-25-19"},{"p":"0011","d":"2017-04-09 23-30-25"}]
    },
    "3453d0..":
    {
      "birth":[{"p":"0011","d":"2017-04-09 22-06-47"},{"p":"0011","d":"2017-04-09 22-35-21"},{"p":"0011","d":"2017-04-09 22-46-34"},{"p":"0011","d":"2017-04-09 22-49-37"},{"p":"0011","d":"2017-04-09 23-11-02"},{"p":"0011","d":"2017-04-09 23-14-05"},{"p":"0011","d":"2017-04-09 23-17-08"}],
      "death":[{"p":"0011","d":"2017-04-09 22-34-19"},{"p":"0011","d":"2017-04-09 22-45-32"},{"p":"0011","d":"2017-04-09 22-48-36"},{"p":"0011","d":"2017-04-09 23-10-01"},{"p":"0011","d":"2017-04-09 23-12-03"},{"p":"0011","d":"2017-04-09 23-15-06"},{"p":"0011","d":"2017-04-09 23-30-25"}]
    },
    "34badc..":
    {
      "birth":[{"p":"0011","d":"2017-04-09 22-50-38"},{"p":"0011","d":"2017-04-09 22-58-48"},{"p":"0011","d":"2017-04-09 23-02-53"},{"p":"0011","d":"2017-04-09 23-05-57"},{"p":"0011","d":"2017-04-09 23-09-00"},{"p":"0011","d":"2017-04-09 23-12-03"}],
      "death":[{"p":"0011","d":"2017-04-09 22-52-41"},{"p":"0011","d":"2017-04-09 22-59-49"},{"p":"0011","d":"2017-04-09 23-03-54"},{"p":"0011","d":"2017-04-09 23-07-58"},{"p":"0011","d":"2017-04-09 23-10-01"},{"p":"0011","d":"2017-04-09 23-13-04"}]
    },
    "3512ee..":
    {
      "birth":[{"p":"001","d":"2017-04-09 21-30-51"},{"p":"0011","d":"2017-04-09 21-59-40"},{"p":"0011","d":"2017-04-09 22-02-43"},{"p":"0011","d":"2017-04-09 22-04-45"},{"p":"0011","d":"2017-04-09 22-08-49"},{"p":"0011","d":"2017-04-09 22-10-52"},{"p":"0011","d":"2017-04-09 22-12-54"},{"p":"0011","d":"2017-04-09 22-15-57"},{"p":"0011","d":"2017-04-09 22-24-07"},{"p":"0011","d":"2017-04-09 22-35-21"},{"p":"0011","d":"2017-04-09 22-50-38"},{"p":"0011","d":"2017-04-09 22-58-48"},{"p":"0011","d":"2017-04-09 23-04-56"},{"p":"0011","d":"2017-04-09 23-11-02"},{"p":"0011","d":"2017-04-09 23-16-07"},{"p":"0011","d":"2017-04-09 23-18-09"},{"p":"0011","d":"2017-04-09 23-22-15"},{"p":"0011","d":"2017-04-09 23-26-20"}],
      "death":[{"p":"0011","d":"2017-04-09 21-56-36"},{"p":"0011","d":"2017-04-09 22-01-41"},{"p":"0011","d":"2017-04-09 22-03-44"},{"p":"0011","d":"2017-04-09 22-05-46"},{"p":"0011","d":"2017-04-09 22-09-50"},{"p":"0011","d":"2017-04-09 22-11-52"},{"p":"0011","d":"2017-04-09 22-13-54"},{"p":"0011","d":"2017-04-09 22-23-05"},{"p":"0011","d":"2017-04-09 22-25-07"},{"p":"0011","d":"2017-04-09 22-36-22"},{"p":"0011","d":"2017-04-09 22-53-41"},{"p":"0011","d":"2017-04-09 23-00-50"},{"p":"0011","d":"2017-04-09 23-05-57"},{"p":"0011","d":"2017-04-09 23-12-03"},{"p":"0011","d":"2017-04-09 23-17-08"},{"p":"0011","d":"2017-04-09 23-19-11"},{"p":"0011","d":"2017-04-09 23-25-19"},{"p":"0011","d":"2017-04-09 23-27-21"}]
    },
    "353d82..":
    {
      "birth":[{"p":"0011","d":"2017-04-09 21-57-37"},{"p":"0011","d":"2017-04-09 22-44-31"}],
      "death":[{"p":"0011","d":"2017-04-09 22-43-30"},{"p":"0011","d":"2017-04-09 23-28-23"}]
    },
    "35e5f4..":
    {
      "birth":[{"p":"001","d":"2017-04-09 21-22-25"},{"p":"0011","d":"2017-04-09 21-59-40"},{"p":"0011","d":"2017-04-09 22-04-45"},{"p":"0011","d":"2017-04-09 22-08-49"},{"p":"0011","d":"2017-04-09 22-10-52"},{"p":"0011","d":"2017-04-09 22-12-54"},{"p":"0011","d":"2017-04-09 22-15-57"},{"p":"0011","d":"2017-04-09 22-17-59"},{"p":"0011","d":"2017-04-09 22-20-02"},{"p":"0011","d":"2017-04-09 22-26-09"},{"p":"0011","d":"2017-04-09 22-29-13"},{"p":"0011","d":"2017-04-09 22-31-15"},{"p":"0011","d":"2017-04-09 22-43-30"},{"p":"0011","d":"2017-04-09 22-46-34"},{"p":"0011","d":"2017-04-09 22-56-45"},{"p":"0011","d":"2017-04-09 23-11-02"},{"p":"0011","d":"2017-04-09 23-15-06"},{"p":"0011","d":"2017-04-09 23-17-08"},{"p":"0011","d":"2017-04-09 23-20-12"},{"p":"0011","d":"2017-04-09 23-28-23"}],
      "death":[{"p":"0011","d":"2017-04-09 21-57-37"},{"p":"0011","d":"2017-04-09 22-03-44"},{"p":"0011","d":"2017-04-09 22-05-46"},{"p":"0011","d":"2017-04-09 22-09-50"},{"p":"0011","d":"2017-04-09 22-11-52"},{"p":"0011","d":"2017-04-09 22-14-56"},{"p":"0011","d":"2017-04-09 22-16-58"},{"p":"0011","d":"2017-04-09 22-19-00"},{"p":"0011","d":"2017-04-09 22-25-07"},{"p":"0011","d":"2017-04-09 22-28-11"},{"p":"0011","d":"2017-04-09 22-30-14"},{"p":"0011","d":"2017-04-09 22-41-28"},{"p":"0011","d":"2017-04-09 22-45-32"},{"p":"0011","d":"2017-04-09 22-54-43"},{"p":"0011","d":"2017-04-09 23-10-01"},{"p":"0011","d":"2017-04-09 23-12-03"},{"p":"0011","d":"2017-04-09 23-16-07"},{"p":"0011","d":"2017-04-09 23-18-09"},{"p":"0011","d":"2017-04-09 23-26-20"},{"p":"0011","d":"2017-04-09 23-30-25"}]
    },
    "36a94f..":
    {
      "birth":[{"p":"001","d":"2017-04-09 21-04-48"},{"p":"0011","d":"2017-04-09 21-59-40"},{"p":"0011","d":"2017-04-09 22-02-43"},{"p":"0011","d":"2017-04-09 22-04-45"},{"p":"0011","d":"2017-04-09 22-06-47"},{"p":"0011","d":"2017-04-09 22-08-49"},{"p":"0011","d":"2017-04-09 22-10-52"},{"p":"0011","d":"2017-04-09 22-15-57"},{"p":"0011","d":"2017-04-09 22-17-59"},{"p":"0011","d":"2017-04-09 22-20-02"},{"p":"0011","d":"2017-04-09 22-24-07"},{"p":"0011","d":"2017-04-09 22-26-09"},{"p":"0011","d":"2017-04-09 22-28-11"},{"p":"0011","d":"2017-04-09 22-30-14"},{"p":"0011","d":"2017-04-09 22-35-21"},{"p":"0011","d":"2017-04-09 22-37-23"},{"p":"0011","d":"2017-04-09 22-44-31"},{"p":"0011","d":"2017-04-09 22-46-34"},{"p":"0011","d":"2017-04-09 22-50-38"},{"p":"0011","d":"2017-04-09 22-52-41"},{"p":"0011","d":"2017-04-09 22-54-43"},{"p":"0011","d":"2017-04-09 22-56-45"},{"p":"0011","d":"2017-04-09 22-58-48"},{"p":"0011","d":"2017-04-09 23-09-00"},{"p":"0011","d":"2017-04-09 23-27-21"}],
      "death":[{"p":"0011","d":"2017-04-09 21-57-37"},{"p":"0011","d":"2017-04-09 22-00-40"},{"p":"0011","d":"2017-04-09 22-03-44"},{"p":"0011","d":"2017-04-09 22-05-46"},{"p":"0011","d":"2017-04-09 22-07-48"},{"p":"0011","d":"2017-04-09 22-09-50"},{"p":"0011","d":"2017-04-09 22-13-54"},{"p":"0011","d":"2017-04-09 22-16-58"},{"p":"0011","d":"2017-04-09 22-19-00"},{"p":"0011","d":"2017-04-09 22-23-05"},{"p":"0011","d":"2017-04-09 22-25-07"},{"p":"0011","d":"2017-04-09 22-27-10"},{"p":"0011","d":"2017-04-09 22-29-13"},{"p":"0011","d":"2017-04-09 22-31-15"},{"p":"0011","d":"2017-04-09 22-36-22"},{"p":"0011","d":"2017-04-09 22-40-27"},{"p":"0011","d":"2017-04-09 22-45-32"},{"p":"0011","d":"2017-04-09 22-47-35"},{"p":"0011","d":"2017-04-09 22-51-39"},{"p":"0011","d":"2017-04-09 22-53-41"},{"p":"0011","d":"2017-04-09 22-55-44"},{"p":"0011","d":"2017-04-09 22-57-46"},{"p":"0011","d":"2017-04-09 23-06-58"},{"p":"0011","d":"2017-04-09 23-25-19"},{"p":"0011","d":"2017-04-09 23-30-25"}]
    },
    "37bdf9..":
    {
      "birth":[{"p":"0011","d":"2017-04-09 21-39-11"},{"p":"0011","d":"2017-04-09 21-43-18"},{"p":"0011","d":"2017-04-09 22-14-56"},{"p":"0011","d":"2017-04-09 22-30-14"},{"p":"0011","d":"2017-04-09 22-32-16"},{"p":"0011","d":"2017-04-09 22-35-21"},{"p":"0011","d":"2017-04-09 22-38-25"},{"p":"0011","d":"2017-04-09 22-54-43"},{"p":"0011","d":"2017-04-09 22-57-46"},{"p":"0011","d":"2017-04-09 23-01-51"}],
      "death":[{"p":"0011","d":"2017-04-09 21-42-17"},{"p":"0011","d":"2017-04-09 22-13-54"},{"p":"0011","d":"2017-04-09 22-27-10"},{"p":"0011","d":"2017-04-09 22-31-15"},{"p":"0011","d":"2017-04-09 22-34-19"},{"p":"0011","d":"2017-04-09 22-36-22"},{"p":"0011","d":"2017-04-09 22-52-41"},{"p":"0011","d":"2017-04-09 22-55-44"},{"p":"0011","d":"2017-04-09 22-59-49"},{"p":"0011","d":"2017-04-09 23-30-25"}]
    },
    "4bf2e9..":
    {
      "birth":[{"p":"0100","d":"2017-04-07 13-47-15"},{"p":"0100","d":"2017-04-07 13-54-37"},{"p":"0100","d":"2017-04-07 13-56-57"},{"p":"0100","d":"2017-04-07 14-00-59"},{"p":"0100","d":"2017-04-07 14-03-13"},{"p":"0100","d":"2017-04-07 14-23-13"}],
      "death":[{"p":"0100","d":"2017-04-07 13-53-17"},{"p":"0100","d":"2017-04-07 13-55-39"},{"p":"0100","d":"2017-04-07 13-57-59"},{"p":"0100","d":"2017-04-07 14-02-07"},{"p":"0100","d":"2017-04-07 14-22-07"},{"p":"0100","d":"2017-04-07 14-54-08"}]
    },
    "5086b9..":
    {
      "birth":[{"p":"010","d":"2017-04-11 22-18-00"},{"p":"010","d":"2017-04-11 23-08-18"},{"p":"010","d":"2017-04-11 23-13-24"},{"p":"010","d":"2017-04-11 23-18-30"},{"p":"010","d":"2017-04-11 23-22-35"},{"p":"010","d":"2017-04-12 00-03-23"},{"p":"010","d":"2017-04-12 00-05-26"},{"p":"010","d":"2017-04-12 00-09-31"},{"p":"010","d":"2017-04-12 00-17-46"},{"p":"010","d":"2017-04-12 00-20-49"},{"p":"010","d":"2017-04-12 00-25-56"},{"p":"010","d":"2017-04-12 00-37-20"}],
      "death":[{"p":"010","d":"2017-04-11 23-06-15"},{"p":"010","d":"2017-04-11 23-10-20"},{"p":"010","d":"2017-04-11 23-16-27"},{"p":"010","d":"2017-04-11 23-20-32"},{"p":"010","d":"2017-04-11 23-24-37"},{"p":"010","d":"2017-04-12 00-04-24"},{"p":"010","d":"2017-04-12 00-08-30"},{"p":"010","d":"2017-04-12 00-16-45"},{"p":"010","d":"2017-04-12 00-19-48"},{"p":"010","d":"2017-04-12 00-24-55"},{"p":"010","d":"2017-04-12 00-36-18"},{"p":"010","d":"2017-04-12 00-46-58"}]
    },
    "515b32..":
    {
      "birth":[{"p":"010","d":"2017-04-11 22-07-03"},{"p":"010","d":"2017-04-11 22-19-01"},{"p":"010","d":"2017-04-11 22-32-33"},{"p":"010","d":"2017-04-11 22-35-36"},{"p":"010","d":"2017-04-11 22-40-43"},{"p":"010","d":"2017-04-11 22-43-47"},{"p":"010","d":"2017-04-11 22-47-52"},{"p":"010","d":"2017-04-11 22-50-57"},{"p":"010","d":"2017-04-11 22-56-04"},{"p":"010","d":"2017-04-11 23-01-09"},{"p":"010","d":"2017-04-11 23-04-13"},{"p":"010","d":"2017-04-11 23-09-19"},{"p":"010","d":"2017-04-11 23-14-25"},{"p":"010","d":"2017-04-11 23-20-32"},{"p":"010","d":"2017-04-12 00-03-23"},{"p":"010","d":"2017-04-12 00-12-38"},{"p":"010","d":"2017-04-12 00-25-56"},{"p":"010","d":"2017-04-12 00-27-59"},{"p":"010","d":"2017-04-12 00-31-09"},{"p":"010","d":"2017-04-12 00-34-15"}],
      "death":[{"p":"010","d":"2017-04-11 22-18-00"},{"p":"010","d":"2017-04-11 22-31-33"},{"p":"010","d":"2017-04-11 22-34-35"},{"p":"010","d":"2017-04-11 22-38-40"},{"p":"010","d":"2017-04-11 22-41-44"},{"p":"010","d":"2017-04-11 22-44-48"},{"p":"010","d":"2017-04-11 22-48-53"},{"p":"010","d":"2017-04-11 22-51-58"},{"p":"010","d":"2017-04-11 22-57-05"},{"p":"010","d":"2017-04-11 23-02-10"},{"p":"010","d":"2017-04-11 23-05-14"},{"p":"010","d":"2017-04-11 23-10-20"},{"p":"010","d":"2017-04-11 23-18-30"},{"p":"010","d":"2017-04-11 23-21-34"},{"p":"010","d":"2017-04-12 00-04-24"},{"p":"010","d":"2017-04-12 00-13-40"},{"p":"010","d":"2017-04-12 00-26-58"},{"p":"010","d":"2017-04-12 00-29-01"},{"p":"010","d":"2017-04-12 00-32-12"},{"p":"010","d":"2017-04-12 00-35-16"}]
    },
    "519d35..":
    {
      "birth":[{"p":"010","d":"2017-04-11 21-40-42"},{"p":"010","d":"2017-04-11 22-32-33"},{"p":"010","d":"2017-04-11 22-36-38"},{"p":"010","d":"2017-04-11 22-40-43"},{"p":"010","d":"2017-04-11 22-46-51"},{"p":"010","d":"2017-04-11 22-49-54"},{"p":"010","d":"2017-04-11 22-55-02"},{"p":"010","d":"2017-04-11 22-59-07"},{"p":"010","d":"2017-04-11 23-05-14"},{"p":"010","d":"2017-04-11 23-07-16"},{"p":"010","d":"2017-04-11 23-11-21"},{"p":"010","d":"2017-04-11 23-13-24"},{"p":"010","d":"2017-04-11 23-19-31"},{"p":"010","d":"2017-04-11 23-23-36"},{"p":"010","d":"2017-04-11 23-25-39"},{"p":"010","d":"2017-04-11 23-33-48"},{"p":"010","d":"2017-04-11 23-36-51"},{"p":"010","d":"2017-04-11 23-41-57"},{"p":"010","d":"2017-04-11 23-46-03"},{"p":"010","d":"2017-04-11 23-49-06"},{"p":"010","d":"2017-04-11 23-56-14"},{"p":"010","d":"2017-04-11 23-58-17"},{"p":"010","d":"2017-04-12 00-00-19"}],
      "death":[{"p":"010","d":"2017-04-11 22-31-33"},{"p":"010","d":"2017-04-11 22-33-35"},{"p":"010","d":"2017-04-11 22-37-39"},{"p":"010","d":"2017-04-11 22-41-44"},{"p":"010","d":"2017-04-11 22-47-52"},{"p":"010","d":"2017-04-11 22-53-00"},{"p":"010","d":"2017-04-11 22-58-06"},{"p":"010","d":"2017-04-11 23-01-09"},{"p":"010","d":"2017-04-11 23-06-15"},{"p":"010","d":"2017-04-11 23-09-19"},{"p":"010","d":"2017-04-11 23-12-23"},{"p":"010","d":"2017-04-11 23-16-27"},{"p":"010","d":"2017-04-11 23-22-35"},{"p":"010","d":"2017-04-11 23-24-37"},{"p":"010","d":"2017-04-11 23-26-39"},{"p":"010","d":"2017-04-11 23-34-49"},{"p":"010","d":"2017-04-11 23-39-54"},{"p":"010","d":"2017-04-11 23-43-59"},{"p":"010","d":"2017-04-11 23-47-03"},{"p":"010","d":"2017-04-11 23-51-08"},{"p":"010","d":"2017-04-11 23-57-16"},{"p":"010","d":"2017-04-11 23-59-18"},{"p":"010","d":"2017-04-12 00-01-21"}]
    },
    "51be37..":
    {
      "birth":[{"p":"010","d":"2017-04-11 22-54-01"},{"p":"010","d":"2017-04-11 22-56-04"},{"p":"010","d":"2017-04-11 22-59-07"},{"p":"010","d":"2017-04-11 23-03-11"},{"p":"010","d":"2017-04-11 23-06-15"},{"p":"010","d":"2017-04-11 23-16-27"},{"p":"010","d":"2017-04-11 23-20-32"},{"p":"010","d":"2017-04-11 23-22-35"},{"p":"010","d":"2017-04-11 23-30-44"},{"p":"010","d":"2017-04-11 23-32-47"},{"p":"010","d":"2017-04-11 23-53-10"},{"p":"010","d":"2017-04-11 23-56-14"},{"p":"010","d":"2017-04-11 23-59-18"},{"p":"010","d":"2017-04-12 00-03-23"},{"p":"010","d":"2017-04-12 00-08-30"},{"p":"010","d":"2017-04-12 00-25-56"}],
      "death":[{"p":"010","d":"2017-04-11 22-55-02"},{"p":"010","d":"2017-04-11 22-57-05"},{"p":"010","d":"2017-04-11 23-00-08"},{"p":"010","d":"2017-04-11 23-05-14"},{"p":"010","d":"2017-04-11 23-10-20"},{"p":"010","d":"2017-04-11 23-17-29"},{"p":"010","d":"2017-04-11 23-21-34"},{"p":"010","d":"2017-04-11 23-23-36"},{"p":"010","d":"2017-04-11 23-31-46"},{"p":"010","d":"2017-04-11 23-52-09"},{"p":"010","d":"2017-04-11 23-54-11"},{"p":"010","d":"2017-04-11 23-58-17"},{"p":"010","d":"2017-04-12 00-02-22"},{"p":"010","d":"2017-04-12 00-07-29"},{"p":"010","d":"2017-04-12 00-24-55"},{"p":"010","d":"2017-04-12 01-41-23"}]
    },
    "522189..":
    {
      "birth":[{"p":"010","d":"2017-04-11 22-26-25"},{"p":"010","d":"2017-04-11 22-31-33"},{"p":"010","d":"2017-04-11 22-44-48"},{"p":"010","d":"2017-04-11 22-47-52"},{"p":"010","d":"2017-04-11 22-49-54"},{"p":"010","d":"2017-04-11 22-54-01"},{"p":"010","d":"2017-04-11 22-59-07"},{"p":"010","d":"2017-04-11 23-03-11"},{"p":"010","d":"2017-04-11 23-05-14"},{"p":"010","d":"2017-04-11 23-15-26"},{"p":"010","d":"2017-04-11 23-24-37"},{"p":"010","d":"2017-04-11 23-27-41"},{"p":"010","d":"2017-04-11 23-30-44"},{"p":"010","d":"2017-04-11 23-33-48"},{"p":"010","d":"2017-04-11 23-35-50"},{"p":"010","d":"2017-04-11 23-39-54"},{"p":"010","d":"2017-04-11 23-51-08"},{"p":"010","d":"2017-04-11 23-56-14"},{"p":"010","d":"2017-04-12 00-03-23"},{"p":"010","d":"2017-04-12 00-11-34"},{"p":"010","d":"2017-04-12 00-22-53"},{"p":"010","d":"2017-04-12 00-42-42"}],
      "death":[{"p":"010","d":"2017-04-11 22-29-29"},{"p":"010","d":"2017-04-11 22-40-43"},{"p":"010","d":"2017-04-11 22-46-51"},{"p":"010","d":"2017-04-11 22-48-53"},{"p":"010","d":"2017-04-11 22-51-58"},{"p":"010","d":"2017-04-11 22-55-02"},{"p":"010","d":"2017-04-11 23-02-10"},{"p":"010","d":"2017-04-11 23-04-13"},{"p":"010","d":"2017-04-11 23-06-15"},{"p":"010","d":"2017-04-11 23-21-34"},{"p":"010","d":"2017-04-11 23-26-39"},{"p":"010","d":"2017-04-11 23-28-42"},{"p":"010","d":"2017-04-11 23-31-46"},{"p":"010","d":"2017-04-11 23-34-49"},{"p":"010","d":"2017-04-11 23-36-51"},{"p":"010","d":"2017-04-11 23-40-56"},{"p":"010","d":"2017-04-11 23-55-13"},{"p":"010","d":"2017-04-12 00-01-21"},{"p":"010","d":"2017-04-12 00-08-30"},{"p":"010","d":"2017-04-12 00-20-49"},{"p":"010","d":"2017-04-12 00-41-39"},{"p":"010","d":"2017-04-12 00-58-50"}]
    },
    "5305f5..":
    {
      "birth":[{"p":"010","d":"2017-04-11 21-57-52"},{"p":"010","d":"2017-04-11 22-31-33"},{"p":"010","d":"2017-04-11 22-38-40"},{"p":"010","d":"2017-04-12 00-03-23"},{"p":"010","d":"2017-04-12 00-12-38"},{"p":"010","d":"2017-04-12 00-25-56"},{"p":"010","d":"2017-04-12 00-27-59"}],
      "death":[{"p":"010","d":"2017-04-11 22-30-30"},{"p":"010","d":"2017-04-11 22-35-36"},{"p":"010","d":"2017-04-11 22-40-43"},{"p":"010","d":"2017-04-12 00-04-24"},{"p":"010","d":"2017-04-12 00-13-40"},{"p":"010","d":"2017-04-12 00-26-58"},{"p":"010","d":"2017-04-12 00-29-01"}]
    },
    "53d215..":
    {
      "birth":[{"p":"010","d":"2017-04-11 21-48-31"},{"p":"010","d":"2017-04-11 22-31-33"},{"p":"010","d":"2017-04-11 22-38-40"},{"p":"010","d":"2017-04-11 22-40-43"},{"p":"010","d":"2017-04-11 22-50-57"},{"p":"010","d":"2017-04-11 22-55-02"},{"p":"010","d":"2017-04-11 22-59-07"},{"p":"010","d":"2017-04-11 23-09-19"},{"p":"010","d":"2017-04-11 23-13-24"},{"p":"010","d":"2017-04-11 23-18-30"},{"p":"010","d":"2017-04-12 00-03-23"},{"p":"010","d":"2017-04-12 00-12-38"}],
      "death":[{"p":"010","d":"2017-04-11 22-30-30"},{"p":"010","d":"2017-04-11 22-37-39"},{"p":"010","d":"2017-04-11 22-39-41"},{"p":"010","d":"2017-04-11 22-43-47"},{"p":"010","d":"2017-04-11 22-51-58"},{"p":"010","d":"2017-04-11 22-56-04"},{"p":"010","d":"2017-04-11 23-06-15"},{"p":"010","d":"2017-04-11 23-10-20"},{"p":"010","d":"2017-04-11 23-16-27"},{"p":"010","d":"2017-04-11 23-20-32"},{"p":"010","d":"2017-04-12 00-04-24"},{"p":"010","d":"2017-04-12 00-13-40"}]
    },
    "53d3fd..":
    {
      "birth":[{"p":"010","d":"2017-04-11 21-24-09"},{"p":"010","d":"2017-04-11 22-44-48"},{"p":"010","d":"2017-04-11 22-48-53"},{"p":"010","d":"2017-04-11 22-51-58"},{"p":"010","d":"2017-04-11 22-54-01"},{"p":"010","d":"2017-04-11 23-00-08"},{"p":"010","d":"2017-04-11 23-09-19"},{"p":"010","d":"2017-04-11 23-14-25"},{"p":"010","d":"2017-04-11 23-24-37"},{"p":"010","d":"2017-04-11 23-32-47"},{"p":"010","d":"2017-04-11 23-36-51"},{"p":"010","d":"2017-04-11 23-39-54"},{"p":"010","d":"2017-04-11 23-46-03"},{"p":"010","d":"2017-04-11 23-51-08"},{"p":"010","d":"2017-04-11 23-57-16"},{"p":"010","d":"2017-04-12 00-02-22"},{"p":"010","d":"2017-04-12 00-05-26"},{"p":"010","d":"2017-04-12 00-12-38"},{"p":"010","d":"2017-04-12 00-23-53"},{"p":"010","d":"2017-04-12 00-25-56"}],
      "death":[{"p":"010","d":"2017-04-11 22-41-44"},{"p":"010","d":"2017-04-11 22-46-51"},{"p":"010","d":"2017-04-11 22-49-54"},{"p":"010","d":"2017-04-11 22-53-00"},{"p":"010","d":"2017-04-11 22-55-02"},{"p":"010","d":"2017-04-11 23-06-15"},{"p":"010","d":"2017-04-11 23-12-23"},{"p":"010","d":"2017-04-11 23-20-32"},{"p":"010","d":"2017-04-11 23-25-39"},{"p":"010","d":"2017-04-11 23-34-49"},{"p":"010","d":"2017-04-11 23-37-53"},{"p":"010","d":"2017-04-11 23-40-56"},{"p":"010","d":"2017-04-11 23-47-03"},{"p":"010","d":"2017-04-11 23-55-13"},{"p":"010","d":"2017-04-12 00-01-21"},{"p":"010","d":"2017-04-12 00-04-24"},{"p":"010","d":"2017-04-12 00-11-34"},{"p":"010","d":"2017-04-12 00-20-49"},{"p":"010","d":"2017-04-12 00-24-55"},{"p":"010","d":"2017-04-12 02-15-35"}]
    },
    "5410da..":
    {
      "birth":[{"p":"010","d":"2017-04-12 00-46-58"},{"p":"010","d":"2017-04-12 01-03-10"},{"p":"010","d":"2017-04-12 01-42-43"},{"p":"010","d":"2017-04-12 01-44-58"}],
      "death":[{"p":"010","d":"2017-04-12 01-01-58"},{"p":"010","d":"2017-04-12 01-41-23"},{"p":"010","d":"2017-04-12 01-43-45"},{"p":"010","d":"2017-04-12 02-07-02"}]
    },
    "5413fd..":
    {
      "birth":[{"p":"010","d":"2017-04-12 00-42-42"},{"p":"010","d":"2017-04-12 01-03-10"}],
      "death":[{"p":"010","d":"2017-04-12 01-01-58"},{"p":"010","d":"2017-04-12 01-22-05"}]
    },
    "567bb3..":
    {
      "birth":[{"p":"010","d":"2017-04-11 21-29-42"},{"p":"010","d":"2017-04-11 22-36-38"},{"p":"010","d":"2017-04-11 22-40-43"},{"p":"010","d":"2017-04-11 22-46-51"},{"p":"010","d":"2017-04-11 22-50-57"},{"p":"010","d":"2017-04-11 22-54-01"},{"p":"010","d":"2017-04-11 22-56-04"},{"p":"010","d":"2017-04-11 22-59-07"},{"p":"010","d":"2017-04-11 23-08-18"},{"p":"010","d":"2017-04-11 23-11-21"},{"p":"010","d":"2017-04-11 23-14-25"},{"p":"010","d":"2017-04-11 23-17-29"},{"p":"010","d":"2017-04-11 23-23-36"},{"p":"010","d":"2017-04-11 23-31-46"},{"p":"010","d":"2017-04-11 23-37-53"},{"p":"010","d":"2017-04-11 23-41-57"},{"p":"010","d":"2017-04-11 23-43-59"},{"p":"010","d":"2017-04-11 23-46-03"},{"p":"010","d":"2017-04-11 23-48-05"},{"p":"010","d":"2017-04-11 23-51-08"}],
      "death":[{"p":"010","d":"2017-04-11 22-35-36"},{"p":"010","d":"2017-04-11 22-37-39"},{"p":"010","d":"2017-04-11 22-44-48"},{"p":"010","d":"2017-04-11 22-47-52"},{"p":"010","d":"2017-04-11 22-53-00"},{"p":"010","d":"2017-04-11 22-55-02"},{"p":"010","d":"2017-04-11 22-57-05"},{"p":"010","d":"2017-04-11 23-07-16"},{"p":"010","d":"2017-04-11 23-09-19"},{"p":"010","d":"2017-04-11 23-13-24"},{"p":"010","d":"2017-04-11 23-15-26"},{"p":"010","d":"2017-04-11 23-21-34"},{"p":"010","d":"2017-04-11 23-29-43"},{"p":"010","d":"2017-04-11 23-36-51"},{"p":"010","d":"2017-04-11 23-40-56"},{"p":"010","d":"2017-04-11 23-42-58"},{"p":"010","d":"2017-04-11 23-45-02"},{"p":"010","d":"2017-04-11 23-47-03"},{"p":"010","d":"2017-04-11 23-50-07"},{"p":"010","d":"2017-04-11 23-52-09"}]
    },
    "56d73d..":
    {
      "birth":[{"p":"010","d":"2017-04-11 23-04-13"},{"p":"010","d":"2017-04-11 23-07-16"},{"p":"010","d":"2017-04-11 23-10-20"},{"p":"010","d":"2017-04-11 23-14-25"},{"p":"010","d":"2017-04-11 23-24-37"},{"p":"010","d":"2017-04-11 23-30-44"},{"p":"010","d":"2017-04-11 23-32-47"},{"p":"010","d":"2017-04-11 23-37-53"},{"p":"010","d":"2017-04-11 23-46-03"},{"p":"010","d":"2017-04-11 23-51-08"},{"p":"010","d":"2017-04-11 23-58-17"},{"p":"010","d":"2017-04-12 00-03-23"},{"p":"010","d":"2017-04-12 00-50-16"},{"p":"010","d":"2017-04-12 01-01-58"}],
      "death":[{"p":"010","d":"2017-04-11 23-06-15"},{"p":"010","d":"2017-04-11 23-08-18"},{"p":"010","d":"2017-04-11 23-11-21"},{"p":"010","d":"2017-04-11 23-22-35"},{"p":"010","d":"2017-04-11 23-27-41"},{"p":"010","d":"2017-04-11 23-31-46"},{"p":"010","d":"2017-04-11 23-35-50"},{"p":"010","d":"2017-04-11 23-39-54"},{"p":"010","d":"2017-04-11 23-48-05"},{"p":"010","d":"2017-04-11 23-56-14"},{"p":"010","d":"2017-04-12 00-02-22"},{"p":"010","d":"2017-04-12 00-49-14"},{"p":"010","d":"2017-04-12 01-00-54"},{"p":"010","d":"2017-04-12 01-49-19"}]
    },
    "9b2818..":
    {
      "birth":[{"p":"100","d":"2017-04-08 12-51-53"},{"p":"100","d":"2017-04-10 21-03-17"},{"p":"100","d":"2017-04-10 21-10-49"}],
      "death":[{"p":"100","d":"2017-04-10 21-02-14"},{"p":"100","d":"2017-04-10 21-09-42"},{"p":"100","d":"2017-04-11 15-49-01"}]
    },
    "c008ea..":
    {
      "birth":[{"p":"110","d":"2017-04-08 18-45-09"},{"p":"110","d":"2017-04-08 18-49-58"}],
      "death":[{"p":"110","d":"2017-04-08 18-46-11"},{"p":"110","d":"2017-04-08 20-22-02"}]
    },
    "c26ebf..":
    {
      "birth":[{"p":"110","d":"2017-04-08 20-14-58"},{"p":"110","d":"2017-04-08 20-19-58"},{"p":"110","d":"2017-04-08 20-22-02"},{"p":"110","d":"2017-04-08 20-26-15"},{"p":"110","d":"2017-04-08 20-32-59"},{"p":"110","d":"2017-04-08 20-40-59"},{"p":"110","d":"2017-04-08 20-43-21"},{"p":"110","d":"2017-04-08 20-46-00"},{"p":"110","d":"2017-04-08 20-50-59"}],
      "death":[{"p":"110","d":"2017-04-08 20-16-00"},{"p":"110","d":"2017-04-08 20-20-59"},{"p":"110","d":"2017-04-08 20-23-03"},{"p":"110","d":"2017-04-08 20-29-47"},{"p":"110","d":"2017-04-08 20-39-16"},{"p":"110","d":"2017-04-08 20-42-15"},{"p":"110","d":"2017-04-08 20-44-30"},{"p":"110","d":"2017-04-08 20-47-15"},{"p":"110","d":"2017-04-08 20-52-12"}]
    },
    "c273ee..":
    {
      "birth":[{"p":"110","d":"2017-04-08 16-03-55"},{"p":"110","d":"2017-04-08 17-02-07"}],
      "death":[{"p":"110","d":"2017-04-08 17-01-06"},{"p":"110","d":"2017-04-08 18-59-49"}]
    },
    "c2b336..":
    {
      "birth":[{"p":"110","d":"2017-04-08 19-26-07"},{"p":"110","d":"2017-04-08 19-37-13"},{"p":"110","d":"2017-04-08 19-43-21"},{"p":"110","d":"2017-04-08 19-45-58"},{"p":"110","d":"2017-04-08 19-48-58"},{"p":"110","d":"2017-04-08 19-54-15"}],
      "death":[{"p":"110","d":"2017-04-08 19-36-07"},{"p":"110","d":"2017-04-08 19-38-58"},{"p":"110","d":"2017-04-08 19-44-23"},{"p":"110","d":"2017-04-08 19-47-13"},{"p":"110","d":"2017-04-08 19-51-01"},{"p":"110","d":"2017-04-08 19-56-21"}]
    },
    "c3287b..":
    {
      "birth":[{"p":"110","d":"2017-04-08 16-37-07"},{"p":"110","d":"2017-04-08 16-39-57"},{"p":"110","d":"2017-04-08 19-11-32"},{"p":"110","d":"2017-04-08 19-15-06"}],
      "death":[{"p":"110","d":"2017-04-08 16-38-11"},{"p":"110","d":"2017-04-08 16-41-00"},{"p":"110","d":"2017-04-08 19-13-58"},{"p":"110","d":"2017-04-08 19-37-13"}]
    },
    "c3e5cd..":
    {
      "birth":[{"p":"110","d":"2017-04-08 19-11-32"},{"p":"110","d":"2017-04-08 19-18-18"},{"p":"110","d":"2017-04-08 19-22-58"}],
      "death":[{"p":"110","d":"2017-04-08 19-13-58"},{"p":"110","d":"2017-04-08 19-21-24"},{"p":"110","d":"2017-04-08 20-12-13"}]
    },
    "c3f46e..":
    {
      "birth":[{"p":"110","d":"2017-04-08 16-37-07"},{"p":"110","d":"2017-04-08 16-39-57"},{"p":"110","d":"2017-04-08 19-07-13"},{"p":"110","d":"2017-04-08 19-10-25"}],
      "death":[{"p":"110","d":"2017-04-08 16-38-11"},{"p":"110","d":"2017-04-08 16-41-00"},{"p":"110","d":"2017-04-08 19-08-22"},{"p":"110","d":"2017-04-08 19-25-06"}]
    },
    "e33b56..":
    {
      "birth":[{"p":"111","d":"2017-04-07 00-25-25"},{"p":"111","d":"2017-04-07 01-45-16"}],
      "death":[{"p":"111","d":"2017-04-07 01-44-13"},{"p":"111","d":"2017-04-07 02-26-12"}]
    }
  }
}
4 Likes
2

Thanks for this, I had not noticed it on the user forum. I will ask the production team to look into this, @AndreasF and @Fraser with @Qi_Ma will be very interested though for sure.

Thanks again.

3 Likes
3

Sorry, I had missed that in the other forum! Thanks for providing such detailed information!

If your node disconnects from another node, it will remove it from its routing table but at the same time try to reconnect, in case the node just had network issues and isn’t actually offline. If reconnecting still fails for some reason even if that node is still connected to others, your node will give up for now. But it can happen that later, in a SectionUpdate message or during a merge, your section informs your node again about the one you considered lost, and you will connect to it again. So I don’t think it was a case of nodes leaving the network and then rejoining with the same name.

Most of the examples look like that, but some of them look like they retried for ages and always got disconnected again and we’d like to look into that next week. Do you still have the logs about node 36a94f…? It would be great if we could have a look at one such connect-disconnect iteration, e.g. the full log of your node from 2017-04-09 22-19-00 to 2017-04-09 22-20-02.

Also, you wrote that node 4bf2e9… was in section 0011—or was that just a typo?

5 Likes
4

Yes, it was a typo. “4bf2e9…” had resurrections problem (it reappeared 5 times) but it was in section “0100”. Source information is in json part of OP which lists all nodes having more than 2 birth dates.

I don’t have “36a94f…” log, nor any log of problematic nodes. The only logs I have are for nodes “bb922f…”, “96260a…” and “1c4562…” which “properly” died (meaning without any subsequent resurrections).

All the information I have about other nodes (node ids and section prefixes) were obtained with MessageContent::RoutingTableRequest message sent initially from “96260a…” and later from “1c4562…”.

I have identified 39 nodes having resurrections problem listed in json part of OP, namely:

"143156.."
"14dbba.."
"15b812.."
"1f2df8.."
"26c398.."
"3022d9.."
"3101b5.."
"3271a1.."
"32cc41.."
"336045.."
"3453d0.."
"34badc.."
"3512ee.."
"353d82.."
"35e5f4.."
"36a94f.."
"37bdf9.."
"4bf2e9.."
"5086b9.."
"515b32.."
"519d35.."
"51be37.."
"522189.."
"5305f5.."
"53d215.."
"53d3fd.."
"5410da.."
"5413fd.."
"567bb3.."
"56d73d.."
"9b2818.."
"c008ea.."
"c26ebf.."
"c273ee.."
"c2b336.."
"c3287b.."
"c3e5cd.."
"c3f46e.."
"e34b56.."

If Maidsafe doesn’t have the log of any of them, then a call to the community could be launched to try to get some of them.

2 Likes
5

Thanks!
We believe that this issue has been fixed in PR 1437.

2 Likes