repository msafe/vault not found


#1

From https://github.com/maidsafe/safe_vault/blob/master/installer/docker/README.md

Safe Vault image

Safe Vault is available as a docker image called msafe/vault on Docker Hub. Alternatively, it’s possible to build the image using the Dockerfile found in the root safe_vault directory.

To pull the latest Safe Vault image from Docker Hub:

docker pull msafe/vault

but …
willie@gagarin:~/projects/maidsafe$ docker pull msafe/vault
Using default tag: latest
Error response from daemon: manifest for msafe/vault:latest not found
willie@gagarin:~/projects/maidsafe$ docker pull -a msafe/vault
Error response from daemon: repository msafe/vault not found

```

#2

@southside - the guys are investigating right now…
update to come.


#3
Using default tag: latest
Error response from daemon: manifest for msafe/vault:latest not found

Yes, I’m getting the same issue - looking into it to see if I can establish cause / resolution…


#4

OK, I’ve worked out what’s going on - the msafe/vault image on Dockerhub is basically out of date and unmaintained so there’s nothing with the tag “latest” for Docker to pull.

I’ve managed to build the Docker image locally and try and push it to Dockerhub. I’ll post here to let you know how I get on - most likely the readme for the Safe Vault will need updated too.


#5

That’s pretty much what I suspected. Vaults are somewhat of a niche interest at this particular stage and it is unsurprising that effort has been elsewhere.
Thank you for a quick investigation and report.


#6

Happy to help. We have another (maintained) Dockerhub repo so I’ve actually pushed my locally built image there. If you do:

docker pull maidsafe/vault

you should get it. (Haven’t actually tried running it though…!)


#7

I tried. Havent looked at it too hard though - Busy with other stuff today

willie@gagarin:~/projects/maidsafe$ docker images
REPOSITORY                        TAG                    IMAGE ID            CREATED             SIZE
maidsafe/vault                    latest                 0bd0645af070        2 hours ago         436MB
willie@gagarin:~/projects/maidsafe$ docker run -i -t 0bd0645af070 /bin/bash
    Updating crates.io index
    Updating git repository `https://github.com/povilasb/routing`                                                                                                            
    Updating git repository `https://github.com/povilasb/crust`                                                                                                              
    Updating crates.io index                                                                                                                                                 
    Updating git repository `https://github.com/maidsafe/socket-collection`                                                                                                  
error: failed to select a version for `log`.                                                                                                                                 
    ... required by package `crust v0.31.0 (https://github.com/povilasb/crust?rev=testing#6a94e3ca)`
    ... which is depended on by `routing v0.37.0 (https://github.com/povilasb/routing?rev=update-crust#9c1a73b9)`
    ... which is depended on by `safe_vault v0.18.0 (/home/maidsafe/src/safe_vault)`
versions that meet the requirements `~0.4.6` are: 0.4.6

all possible versions conflict with previously selected packages.

  previously selected package `log v0.4.1`
    ... which is depended on by `routing v0.37.0 (https://github.com/povilasb/routing?rev=update-crust#9c1a73b9)`
    ... which is depended on by `safe_vault v0.18.0 (/home/maidsafe/src/safe_vault)`

failed to select a version for `log` which could resolve this conflict
./installer/docker/docker-entrypoint.sh: line 333: ./safe_vault: No such file or directory

Simply reporting the initial attempt - I’ll look at it more closely tonight if I get time. As expected it may need some TLC. Still, we move forward, thank you :grin:


#8

@southside I was just messaging you there! Yeah, I’m having that issue with the docker image too. Think it was something I did though so have for the meantime deleted the image from dockerhub.

When I’ve figured it out I will push a working image back up to dockerhub. If you want / have time you could try the local build option. If you do please let me know how you get on, thanks.


#9

Yes I’ll have a shot a building it local tonight and report back.


#10

This is what happened when I tried to build locally from the Dockerfile

I git cloned into a new fresh dir
willie@gagarin:~/projects/maidsafe/dockerstuff/safe_vault$ docker build -f Dockerfile .

completed without error. Checked the image id…

`willie@gagarin:~/projects/maidsafe/dockerstuff/safe_vault$ docker images`

REPOSITORY TAG IMAGE ID CREATED SIZE
3aead9538c8e 20 seconds ago 436MB
alpine latest caf27325b298 3 weeks ago 5.53MB

then ran the image…

`willie@gagarin:~/projects/maidsafe/dockerstuff/safe_vault$ docker run -i -t 3aead9538c8e /bin/bash`
Updating crates.io index
Downloaded itertools v0.7.8                                                                                                                                                
      Downloaded serde_json v1.0.13                                                                                                                                              
      Downloaded routing v0.37.0                                                                                                                                                 
      Downloaded tempdir v0.3.7                                                                                                                                                  

------ SNIP -some time passed many lines scrolled by with no error -------

   Compiling futures-cpupool v0.1.8                                                                                                                                          
   Compiling msdos_time v0.1.5                                                                                                                                               
   Compiling tar v0.4.16                                                                                                                                                     
   Compiling textwrap v0.6.0                                                                                                                                                 
   Compiling aho-corasick v0.6.4                                                                                                                                             
   Compiling tokio-timer v0.2.1                                                                                                                                              
error: failed to run custom build command for `openssl-sys v0.9.35`                                                                                                          
process didn't exit successfully: `/home/maidsafe/src/safe_vault/target/release/build/openssl-sys-29624bec371f6aa0/build-script-main` (exit code: 101)
--- stdout
cargo:rerun-if-env-changed=X86_64_ALPINE_LINUX_MUSL_OPENSSL_LIB_DIR
cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
cargo:rerun-if-env-changed=X86_64_ALPINE_LINUX_MUSL_OPENSSL_INCLUDE_DIR
cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
cargo:rerun-if-env-changed=X86_64_ALPINE_LINUX_MUSL_OPENSSL_DIR
cargo:rerun-if-env-changed=OPENSSL_DIR
run pkg_config fail: "`\"pkg-config\" \"--libs\" \"--cflags\" \"openssl\"` did not exit successfully: exit code: 1\n--- stderr\nPackage openssl was not found in the pkg-config search path.\nPerhaps you should add the directory containing `openssl.pc\'\nto the PKG_CONFIG_PATH environment variable\nPackage \'openssl\', required by \'virtual:world\', not found\n"

--- stderr
thread 'main' panicked at '

Could not find directory of OpenSSL installation, and this `-sys` crate cannot
proceed without this knowledge. If OpenSSL is installed and this crate had
trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
compilation process.

Make sure you also have the development packages of openssl installed.
For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

If you're in a situation where you think the directory *should* be found
automatically, please open a bug at https://github.com/sfackler/rust-openssl
and include information about your system as well as this message.

$HOST = x86_64-alpine-linux-musl
$TARGET = x86_64-alpine-linux-musl
openssl-sys = 0.9.35

', /home/maidsafe/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-sys-0.9.35/build/main.rs:265:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.

warning: build failed, waiting for other jobs to finish...
error: build failed                                                                                                                                                          
./installer/docker/docker-entrypoint.sh: line 333: ./safe_vault: No such file or directory

I did some checking as below and I think the libssl-dev package should be found automatically so I’m going to file a bug as advised above on sfackler/rust-openssl.

willie@gagarin:~/projects/maidsafe/dockerstuff/safe_vault$ openssl version -d
OPENSSLDIR: "/usr/lib/ssl"
willie@gagarin:~/projects/maidsafe/dockerstuff/safe_vault$ echo $OPENSSL_DIR
/usr/lib/ssl
willie@gagarin:~/projects/maidsafe/dockerstuff/safe_vault$ echo $PKG_CONFIG_PATH
/usr/lib/x86_64-linux-gnu/pkgconfig/
willie@gagarin:~/projects/maidsafe/dockerstuff/safe_vault$ locate openssl.pc
/usr/lib/x86_64-linux-gnu/pkgconfig/openssl.pc
willie@gagarin:~/projects/maidsafe/dockerstuff/safe_vault$ openssl version
OpenSSL 1.1.0g  2 Nov 2017
willie@gagarin:~/projects/maidsafe/dockerstuff/safe_vault$ sudo apt-get install pkg-config libssl-dev
[sudo] password for willie: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
pkg-config is already the newest version (0.29.1-0ubuntu2).
pkg-config set to manually installed.
libssl-dev is already the newest version (1.1.0g-2ubuntu4.3).
0 to upgrade, 0 to newly install, 0 to remove and 0 not to upgrade.

#11

Yeah, that looks like the same issue we got when we tried to run (after rebuilding it locally). I will try and find a way to fix - will let you know if / when I do.


#12

BTW, it’s probably not going to be today - we have an upcoming SAFE Browser release I need to test…


#13

Absolutely no pronlem. I have new multiple conflicting work/personal priorities fro next few days so I won’t have time and Im certainly not pushing.

And the browser is far more important at this time anyway.

I have opened an issue at https://github.com/sfackler/rust-openssl/issues/1065sfackler/ and it is receiving attention. I’ll keep you up to date


#14

Hi again.

I’ve spent a wee bit more time on this issue today (have commented on the issue you raised on GitHub BTW but I don’t think we’re going to be able to resolve simply as I will expand on).

As noted, the root issue stems from the absence of the openssl-dev package when we run the docker image. I thought at first a quick fix might be to add an install command to Dockerfile. So, I tried adding the line
apk openssl-dev
to the part of the Dockerfile that adds packages. This didn’t work however…

After speaking to Chris our devops guy, it seems the issue is probably more deeply routed than a simple missing package.

This docker image bases from the Alpine image, a very minimal version of Linux. The failure occurs because when the container runs, it attempts to build safe_vault if a pre-built binary hasn’t already been supplied. The problem is that Alpine isn’t a really good OS for a build environment, so it’s not a one-line change to simply install the missing package it’s complaining about, as it doesn’t have said package available.

We might want to refactor it into 2 different docker images: one for building safe_vault and one for deploying a pre-built safe binary. We would need to take some time to think about exactly how we would go about this.

So, basically we are building a docker container which then, when run, itself builds the safe_vault. A better solution would be to build the safe_vault initially and then supply that to docker when we build the container. In that case when we run the container, we would be deploying a pre made safe_vault.

Unfortunately at this moment in time our docker container doesn’t do this and to get it to a position where it does would require some work. I’m afraid it isn’t high priority right now so I really don’t know if / when it might happen…

If you are up for a docker learning “challenge” you could try it yourself maybe?

Sorry we haven’t been able to resolve this more satisfactorily.


#15

Exactly! You build the image in 2 steps: First you compile the source in a rust image and then you copy the generated executable to the final image.

This is a common technique, and I have used it to build the image for the network community (tsafe/vault3 on docker hub). Here is the Dockerfile to build it.

# Clone repo and build application
# Note: rust image contains git
FROM rust:1.30.1 as build

# Get Vault source files from github
WORKDIR /usr/src
RUN git clone https://github.com/Thierry61/safe_vault.git

# Build application
WORKDIR /usr/src/safe_vault
RUN cargo build --release

# Small final image
FROM ubuntu:latest
WORKDIR /app

# Don't copy other things than final executable, notably not build (132MB) nor deps (490MB)
COPY --from=build /usr/src/safe_vault/target/release/safe_vault .

# Copy config files
COPY log.yml *.config ./

# Launch vault. For the first one, environment variable vault_args must be set to -f with: -e "vault_args=-f"
CMD ./safe_vault $vault_args

There are many things you can do based on this:

  • Use a more recent version of rust
  • Use Maidsafe safe_vault repo instead of mine
  • Put your own config files in context directory (mine were adapted to the community network)
  • Replace log.yml with log.toml
  • Use Alpine instead of Ubuntu for a smaller final image size (It’s just that I know Ubuntu and not Alpine and 100 MB is not that big)

Edit: For illustration, I have just rebuilt my safe_vault image for the community network (with a new contact list) and the resulting sizes are:

PS C:\Users\212529844\Documents\Docker\tsafe\vault3> docker images
REPOSITORY                  TAG                 IMAGE ID            CREATED              SIZE
tsafe/vault3                3.0                 3d5d924423ed        About a minute ago   100MB
<none>                      <none>              fba71e7caf34        About a minute ago   2.15GB

The builder image is 2.15GB but the final image is only 100MB.


#16

Thanks for the input! Will take that on board when we get the chance to tackle this issue more thoroughly.