Nix reproducible build and developer sandbox

Has anyone used Nix or something similar?

I think cargo, rustup, nvm etc make things much easier in terms of getting build environments standardised, but this looks like another level.

I guess the biggest limitation with Nix is that it appears to be Linux only. Any thoughts?

2 Likes

Yes, I’ve tried nix and guix, mainly out of interest of deterministic builds for safe network (see this topic). So far neither have really seemed like ‘it’ to me. You’ll see from the link that bitcoin uses or is interested in using guix, I think they mainly rely on gitian at the moment)

I’ve tried chroot and firejail but neither really stuck for me, I’ve tried virtualbox and qubes-os but both were a little heavy handed (I really liked qubes but the overhead is still a bit much to swallow), I’ve not yet tried containers/docker though. All the benefits I was looking for from these things could be had by simply ‘knowing my tools’, and I’ve been able to run everything locally without conflict. All these extra tools didn’t justify their presence, but that’s obviously just a personal preference and I can see how they’re all still fantastic tools.

My main feeling on reproducible builds is that content-addressable libs packages and dependencies is going to be pretty handy to have. Nix has been using the content-addressable feature of IPFS (article) so it seems natural that this will also be utilized on Safe Network too.

2 Likes

Thanks @mav, I remember your interest in reproducible builds. Later I read a Twitter thread by a guy who was arguing that everything people say they want reproducible builds for is invalid. Have you seen that? I read the thread and think he’d written longer blogs about it, and it seemed convincing although I didn’t study it.

When I read about Nix I remembered all the time I spent trying to get one build environment or another working over the last few years and thought that it could be a good way to help others avoid that pain.

I hadn’t thought about this, very interesting, thanks for that. I suspect there’s a lot of things Safe CAFS will be good for that I haven’t anticipated!

1 Like

No I haven’t, if you have a link to it I’d be keen to read.

Yeah I feel you on this, although much less so lately (I’ve probably experienced the pain of complex dev envs too much and have shaped my dev style around using the simplest possible environments).

What I’ve been doing lately is keeping a bash script that records the steps I take when setting up my environment. I guess the eventual hope for me is to do all my dev in a fresh debian vm, and be able to run dev_env_setup.sh from the repo I’m working on, then at the end of coding session push code changes and destroy the vm. This ensures the script is run often so is kept up to date. But… I was hoping to use something lighter than VMs. I have the scripts but they act mainly as a kind of note taking rather than a tool I actually use in practice.

My main motivation for taking this direction is I really do not want to run nodejs locally (I can’t even justify it rationally so don’t ask, it’s a visceral thing) so I decided to begin taking steps toward only developing in ‘temporary’ dev environments. Not there yet because it’s less work to do everything locally like I am now, but that workflow lends itself to the idea of nix style deterministic environments. This is also a very normal and simple workflow in qubes, but like my feeling on VMs it feels a little heavy and the overhead is hard to justify.

On Nix, I’m thinking more about making it easier for anyone to build stuff rather than to make my life easier. Like you I’m mostly over the pain, but it still wastes time. If I could just run a Nix config, or publish it for anyone wanting to build my stuff it could get more people involved and save them a lot of time. Like you I keep a note (but manually) whenever I build something new to make it easier to share our repeat the steps needed.

In the need for reproducible builds, I’m pretty sure this is the guy, although I only read his tweet dialog at the time:

And here’s a HN thread which I’ve not read either!

If you read I’d be interested to know if you agree or disagree on any important uses.

1 Like

The article has some really good points.

To me reproducible builds is about setting an expectation for the degree of communication and quality of the software. A reproducible build means there must be a certain build process specified, it must be communicated in a useful way, the benefit vs cost must be considered.

That’s a bit of a soft answer, and those things can be achieved without reproducible builds. But an expectation of reproducible builds sort of ‘enforces’ a certain degree of this, whereas documentation alone or bash scripts alone can drift and decay with no clear indication of how much drift has actually happened.

I feel like the degrees of benefit are ignored (it’s not a binary yes benefit or no benefit). It’s better to be able to have ten separate individual builds which conform than a single one. Even if those ten could be colluding for malicious purposes or whatever, it seems like the degree of benefit is higher than an ad-hoc build.

I also feel like the case they present where ‘build it yourself means the external reproducible build becomes redundant’ is incorrect. Doing the build process yourself adds a data point of trust for the other people who have also done the build (and shared their results). A reproducible build encourages a group experience of the build process, whether doing the build or simply consuming the outputs as an end user. I feel there’s value in the social dynamic enabled by reproducible builds that can’t as easily be had with ad-hoc builds.

But, overall, I really appreciate the arguments put forward against reproducible builds. Thanks for the link.

1 Like

You’re welcome @mav, and make useful valid points. A lot flows from the nuance, so good for me to look out for that in other contexts to. Thanks very much.

1 Like