The tale of two *ixs: The Nix and Guix overview

Many people complain about the accesibily of declarative package management tools, namely Guix and Nix. The problem with sources for the former is a certain assumption of scheme knowledge and with the latter the fragmentation of documentation.

Usually people explain Guix/Nix by example, for that I’m afraid you will have to look elsewhere. I am here to show you the theoretical overview. Hopefully at the end of this article you will be equipped to understand Nix and Guix and how they relate to eachother. Even if you are only interested in one of the two, I implore you to read the whole article, unless you get very bored and want to stop reading altogether, as I have written it not with Nix/Guix only reading in mind.

Since this article will refer to Guix and Nix a lot, allow me to refer to them collectively as *ix.

What is the store: Wait it’s all hashmaps?

First thing a person using *ix hears about is the store. What is it? Can I buy my drinks there? The store is very similar to what git does.

The store does not know what packages are, but it can lookup stuff by an unique identifier, the store is a hashmap. The only difference is that it is on the filesystem. For the store, there is nothing anywhere else in the system, just this one hashmap. If we ever want to access anything else? We need to first put it in the store.

Usually we have the store in /*ix/store inside the filesystem.

Deriving a derivation: This literally does not tell me anything

Okay so we got a hashmap, but how do we get from A (a package source) to B (a package binary)? Well my dear reader, that is where derivations come into play. Derivations are recipies, they take dependencies, and a build script a and describe the output.

For this we will take the gnu hello package. It is a simple hello world package made for showing us how packages are built.

Note that I have shortened the store paths by replacing the hashes with ... and formatted it manually, derivations are usually not this pretty.

Derive
(
    ## output
    [
        ("out","/gnu/store/...-hello-2.12.2","","")
    ],
    ## inputs
    [
        ("/gnu/store/...-patch-2.7.6.drv" ,["out"]),
        ("/gnu/store/...-module-import-compiled.drv",["out"]),
        ("/gnu/store/...-coreutils-9.1.drv",["out"]),
        ("/gnu/store/...-zstd-1.5.6.drv",["out"]),
        ("/gnu/store/...-make-4.4.1.drv",["out"]),
        ("/gnu/store/...-file-5.46.drv",["out"]),
        ("/gnu/store/...-xz-5.4.5.drv",["out"]),
        ("/gnu/store/...-gzip-1.14.drv",["out"]),
        ("/gnu/store/...-glibc-2.41.drv",["out","static"]),
        ("/gnu/store/...-guile-3.0.9.drv",["out"]),
        ("/gnu/store/...-tar-1.35.drv",["out"]),
        ("/gnu/store/...-linux-libre-headers-6.12.17.drv",["out"]),
        ("/gnu/store/...-findutils-4.10.0.drv",["out"]),
        ("/gnu/store/...-gawk-5.3.0.drv",["out"]),
        ("/gnu/store/...-gcc-14.3.0.drv",["out"]),
        ("/gnu/store/...-sed-4.9.drv",["out"]),
        ("/gnu/store/...-diffutils-3.12.drv",["out"]),
        ("/gnu/store/...-binutils-2.44.drv",["out"]),
        ("/gnu/store/...-bash-minimal-5.2.37.drv",["out"]),
        ("/gnu/store/...-grep-3.11.drv",["out"]),
        ("/gnu/store/...-ld-wrapper-0.drv",["out"]),
        ("/gnu/store/...-bzip2-1.0.8.drv",["out"]),
        ("/gnu/store/...-hello-2.12.2.tar.gz.drv",["out"])
    ],
    ## builder dependencies
    ["/gnu/store/...-module-import","/gnu/store/...-hello-2.12.2-builder"],
    ## platform
    "x86_64-linux",
    ## interpreter
    "/gnu/store/...-guile-3.0.9/bin/guile",
    ## args
    [
        "--no-auto-compile","-L",
        "/gnu/store/...-module-import","-C",
        "/gnu/store/...-module-import-compiled",
        "/gnu/store/...-hello-2.12.2-builder"
    ],
    ## env variables
    [
        ("LC_CTYPE","C.UTF-8"),
        ("out","/gnu/store/...-hello-2.12.2")
    ]
)

So first thing to note that we actually cannot produce gnu hello only from it’s source code. We need other stuff to build it. The first obvious thing is gcc, the C compiler, but also other build tools. To specify dependencies we do not directly refer to the build dependencies, we would simply loose the information on how were they produced. We instead refer to their derivations.

The first argument to our derivation is the name of the output store item, our compiled package. And as you remember the store only works with store items so we can’t produce something outside the store.

You can see “out” written around every derivation reference, this is the output specification, as derivations can produce multiple outputs, but every derivation produces at least “out”.

Then we take all the dependencies and all the builder dependencies. This is a Guix build, so the builder is actually guile code. In Nix it would be a shell script. But there is nothing stopping us from using a different builder.We can also specify the platform and the interpreter, it’s arguments and environment variables passed to the process.

Now that we have the derivation, we can build it. But oh wait! What if someone wanted to mess with our store, by changing the contents of a derivation or the dependency? And how will we know what to build first? And rust-forbin, we have a race condition?

This is where the daemon comes in. The daemon is process that has the sole ownership over the store. We can still access the store items, but we can’t change them.

Any time we want to run an operation on the store, we ask the daemon to perform it on our behalf. Usually this is done through some RPC. Which means that technically we can just detach the whole store with the daemon and put there somewhere else, even on the internet. And that is correct, there are ways to build packages remotely, and thats one of the advantages of *ix.

As you may have noticed, the derivation outputs are cached between multiple calls, so that we don’t have to build the same gcc over and over. In that regard there is a small issue. What if our binary depends on the current time or something on disk? We may never know. So that’s why some care must be taken in order to never build non-deterministic derivations. If you ever wondered why is every store item from January 1st 1970, thats why.

Being lazy with derivations: Converting packages to derivations

Okay so we can build a derivation easily, but writing them is another beast. I don’t know about you, but I do not wish to anyone writing hashes of store items by hand every time. And that is why in *ix we don’t have to do that. There are some differences though.

Calling package: Hey package!

In Nix if we want to create derivation, we usually first need to construct an standard derivation object. We could directly construct a derivation with the builtin derivation but we won’t see that many plain derivations in the wild today as it has its shortcomings.

The mkDerivation is the first part of our equation, it allows us to override some of its parts. The other one is that it already provides us with a builder script. As we will see shortly it is quite similar to package in Guix.

{
  callPackage,
  lib,
  stdenv,
  fetchurl,
  nixos,
  testers,
  versionCheckHook,
  hello,
}:

stdenv.mkDerivation (finalAttrs: {
  pname = "hello";
  version = "2.12.2";

  src = fetchurl {
    url = "mirror://gnu/hello/hello-${finalAttrs.version}.tar.gz";
    hash = "sha256-...";
  };
  
  nativeBuildInputs = [];
  buildInputs = [];

  meta = {
    description = "Program that produces a familiar, friendly greeting";
    homepage = "https://www.gnu.org/software/hello/manual/";
    license = lib.licenses.gpl3Plus;
    maintainers = with lib.maintainers; [ stv0g ];
    mainProgram = "hello";
  };
})

Every package in Nix is a function, that has some arguments, they are filled automatically by our call to callPackage but this means that we can override them if we desire to do so.

callPackage ./hello.nix {}

There are actually two attribute functions for every callPackage call. These are override and overrideAttrs, I can never remember which is which. override is for changing the function arguments passed to the function and overrideAttrs is for changing the attributes of the mkDerivation. We call them as follows (callPackage ./hello.nix {}).override or [...].overrideAttrs

Package to derivation: Wait what is a package?

Guix is very similar to Nix in how it specifies packages. There is some difference in that Nix works only with derivations, but Guix preserves the package record as long as possible, so you pass around lists of packages.

(define-public hello
  (package
    (name "hello")
    (version "2.12.2")
    (source (origin
              (method url-fetch)
              (uri (string-append "mirror://gnu/hello/hello-" version
                                  ".tar.gz"))
              (sha256
               (base32
                "..."))))
    (build-system gnu-build-system)
    (synopsis "Example GNU package")
    (description "...")
    (home-page "https://www.gnu.org/software/hello/")
    (license gpl3+)))

The other difference you may have noticed is the build-system field. It specifies the builder script, in Guix its in Guile, that will build this package. It also contains some dependencies.

If we wanted to realize package manually, we can call package->derivation to convert the package to the derivation.

100 gexps: The guix aside

In Nix we can specify derivation inputs by string substitutions. But in Guix we have a more elegant system. It’s called Gexps, it stands for Guix-Expressions and they function similarly to quote and unquote. We usually write them as #~ and #$ Gexps are records that hold all the info needed to build a derivation. Let’s look at an example.

#~(begin
    (mkdir #$output)
    (chdir #$output)
    (symlink (string-append #$coreutils "/bin/ls") "list-files"))

If we evaluate the expression we receive this.

∫ guix repl
scheme@(guix-user)> (use-modules (guix gexp))
scheme@(guix-user)> (use-modules (gnu packages base))
scheme@(guix-user)> #~(begin
    (mkdir #$output)
    (chdir #$output)
    (symlink (string-append #$coreutils "/bin/ls") "list-files"))
$1 = #<gexp (begin (mkdir #<gexp-output out>) (chdir #<gexp-output out>) (symlink (string-append #<gexp-input #<package coreutils@9.1 gnu/packages/base.scm:471 7f34089412c0>:out> "/bin/ls") "list-files")) 7f340a1cdc00>

We can access previously evaluated values with the $n syntax, it reduces the noise in this article quite a bit.

As we can see, the what we got back is the gexp record. And if we evaluate it, we get a derivation.

scheme@(guix-user)> (use-modules (guix store))
scheme@(guix-user)> ((gexp->derivation "example" $1) (open-connection))
$2 = #<derivation /gnu/store/...-example.drv => /gnu/store/...-example 7f34160212d0>
$3 = #<store-connection 256.100 7f24b6ddf5a0>

The gexp->derivation function returns a function that takes the connection to the store and realizes the derivation. The (open-connection) is just opening a socket to RPC the daemon. This has actually put the derivation into the store, but it hasn’t yet run the build code.

scheme@(guix-user)> (use-modules (guix derivations))
scheme@(guix-user)> (derivation-file-name $2)
$4 = "/gnu/store/...-example.drv"
scheme@(guix-user)> (build-things $3 (list $4))
$5 = #t

The builder itself does not understand the derivation record, so we need to first get the derivation file name from it and only then we can build it.

Inputs: Why are there so many inputs?

When specifying dependencies one could not be blamed for confusing what is what. There are so many input variants and the names differ between Nix and Guix, so here is a table for translation:

*ix allows for cross-compiling to different architectures, ie. x86->arm etc. For that it differentiates between different platforms, the host platform is the dependency’s platform it should be running on, and the target platform is the platform the derivation is building towards.

On Guix if we would want to have something like build->host compiler, we tell Guix and it solves it automatically by building a cross copiler in the background.