Overlays in tests

Run this example test yourself

To run this test directly from the example repository, run:

nix build -L github:applicative-systems/nixos-test-driver-manual#test-overlay

This tutorial shows how to use nixpkgs overlays inside tests.

The example describes an overlay that patches GNU Hello to print "Bye, world!" instead of "Hello, world!" and then tests this output.

overlay.nix

let
  hello-bye = _final: prev: {
    # (1)
    hello = prev.hello.overrideAttrs (_old: {
      postPatch = ''
        substituteInPlace src/hello.c \
          --replace-fail "Hello, world!" "Bye, world!"
      '';
      doCheck = false;
    });
  };
in
{
  name = "Test with overlays";

  node.pkgsReadOnly = false; # (2)

  containers.machine =
    { pkgs, ... }:
    {
      nixpkgs.overlays = [ hello-bye ]; # (3)
      environment.systemPackages = [ pkgs.hello ];
    };

  testScript = ''
    output = machine.succeed("hello")
    t.assertIn("Bye, world!", output, "Patched GNU Hello must output bye")
  '';
}

1. Defining overlays

   This is an example overlay. In this tutorial, we don't explain how overlays in nixpkgs work.

   If you are interested in learning about overlays, please refer to the guides at the end of this page

2. Disabling read-only pkgs

   If we don't disable this setting which is true by default, we will get the error message from here (next section).

3. Adding the overlay

   At this point, we are setting the overlay for the machine, and only this machine.

Testing app output with other testers

Using the NixOS test driver for this particular scenario can be considered over-engineering. For the sake of simplicity, this is just a minimal example scenario to explain test driver mechanics.

To test CLI-app output in isolation in such simple cases in production, please have a look at the other tester functions in nixpkgs.

Setting node.pkgsReadOnly

If we forget setting node.pkgsReadOnly to false (as also described in the NixOS manual), we will get an error message like this:

error: The option `containers.machine.nixpkgs.overlays' is defined multiple times while it's expected to be unique.
nixpkgs.overlays is set to read-only

Why does this option even exist?

The pkgs.testers.runNixOSTest performs a certain optimization: It evaluates pkgs only once for all machines and then injects it read-only into all node configurations.

As a clear majority of tests does not use overlays per node, this saves a lot of evaluation time.

Overriding the test driver itself

So far, overlays have been applied to packages inside the guest. An overlay can also replace the test driver itself — the Python program that orchestrates the VMs and containers and runs your testScript. This is useful when you need to patch driver behaviour for an experimental feature that is not yet upstream.

This feature has been recently upstreamed

Since PR #503686, the Python test driver is exposed as pkgs.nixos-test-driver and wired into tests via the module-system option config.pythonTestDriverPackage.

If pkgs.nixos-test-driver doesn't exist in your NixOS version, please upgrade to the latest nixpkgs.

The mechanism is a standard Nixpkgs overlay that calls overrideAttrs on the nixos-test-driver derivation — for example, to add a patch:

driver-overlay.nix

_: prev: {
  nixos-test-driver = prev.nixos-test-driver.overrideAttrs (old: {
    patches = old.patches or [ ] ++ [ ./my-driver.patch ];
  });
}

Apply the overlay in the host pkgs (as the driver typically does not run inside the guests): Add it to the overlays argument in your nixpkgs include. If you're using Nix flakes, this means that you need to re-import nixpkgs. See also the CUDA example:

Patching the NixOS test driver to run CUDA tests

The CUDA tutorial uses exactly this pattern to give nspawn containers access to the host's /run directory — a capability required for GPU driver access that is not yet upstream.

Further references

• Mastering Nixpkgs Overlays: Techniques and Best Practice

  ---

  

  Overlays in Nixpkgs are a powerful mechanism, but they are also complex to grasp. We tell you everything you need to know about Nixpkgs overlays.

  Visit

• What You Need to Know About Lazy Evaluation in Nix

  ---

  

  If you have no prior experience with functional programming, don't miss this article which explains the most important intricacies of lazy evaluation!

  Visit