Introduction to Nix
Nixos
Nix is a language which can be used to represent the state of any linux workstation. This is powerful because, we can use this to make the deployment of any software trivial by sending the blueprint to build the infrastructure where the code will executed.
Nix
Nix can build any version of any software on any machine. Nix is a DSL for creating packages for software deployments. As It was introduced in a paper so It is a turing complete language for coding the package.
Use nix repl to try out a nix code snippet.
Nix Language
Varialbles
# Integer
5
# Float
2.5
# String
"This is a string"
'' This is a multiline
String''
# String templating
"This is a string with ${myVariable}!"
# List
["hi" world 42.0]
# Object or Attribute Set
{
name = "shaswat";
numOfEyes = 2;
}
Functions
Every thing in Nix is an expression and will evaluate to a value
# Function in short hand
aParamater : aParameter + 1
myFunction 42
As Nix is a functional language, Any Function can take a single Paramter. So to use multiple parameters we have to either use
- nested functions More useful as the functions with fewer parameters can be reused.
a: b: c: a + b + c
- an Attribute sets More Used as it is more readable Also the Nix compiler will check if all the attributes are defined on invocation.
/* f = */
{a, b, c}: a + b + c
f {a = 1; b = 2; c = 3;}
# => 43
Optional Arguments in Attribute Sets
Optional Arguments in Attribute Sets
/* f = */
{a, b, c ? 40}: a + b + c
f {a = 1; b = 2;}
# => 43
Variables Binding
In Nix Code we can assign values to a variable only once. So To limit the scope of variable we use let scopes. In Short, there are no mutable variables in Nix.
let
f = a: a + 2;
in
f 40
As nix is lazily evaluated and the binding are immediately enforced, we can bind a result of a expression inside another bind
let
f = a: a + 2;
result = f 40;
in
result
Another example to obsfucate the usage.
let
myAttrs = {a = 1; b = 2; c = 3;};
in
myAttrs.a + myAttrs.b + myAttrs.c
# => 6
Nix has a with keyword which can be used to expose all the attributes of an attribute set inside a let in block. So we can avoid writing the name of the attribute set again and again.
let
myAttrs = {a = 1; b = 2; c = 3;};
in
with myAttrs;
a + b + c
# => 6
We should not use the above ‘with’ keyword as all the attributes decome visible in the “in” block. To be more concise and explicit we can use
let
myAttrs = {a = 1; b = 2; c = 3;};
inherit(myAttrs) a b c;
in
a + b + c
# => 6
This also allows LSPs and other tools to do better static analysis.
builtins functions
Nix comes will a minimal but powerful set of builtin functions which are defined in global variable called builtins.
# This function will return a list of all the attributes in an attribute set.
builtins.attrNames {a = 1; b = 2;}
# => ["a" "b"]
# This function gives all the attributes that a function accepts
# and tells us which of them have default values
let
f = {a, b ? 40}: a + b
in
builtins.functionArgs f
# => { a = false; b = true;}
Currying
Passing a function in another function
MultiplyTwo = a: a * 2;
AddOne = a: a + 1;
MultiplyTwoAddOne = a: MultiplyTwo(AddOne(a));
MultiplyTwoAddOne 5
Files
Nix allows us to break the code into files
# a.nix
4
# b.nix
import ./a.nix # Imported using relative path
# => 4
Paths
Paths in most languages are implemented with strings But in Nixos They are first class citizens
Absolute path is the display value of the path
./some-file.txt
# => /Users/shaswat/bla/some-file.txt
When the paths are evaluated to strings the path evaluated to the nix-store path for the file.
builtins.toString ./some-file.txt
# => /nix/store/b49dir13slsa2wsqcjc4jmsp00ybq424-some-file.txt
First part /nix/store is the path to the nix store. It can be anywhere but it is not recommended to move the nix store.
Second part b49dir13slsa2wsqcjc4jmsp00ybq424 is the hash depending on the content of the referenced file and the name of the file.
More reading at Nix Thesis -> chapter 5 “The extensional Model”
let
myHelpers = import ./helpers.nix;
in
myHelpers.bundle ./src/
Every file is copied to the nix store and made readonly and given a unique hashed file name. It is an addressable immutable data store similar to git.
Derivation
Derivations are a builtin function that executes a command and captures the produced output.
derivation {
name = "my-program"; # Every item in nix store needs a name
system = "aarch64-darwin"; # Tells which system to target
builder = "/usr/bin/clang"; # Program to build the file
src = ./main.c; # What file to target
args = [];
}
This file will be hashed and copied to nix store Nix really cares about the name, system, builder and args. The values of any additional attributes like src will provided via environment variables.
As clang can’t read the name of the file to compile from environment variables so we use bash scripting to do the compilation
derivation {
name = "my-program"; # Every item in nix store needs a name
system = "aarch64-darwin"; # Tells which system to target
builder = "/bin/bash"; # Program to build the file
src = ./main.c; # What file to target
args = ["-c" # Option for Reading bash commands from strings ''
/usr/bin/clang $src
'' ]; # The $src will be replaced by the path to main.c file in the nix store
}
then nix-instantiate my-derivation.nix it doesn’t executes the derivation but serializes the derivation in deterministic manner.
/nix/store/mgan1f57b46nfwwffvr591qbbafn2i6i-my-program.drv
On cat ting the file we will see unstructured form of the serialized derivation which is hard to read.
We can use
nix derivation show /nix/store/mgan1f57b46nfwwffvr591qbbafn2i6i-my-program.drv to pretty print the derivation as JSON.
To execute a derivation we call it realizing the derivation which can be done by
nix-store --realize /nix/store/mgan1f57b46nfwwffvr591qbbafn2i6i-my-program.drv
This will execute the derivation inside a temporary folder known as sandbox will all the environment variables set.
This helps prevent the builder from pulling system configuration files or system libraries which can make the derivation not reproduciable.
Any derivation is also provided an out folder where all the outputs should be placed. If the path remains non existant then It’s an error, so we must tell the compiler where to place the file.
We can use nix-build ./my-derivation.nix to both realise and serialize the derivation in one step.
This will create a symlink to the result inside the nix store.
As we have hardcoded the paths to bash and clang which will make the program non reproducable. So then we can build derivations to both bash and clang to get a particular version.
derivation {
name = "clang-16";
system = "aarch64-darwin";
src = builtins.fetchTarball { # Builtin function to fetch a tarball
url = "https://github.com/.../clang-16.0.0.src.tar.gz";
sha256 = "sha256: ..."; # Optional, If not given nix will download the file everytime to check if file has changed.
# Helps in time wasting download and supply change attacks.
};
builder = "/bin/bash";
args = [ "-c" ''
/usr/bin/tar -xtf $src
# ... building using system tools ...
''];
}
This Clang compiler is build using system tools which makes it not reproducible and thus nix labels anything build with system tools as “impure”. So any tools created using some other tools takes it as input to create further new tools which is called a [[Merkle tree]]. So this hash can hold information about the every dependency and every sub - dependency which can be used to check if an derivation is build before. So even any simple change with these tools will cause difference in hash and make a specific different nixstore item for that specific variant.
Stage one Compiler is created using impure inputs Stage two compiler is created using inputs created by stage one compiler which is now specified or pure .
This can be done for every software that we use and have a large repository of programs which can be run anytime.
As we are using a sandbox which is the most bare minimum ie nothing.
And everything needs to be passed explicitly. SO we can creaet a dependency tree
This creates a direct dependency tree
nix-store --query --references /nix/store/si...4m-coreuitls.drv
This creates a transitive dependecny tree which contains all direct dependencies and sub-dependencies and sub-sub-dependencies and so on
nix-store --query --requisties /nix/store/si...4m-coreuitls.drv
Stdenv
A Collection of pure build Tools is knwon as a standard environment.
let
pkgs = import (builtins.fetchTarball{
url = "github.com/nixpkgs.24.05.tar.gz"; # Link to TarBall of Nixpkgs
sha256 = "sha256: ...";
}) {};
in
derivation {
name = "my-program";
system = "aarch64-darwin";
builder = "${pkgs.bash}/bin/bash";
src = ./main.c;
args = ["-c"
/${pkgs.clang}/bin/clang $src
'' ];
}
This will code very messy, so we can use
let
pkgs = import (builtins.fetchTarball{
url = "github.com/nixpkgs.24.05.tar.gz"; # Link to TarBall of Nixpkgs
sha256 = "sha256: ...";
}) {};
in
pkgs.stdenv.mkDerivation {
name = "my-program";
system = "aarch64-darwin";
builder = "${pkgs.bash}/bin/bash";
src = ./main.c;
args = ["-c"
/${pkgs.clang}/bin/clang $src
'' ];
}
Here we make a standard environment and we can call the mkDerivation which is a high level wrapper function over the derivation function. It will populate the sandbox will lots of useful tools like clang, bash, cmake, tar, etc. So we can remove lots of attributes.
let
pkgs = import (builtins.fetchTarball{
url = "github.com/nixpkgs.24.05.tar.gz"; # Link to TarBall of Nixpkgs
sha256 = "sha256: ...";
}) {};
in
pkgs.stdenv.mkDerivation {
name = "my-program";
src = ./main.c;
nativeBuildInputs = []; # It can contain the tools that are required to build the derivation.
buildINputs = []; # Dependencies which are required at runtime.
# Dynamc libraries or configuration files.
dontUnpack = true; # As our program is not a tarball we dont' need to unpack it
buildPhase = '' # this describes how the building is done.
clang $src -o my-program
'';
installPhase = '' # This descibes the how we populate the output folder.
# This needs to follow the Linux File Hierarchy Standard
# bin for binaries lib for libraries and include for headers
mkdir -p $out/bin
cp my-program $out/bin
'';
}
We can use this to bootstrap support to any language But nix provides support for numerous languages and libarries
pkgs.rustPlatform.buildRustPackage rec {
name = "my-rust-crate";
src = ./.;
cargoLock = {
lockFile = "${src}/Cargo.lock";
};
}
Similar helpers exist for NodeJs, Go
pkgs.buildNpmPackage {
name = "my-project";
src = ./.;
}
pkgs.buildGoPackage {
name = "my-project";
src = ./.;
}
pkgs.buildDartApplication {
name = "my-project";
src = ./.;
}
pkgs.buildDotnetPackage {
name = "my-project";
src = ./.;
}
pkgs.buildRubyGem {
name = "my-project";
src = ./.;
}
Lib
This contains utilities We have more options to download source code form internet. Also Functions to work with nix values
pkgs.lib.strings.hasPrefix
pkgs.lib.list.map
pkgs.lib.attrsets.recursiveUpdate
All are documented in the manual nixos.org/manual/nixpkgs noogle.dev to search nixos function by name
For Security read, Model Nix Thesis: Chapter 6
Python
Horrible package manager - pip add python3 to the packages or shell we need to expose the compiled files to python
Shell
create shell.nix
{pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
packages = [
pkgs.python3
];
env.LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath [
pkgs.stdenv.cc.cc.lib
pkgs.libz
];
}
then use activate the shell with nix-shell then source ./venv/bin/activate
Flake way
{
description = "flake";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
};
outputs = {self, nixpkgs, ... }: let
pkgs = nixpkgs.legacyPackages."x86_64-linux";
in {
devShells.x86_64-linux.default = pkgs.mkShell {
packages = [
pkgs.python3
];
env.LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath [
pkgs.stdenv.cc.cc.lib
pkgs.libz
];
};
};
};
};
Nix Way
Using python package to take scope of modules
{pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
packages = [
(pkgs.python3.withPackages(pypkgs:[
pypkgs.numpy
pypkgs.requests
pypkgs.pandas
]))
];
}
How to manage python packages without actual packages is to build oneself
nix run github:nix-community/pip2nix -- generate requests
nix run github:nix-community/pip2nix -- ./requirements.nix
nix shell github:nix-community/pip2nix
pip2nix generate "requests=2.32" numpy
{pkgs ? import <nixpkgs> {} }:
let
packageOverrides = pkgs.callPackage ./python-packages.nix {};
python = pkgs.python3.override {inherit packageOverrides};
in
pkgs.mkShell {
packages = [
(python.withPackages(p:[
p.seaborn
]))
];
}
Flakes
Experimental features on nixos Pin packages versions
Home - Manager
to declare files inside the home directory To make it useful for other distros Finding Options and functions Options for nixos and homemanager
Packages
nix search nixpkgs <searchterm>
nix repl
sudo nix-rebuild build-vm .#<hostname>
Direnv
Load custom envirionment for projects searchs for .envrc and .env
Insertes the environment as directed by .envrc Fast Language agnostic
DIRENVLOGFORMAT = "" //fast direnv allow Nixos-env allows for
working with nks pgs Help avoid garbage collection Caches the
environment use nix in .envrc
use nix myenv.nix
For flakes use flake Editors can import the environment
Further Learning
Foundations: how Nix actually works
https://nixos.org/guides/nix-pills/
Nix language itself
Nix Language Reference
https://nixos.org/manual/nix/stable/language/
Nixpkgs manual – “Language” + “Functions”
https://nixos.org/manual/nixpkgs/stable/
Packaging software (you’re already here)
NixOS (the module system)
NixOS Manual
https://nixos.org/manual/nixos/stable/
NixOS Module System (deep dive)
https://nixos.org/manual/nixos/stable/index.html#sec-writing-modules
Flakes
Official Flakes documentation
https://nixos.wiki/wiki/Flakes
Flake schema
https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html
Home-Manager
Home-Manager Manual
https://nix-community.github.io/home-manager/
Learn by reading real configs
- https://github.com/Misterio77/nix-config
- https://github.com/hlissner/dotfiles
- https://github.com/rycee/configuration.nix
- https://github.com/gvolpe/nix-config
Advanced topics
Communities that actually help
- Matrix:
#nixos:matrix.org - Discourse: https://discourse.nixos.org
- Reddit: r/NixOS (mixed quality)
- GitHub issues (best for real answers)
- Go throught youtubes
- nix-hero
- vimjoyer
Nix Developer Environment
Nix Shell
# shell.nix
{pkgs ? import <nixpkgs> { }}:
pkgs.mkShell
{
# ...
}
We can launch the shell by using nix-shell command
It can also be added to a flake by using
devShells.x86_64-linux.default = (import ./shell.nix {inherit pkgs; });
Inside the let in block
To enter the dev env from a flake we use the command nix develop.
# shell.nix
{ pkgs ? import <nixpkgs> { }}:
pkgs.mkShell
{
nativeBuildInputs = [
# packages
# pkgs.nodejs
];
}
# shell.nix
{ pkgs ? import <nixpkgs> { }}:
pkgs.mkShell
{
nativeBuildInputs = with pkgs; [
nodejs
];
}
To change the shell we can use, to specify the first command that will be executed when we enter the environment.
use nix-shell --command zsh for non flake setup
use nix develop --command zsh for flake setup
We can use shellHook to run some larger commands
# shell.nix
{ pkgs ? import <nixpkgs> { }}:
pkgs.mkShell
{
nativeBuildInputs = with pkgs; [
nodejs
];
shellHook = ''
echo "welcome"
source ./something.sh
echo "to my shell!" | ${pkgs.lolcat}/bin/lolcat
'';
}
To declare environment variables we can use
# shell.nix
{ pkgs ? import <nixpkgs> { }}:
pkgs.mkShell
{
nativeBuildInputs = with pkgs; [
nodejs
];
shellHook = ''
echo "welcome"
source ./something.sh
echo "to my shell!" | ${pkgs.lolcat}/bin/lolcat
'';
COLOR = "blue"; # Can be defined anywhere to get the variable COLOR
PASSWORD = import ./password.nix; # Can be used to import sensitive informations like password to be placed in gitignore
}
Use Nixhub.io to find the right nix package for application verisioning.
To get packages without the shell file
use nix-shell -p python3 for non flake system or nix shell nixpkgs#python3 for flake system
We can use nix shell to get packages from other nixpkgs versions or even github
nix shell nixpkgs#python3
nix shell nixpkgs/nixos-22.11#python3
nix shell github:vimjoyer/ansimage
Python Dev Env
To Run the City Poster
nix shell nixpkgs#python3
nix shell nixpkgs#python3 nixpkgs#gcc
export LD_LIBRARY_PATH=/nix/store/cf1a53iqg6ncnygl698c4v0l8qam5a2q-gcc-14.3.0-lib/lib:$LD_LIBRARY_PATH