Had a lot of headaches the last week or two trying to optimize star citizen as well as fix a vulkan RHI bug which was affecting unreal engine games.

Apparently rolling release schedules (like NixOS23.05, 23.11, 24.05...) are better for servers since they're less prone to change, where as nightlies like for the unstable Branch are better for gaming since those latest drivers are likely the current ones for a newly released game

This idea is inspired by nixos-mailserver. It was so easy to spin up the mailserver after changing some DNS records and putting in some settings. I thought it might be a good idea to do the same for services that need public, decentralized infrastructure to support. Some ideas include

• Tor relay, or exit node
• Encrypted messaging nodes. It looks like SimpleX chat relies on SMP servers to relay communication
• Crypto miners (I know, I know, but you understand how it fits the "public contribution" usecase)
• Search engines like searxng (I currently use a public instance)
• Libredirect services, like proxy clients for social media

Maybe federated services, but those require more than just the software running on the public internet. Those require moderation and long term maintenance. Ideally, the services in this config would be ephemeral.

Does this sound like a good idea? Would you spin one of these up on a $10 VPS? I understand that this is the NixOS community, not necessarily the privacy community, but I figured thered be overlap.

What other services do you think would be applicable?

Went through the pain of packaging a python project on Nixos. Here's some issues I hit, and how I got lucky resolving them. I feel the most reliable way of doing this in the future is to use docker and just imperatively build.

Here's how I got web drivers, AI dependencies, gpu dependencies, and an api dependency bundled together into an ephemeral shell for python development, on NixOS 23.11

0. Enable Flakes

1. Start with setting up poetry2nix

2. Get the template flake by running nix flake init --template github:nix-community/poetry2nix

3. in the flake.nix, sometimes changing projectDir = self to projectDir = ./. fixed some issues

4. in your terminal, run nix develop . to build the poetry app with python packages described in pyproject.toml

5. By default, just poetry and python latest should be installed. the dependencies for the project (which gets reflected in the pyproject.toml) are updated with poetry add, such as poetry add numpy selenium scikit-learn

6. Exit out of the ephemeral shell from nix develop ., and rerun to have poetry2nix rebuild and link the newly declared packages

Poetry2nix has worked pretty well for the more obscure python packages, but failed in others. For example, sentence-transformers would depend on maturin, which would fail to link setuptools. If poetry doesn't work, you can try and get the package from nixpkgs, or specify sha256s from pypi.org

Here's an example of what I added to my flake.nix to get gpu acceleration, sentence-transfomers, firefox drivers for selenium, and other packages poetry failed to setup:

packages = [ pkgs.poetry pkgs.python311Packages.sentence-transformers pkgs.firefox 
            pkgs.python311Packages.openai pkgs.python311Packages.yt-dlp pkgs.python311Packages.pyopencl
];

was added to this flake.nix, as in,

{
  description = "Application packaged using poetry2nix";

  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    poetry2nix = {
      url = "github:nix-community/poetry2nix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
  outputs = { self, nixpkgs, flake-utils, poetry2nix }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        # see https://github.com/nix-community/poetry2nix/tree/master#api for more functions and examples.
        pkgs = nixpkgs.legacyPackages.${system};
        inherit (poetry2nix.lib.mkPoetry2Nix { inherit pkgs; }) mkPoetryApplication;
      in
      {
        packages = {
          myapp = mkPoetryApplication {
            projectDir = ./.;
          };
          default = self.packages.${system}.myapp;
        };
        devShells.default = pkgs.mkShell {
          inputsFrom = [ self.packages.${system}.myapp ];
          packages = [ pkgs.poetry pkgs.python311Packages.sentence-transformers pkgs.firefox 
            pkgs.python311Packages.openai pkgs.python311Packages.yt-dlp pkgs.python311Packages.pyopencl
          ];
          nativeBuildInputs = [(
            pkgs.python311Packages.buildPythonPackage rec {
              pname = "serpapi";
              version = "0.1.5";
              src = pkgs.python311Packages.fetchPypi {
                inherit pname version;
                sha256 = "b9707ed54750fdd2f62dc3a17c6a3fb7fa421dc37902fd65b2263c0ac765a1a5";
              };
            }
          )];
        };
      });
}

There was one package (serpapi), which was not in nixpkgs, and poetry failed as well. Adding this to native build inputs got serpapi installed

nativeBuildInputs = [(
            pkgs.python311Packages.buildPythonPackage rec {
              pname = "serpapi";
              version = "0.1.5";
              src = pkgs.python311Packages.fetchPypi {
                inherit pname version;
                sha256 = "b9707ed54750fdd2f62dc3a17c6a3fb7fa421dc37902fd65b2263c0ac765a1a5";
              };
            }
)];

All in all, it works, and I have no doubt I've made a reproducible environment. What attracts me is I've never had an easier time setting up cuda/cudnn/tensorrt/... system drivers have been near effortless, and much faster to setup than on debian. Tools like sentence-transformers and torch default to packages which leverage the GPU.

What pushes me away, is I've had failures in each of the three methods for specifying package dependencies, even though one of the three eventually was the fix for integrating the dependencies into my shell. For now, I'll stick with it, but it's hard for me to suggest to a team we use this in development

So I tried to follow some tutorial about flakes, but it seems these are extra-experimental still.

I am using NixOS 23.11 with Nix 2.18.1 in a VM (those are the most recent stable versions, right?).

Trying around I already found out that instead of eg. nix flake update I have to use --extra-experimental-features two times to get this simple command:

nix --extra-experimental-features nix-command --extra-experimental-features flakes flake update

Searching the web I found several different things that people put into their /etc/nixos/configuration.nix to enable this globally, but none of those worked for me. I assume there is still a way to do this - can someone please tell me the correct syntax for Nix 2.18.1?

What makes things worse is that I cannot start playing around with home-manager and flakes, because home-manager switch flake . seems to use nix flake internally, which leads to errors instead of results.

I heard a lot about the concepts of nix and NixOS and I'd love to try it.

After installing the VirtualBox demo, I keep getting stuck with every tiny step I take, though.

So I was wondering if there are any tutorials for beginners that you can recommend?

I couldn't find anything on the internet - everything that looks like a tutorial presumes a lot of things everybody seems to know about nix, so no need to explain those.

Where can I find those explanations to make the first baby steps with NixOS?

To put it in other words: Where is NixOS for dummies?