Project overview

HostMap is a set of tools used to track what is installed on machines managed by a team of platform engineers.
It consists of:

1. One or more servers running a web-server and a database hosting the data and overview.
2. A wrapper around the switch-to-configuration script on each managed host.
   The wrapper script is logging information about each activation -- Timestamp, username, the store path and action taken.
3. An activation-logger running on each managed host, which can serve the collected activations.
4. A scraper running on the HostMap servers will contact each Nix-host in the configuration and fetch the list and post it to the web server where it is stored in the database.
5. A hostmap-update script running on the build server that will save information about the building of hostname, git commit hashes, branch name, store path and a timestamp. The files generated by hostmap-update is transferred to the hostmap web server

Host configuration example:

[
  {
    "hostname": "git-host-t01",
    "host_url": "git-host-t01.my-domain.dk",
    "metadata": {
      "env": "stg",
      "host_group_name": "hosts-prod"
    }
  },
  {
    "hostname": "git-host-p01",
    "host_url": "git-host-p01.my-domain.dk",
    "metadata": {
      "env": "stg",
      "host_group_name": "hosts-prod"
    }
  },
  {
    "hostname": "localhost",
    "host_url": "localhost",
    "metadata": {
      "env": "stg",
      "host_group_name": "hosts-prod"
    }
  }
]

Scraped file Example:

2026-04-13 09:31:04+02:00;user1;/nix/store/r1d177x9dan0jgj0280w75li5fqlpp8w-nixos-system-hosts-p03-25.11pre-git;boot
2026-04-21 12:07:42+02:00;user2;/nix/store/wc8sgkdad9wnb2q9bds5mlsfy7999990-nixos-system-hosts-p03-25.11pre-git;switch
2026-04-22 09:31:04+02:00;user1;/nix/store/r1d177x9dan0jgj0280w75li5f999991-nixos-system-hosts-p03-25.11pre-git;dry-activate

Nix git build example:

[
  {
    "hostname": "image-server-p101",
    "store_path": "/nix/store/ldk0002skbab10v3b8xy343b33c2vffr-nixos-system-imageserver-p02-25.05pre-git",
    "commit_hash": "dda93d97b7856c4ff7d3d959d0790ca6ad80d2bc",
    "branch": "hostmap-ci-take-1",
    "created_at": "2025-10-14 13:53:19+02:00"
  },
  {
    "hostname": "image-server-p201",
    "store_path": "/nix/store/ldk0002skbab10v3b8xy343b33c2vffr-nixos-system-imageserver-p02-25.05pre-git",
    "commit_hash": "dda93d97b7856c4ff7d3d959d0790ca6ad80d2bd",
    "branch": "hostmap-ci-take-2",
    "created_at": "2025-10-14 13:53:19+02:00"
  },
  {
    "hostname": "image-server-p301",
    "store_path": "/nix/store/ldk0002skbab10v3b8xy343b33c2vffr-nixos-system-imageserver-p02-25.05pre-git",
    "commit_hash": "dda93d97b7856c4ff7d3d959d0790ca6ad80d2be",
    "branch": "hostmap-ci-take-3",
    "created_at": "2025-10-15 13:53:19+02:00"
  },
  {
    "hostname": "ca-platform-p01",
    "store_path": "/nix/store/ldk0002skbab10v3b8xy343b33c2vffr-nixos-system-imageserver-p02-25.05pre-git",
    "commit_hash": "dda93d97b7856c4ff7d3d959d0790ca6ad80d2bf",
    "branch": "hostmap-ci-take-4",
    "created_at": "2025-10-16 13:53:19+02:00"
  }
]

Development

Initial setup

nix develop
pg_initial_setup
cargo sqlx migrate run

Development start

nix develop
pg_start

The following environment variables a set from flake.nix\

PG=$PWD/.dev_postgres
PGDATA=$PG/data
PGPORT=5432
PGHOST=localhost
PGUSER=$USER
PGPASSWORD=postgres
PGDATABASE=hostmap-dev
DATABASE_URL=postgres://$PGUSER:$PGPASSWORD@$PGHOST:$PGPORT/$PGDATABASE

run server:\

For the server to run in development we need a git repository and an API key. The API key is obtained from the Git host UI.
If the API key is saved in the file ./api-key.txt it will be ignored by git.

cargo run server --database-url $DATABASE_URL --repo-url https://somegit.my-domain.dk/me/my-deployment --grouping-key host_group_name --api-key-file ./api-key.txt --columns "loc" 

run activation logger:

./target/debug/hostmap activation-logger --url-path /hostmap/hostmap-activation-logs.csv --activation-log-file /var/log/hostmap-activation-logs/hostmap-activation-logs.csv --server-ip 0.0.0.0 --port 9488

run scraper:

cargo run scraper --hosts-file ./test-assets/minimalTargetList.json --scrape-interval 5 --activation-logger-port 9488 --api-key-file ./api-key.txt