nix-portable not working due to missing libssl.so.10

[PeterStolz]

Hi,
first of all I love nix-portable and use it frequently, but now stumbled over a problem that I can't reproduce on other servers nor fix myself.

Essentially all nix-portable invocations on that specific server that are not just nix-portable fail with /home/hifis/hifis0006/.nix-portable/nix/store/j823gnvpxh1fjibycmldrspkk3pgivwp-nix-2.20.6/bin/nix: error while loading shared libraries: libssl.so.10: cannot open shared object file: No such file or directory, but the library is installed on the host and nix should be able to fetch it internally.
I have also tried nix, proot and bwrap, but the result does not change.

Does anyone recognize what the problem might be or which information I can add to make it easier to troubleshoot?
Thanks in advance for the help

Here is the sequence of commands I ran:

(venv_a) curl -L "https://github.com/DavHau/nix-portable/releases/latest/download/nix-portable-$(uname -m)" > ./nix-portable

(venv_a) chmod +x nix-portable

(venv_a) rm -rf ~/.nix-portable/

(venv_a) uname -m
x86_64

(venv_a) ldconfig -p | grep libssl
        libssl3.so (libc6,x86-64) => /lib64/libssl3.so
        libssl.so.10 (libc6,x86-64) => /lib64/libssl.so.10
        libssl.so.1.1 (libc6,x86-64) => /lib64/libssl.so.1.1
        libssl.so (libc6,x86-64) => /lib64/libssl.so

(venv_a) NP_DEBUG=1 NP_RUNTIME=bwrap ./nix-portable nix-shell
installing files
figuring out ssl certs
SSL_CERT_FILE not defined. trying to find certs automatically
found /etc/ssl/certs/ca-bundle.crt with real path /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
figuring out which runtime to use
bwrap executable: /home/hifis/hifis0006/.nix-portable/bin/bwrap
nix executable: /home/hifis/hifis0006/.nix-portable/bin/nix
proot executable: /home/hifis/hifis0006/.nix-portable/bin/proot
testing all available runtimes...
runtime selected via NP_RUNTIME: bwrap
NP_RUNTIME: bwrap
base command will be: /home/hifis/hifis0006/.nix-portable/bin/bwrap --bind /home/hifis/hifis0006/.nix-portable/emptyroot / --dev-bind /dev /dev --bind /home/hifis/hifis0006/.nix-portable/nix /nix --bind /boot /boot --bind /etc /etc --bind /usr/bin /bin --bind /usr/lib /lib --bind /usr/lib64 /lib64 --bind /usr/sbin /sbin --bind /archiv /archiv --bind /bigdata /bigdata --bind /data /data --bind /gpfs /gpfs --bind /home /home --bind /lost+found /lost+found --bind /media /media --bind /misc /misc --bind /mnt /mnt --bind /net /net --bind /opt /opt --bind /proc /proc --bind /root /root --bind /run /run --bind /srv /srv --bind /sys /sys --bind /tmp /tmp --bind /trinity /trinity --bind /usr /usr --bind /var /var --bind /.autorelabel /.autorelabel --bind /.readahead /.readahead --bind /etc/host.conf /etc/host.conf --bind /etc/hosts /etc/hosts --bind /etc/networks /etc/networks --bind /etc/passwd /etc/passwd --bind /etc/group /etc/group --bind /etc/nsswitch.conf /etc/nsswitch.conf --bind /etc/resolv.conf /etc/resolv.conf --bind /usr/share/zoneinfo/Europe/Berlin /usr/share/zoneinfo/Europe/Berlin --bind /home/hifis/hifis0006 /home/hifis/hifis0006 --bind /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /home/hifis/hifis0006/.nix-portable/ca-bundle.crt
extracting missing store paths
registering new store paths to DB
running command: /home/hifis/hifis0006/.nix-portable/bin/bwrap --bind /home/hifis/hifis0006/.nix-portable/emptyroot / --dev-bind /dev /dev --bind /home/hifis/hifis0006/.nix-portable/nix /nix --bind /boot /boot --bind /etc /etc --bind /usr/bin /bin --bind /usr/lib /lib --bind /usr/lib64 /lib64 --bind /usr/sbin /sbin --bind /archiv /archiv --bind /bigdata /bigdata --bind /data /data --bind /gpfs /gpfs --bind /home /home --bind /lost+found /lost+found --bind /media /media --bind /misc /misc --bind /mnt /mnt --bind /net /net --bind /opt /opt --bind /proc /proc --bind /root /root --bind /run /run --bind /srv /srv --bind /sys /sys --bind /tmp /tmp --bind /trinity /trinity --bind /usr /usr --bind /var /var --bind /.autorelabel /.autorelabel --bind /.readahead /.readahead --bind /etc/host.conf /etc/host.conf --bind /etc/hosts /etc/hosts --bind /etc/networks /etc/networks --bind /etc/passwd /etc/passwd --bind /etc/group /etc/group --bind /etc/nsswitch.conf /etc/nsswitch.conf --bind /etc/resolv.conf /etc/resolv.conf --bind /usr/share/zoneinfo/Europe/Berlin /usr/share/zoneinfo/Europe/Berlin --bind /home/hifis/hifis0006 /home/hifis/hifis0006 --bind /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /home/hifis/hifis0006/.nix-portable/ca-bundle.crt /home/hifis/hifis0006/.nix-portable/nix/store/j823gnvpxh1fjibycmldrspkk3pgivwp-nix-2.20.6/bin/nix-store --load-db
Testing if nix can build stuff without sandbox
/home/hifis/hifis0006/.nix-portable/nix/store/j823gnvpxh1fjibycmldrspkk3pgivwp-nix-2.20.6/bin/nix: error while loading shared libraries: libssl.so.10: cannot open shared object file: No such file or directory
Fatal error: nix is unable to build packages

[DavHau]

Hey, I guess that the virtual nix store under ~/nix-portable might be corrupt. If you delete it, does it work?

[PeterStolz]

So I just retried on a fresh environment and it results in the same problem:

(venv_a) ls -a
.  ..  .clusterService  .ipython  .jupyter  .local  .ssh  Welcome.ipynb
(venv_a) curl -L "https://github.com/DavHau/nix-portable/releases/latest/download/nix-portable-$(uname -m)" > ./nix-portable
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 64.9M  100 64.9M    0     0  34.9M      0  0:00:01  0:00:01 --:--:-- 53.7M
(venv_a) chmod +x nix-portable
(venv_a) NP_DEBUG=1 NP_RUNTIME=bwrap ./nix-portable nix-shell
installing files
figuring out ssl certs
SSL_CERT_FILE not defined. trying to find certs automatically
found /etc/ssl/certs/ca-bundle.crt with real path /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
figuring out which runtime to use
bwrap executable: /home/hifis/hifis0007/.nix-portable/bin/bwrap
nix executable: /home/hifis/hifis0007/.nix-portable/bin/nix
proot executable: /home/hifis/hifis0007/.nix-portable/bin/proot
testing all available runtimes...
runtime selected via NP_RUNTIME: bwrap
NP_RUNTIME: bwrap
base command will be: /home/hifis/hifis0007/.nix-portable/bin/bwrap --bind /home/hifis/hifis0007/.nix-portable/emptyroot / --dev-bind /dev /dev --bind /home/hifis/hifis0007/.nix-portable/nix /nix --bind /boot /boot --bind /etc /etc --bind /usr/bin /bin --bind /usr/lib /lib --bind /usr/lib64 /lib64 --bind /usr/sbin /sbin --bind /archiv /archiv --bind /bigdata /bigdata --bind /data /data --bind /gpfs /gpfs --bind /home /home --bind /lost+found /lost+found --bind /media /media --bind /misc /misc --bind /mnt /mnt --bind /net /net --bind /opt /opt --bind /proc /proc --bind /root /root --bind /run /run --bind /srv /srv --bind /sys /sys --bind /tmp /tmp --bind /trinity /trinity --bind /usr /usr --bind /var /var --bind /testfs1 /testfs1 --bind /testfs2 /testfs2 --bind /testfs3 /testfs3 --bind /.autorelabel /.autorelabel --bind /.readahead /.readahead --bind /etc/host.conf /etc/host.conf --bind /etc/hosts /etc/hosts --bind /etc/networks /etc/networks --bind /etc/passwd /etc/passwd --bind /etc/group /etc/group --bind /etc/nsswitch.conf /etc/nsswitch.conf --bind /etc/resolv.conf /etc/resolv.conf --bind /usr/share/zoneinfo/Europe/Berlin /usr/share/zoneinfo/Europe/Berlin --bind /home/hifis/hifis0007 /home/hifis/hifis0007 --bind /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /home/hifis/hifis0007/.nix-portable/ca-bundle.crt
extracting missing store paths
registering new store paths to DB
running command: /home/hifis/hifis0007/.nix-portable/bin/bwrap --bind /home/hifis/hifis0007/.nix-portable/emptyroot / --dev-bind /dev /dev --bind /home/hifis/hifis0007/.nix-portable/nix /nix --bind /boot /boot --bind /etc /etc --bind /usr/bin /bin --bind /usr/lib /lib --bind /usr/lib64 /lib64 --bind /usr/sbin /sbin --bind /archiv /archiv --bind /bigdata /bigdata --bind /data /data --bind /gpfs /gpfs --bind /home /home --bind /lost+found /lost+found --bind /media /media --bind /misc /misc --bind /mnt /mnt --bind /net /net --bind /opt /opt --bind /proc /proc --bind /root /root --bind /run /run --bind /srv /srv --bind /sys /sys --bind /tmp /tmp --bind /trinity /trinity --bind /usr /usr --bind /var /var --bind /testfs1 /testfs1 --bind /testfs2 /testfs2 --bind /testfs3 /testfs3 --bind /.autorelabel /.autorelabel --bind /.readahead /.readahead --bind /etc/host.conf /etc/host.conf --bind /etc/hosts /etc/hosts --bind /etc/networks /etc/networks --bind /etc/passwd /etc/passwd --bind /etc/group /etc/group --bind /etc/nsswitch.conf /etc/nsswitch.conf --bind /etc/resolv.conf /etc/resolv.conf --bind /usr/share/zoneinfo/Europe/Berlin /usr/share/zoneinfo/Europe/Berlin --bind /home/hifis/hifis0007 /home/hifis/hifis0007 --bind /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /home/hifis/hifis0007/.nix-portable/ca-bundle.crt /home/hifis/hifis0007/.nix-portable/nix/store/j823gnvpxh1fjibycmldrspkk3pgivwp-nix-2.20.6/bin/nix-store --load-db
Testing if nix can build stuff without sandbox
/home/hifis/hifis0007/.nix-portable/nix/store/j823gnvpxh1fjibycmldrspkk3pgivwp-nix-2.20.6/bin/nix: error while loading shared libraries: libssl.so.10: cannot open shared object file: No such file or directory
Fatal error: nix is unable to build packages

Even after deleting ~/.nix-portable and ./nix-portable the error persists.

I was using

(venv_a) sha256sum ./nix-portable
b409c55904c909ac3aeda3fb1253319f86a89ddd1ba31a5dec33d4a06414c72a  ./nix-portable
(venv_a) uname -m
x86_64

this time