From 69ff8c5ab9d6861a4752f84521563573637602da Mon Sep 17 00:00:00 2001 From: Rich Turner <7072278+richturner@users.noreply.github.com> Date: Mon, 22 Jun 2026 13:06:01 +0100 Subject: [PATCH 1/3] Document DB snapshot restore --- docs/developer-guide/system-administration.md | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/developer-guide/system-administration.md b/docs/developer-guide/system-administration.md index faaf0d0..efbf436 100644 --- a/docs/developer-guide/system-administration.md +++ b/docs/developer-guide/system-administration.md @@ -272,6 +272,31 @@ mv -v data/old/* $PGDATA rm -r data/new data/old ``` +### Attaching DB snapshot to an EC2 instance +For data recovery or testing purposes it may be desirable to attach a DB snapshot to an existing instance. the following steps explain how to do this using an AWS EC2 EBS snapshot of the docker volumes. + +#### New instance +A snapshot can be easily attached to a new EC2 instance using the `Provision Host` github action or the `provision_host.sh` bash script directly. + +#### Existing instance +1. Create a new EBS volume from the EBS snapshot +1. Attach the new EBS volume to the existing EC2 instance using the AWS console or the `aws ec2 attach-volume` command +1. Check the EBS volume is attached as a disk using `lsblk` (assuming it is attached as `/dev/nvme2n1` for remaining steps) +1. Mount the EBS volume `sudo mount -t xfs -o nouuid /dev/nvme2n1 /mnt/snapshot` +2. Start a temporary PostgreSQL instance to connect to the snapshot DB:Locate the PostgreSQL data volume in `ls /mnt/snapshot` generally called `or_postgresql-data and set the `PGDATA` environment variable to the mount point. + ```shell + sudo docker run -d \ + --rm \ + --name temp_recovery_db \ + -p 5433:5432 \ + -e POSTGRES_PASSWORD=postgres \ + -v /mnt/snapshot//_data:/var/lib/postgresql/data \ + -v /tmp:/export \ + openremote/postgresql:latest + ``` +1. Wait for the DB to be ready; it can take a while when starting from a large snapshot, look at the docker container logs for DB ready message +1. Exec into the container and run DB queries, exports, etc. `docker exec -it temp_recovery_db psql -U postgres -d openremote` + ### Useful resources - [Shared memory](https://www.instaclustr.com/blog/postgresql-docker-and-shared-memory/#:~:text=Docker%20and%20SHM%2DSize&text=This%20means%20that%20instead%20of,default%2C%20this%20limit%20is%2064MB) - [Index maintenance](https://wiki.postgresql.org/wiki/Index_Maintenance) From 67bf73e420e9b8783cbcc2240be73e995e8517d5 Mon Sep 17 00:00:00 2001 From: Rich Turner <7072278+richturner@users.noreply.github.com> Date: Mon, 22 Jun 2026 13:10:52 +0100 Subject: [PATCH 2/3] WIP --- docs/developer-guide/system-administration.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/developer-guide/system-administration.md b/docs/developer-guide/system-administration.md index efbf436..7a75a2d 100644 --- a/docs/developer-guide/system-administration.md +++ b/docs/developer-guide/system-administration.md @@ -295,7 +295,10 @@ A snapshot can be easily attached to a new EC2 instance using the `Provision Hos openremote/postgresql:latest ``` 1. Wait for the DB to be ready; it can take a while when starting from a large snapshot, look at the docker container logs for DB ready message -1. Exec into the container and run DB queries, exports, etc. `docker exec -it temp_recovery_db psql -U postgres -d openremote` +1. Extract/Analyse the data as required either by: + * Exec'ing into the container `docker exec -it temp_recovery_db psql -U postgres -d openremote` + or + * Modify the manager and keycloak DB settings to connect to the temporary DB on port 5433 ### Useful resources - [Shared memory](https://www.instaclustr.com/blog/postgresql-docker-and-shared-memory/#:~:text=Docker%20and%20SHM%2DSize&text=This%20means%20that%20instead%20of,default%2C%20this%20limit%20is%2064MB) From a9f4a70e903daff7e9dd9dc97aca394f03771f8d Mon Sep 17 00:00:00 2001 From: Rich Turner <7072278+richturner@users.noreply.github.com> Date: Mon, 22 Jun 2026 14:16:20 +0100 Subject: [PATCH 3/3] WIP --- docs/developer-guide/system-administration.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/developer-guide/system-administration.md b/docs/developer-guide/system-administration.md index 7a75a2d..f157ee8 100644 --- a/docs/developer-guide/system-administration.md +++ b/docs/developer-guide/system-administration.md @@ -277,13 +277,15 @@ For data recovery or testing purposes it may be desirable to attach a DB snapsho #### New instance A snapshot can be easily attached to a new EC2 instance using the `Provision Host` github action or the `provision_host.sh` bash script directly. +1. Change PostgreSQL data volume permissions `sudo chown -R 70:70 /var/lib/docker/volumes/or_postgresql-data/_data` #### Existing instance 1. Create a new EBS volume from the EBS snapshot 1. Attach the new EBS volume to the existing EC2 instance using the AWS console or the `aws ec2 attach-volume` command 1. Check the EBS volume is attached as a disk using `lsblk` (assuming it is attached as `/dev/nvme2n1` for remaining steps) 1. Mount the EBS volume `sudo mount -t xfs -o nouuid /dev/nvme2n1 /mnt/snapshot` -2. Start a temporary PostgreSQL instance to connect to the snapshot DB:Locate the PostgreSQL data volume in `ls /mnt/snapshot` generally called `or_postgresql-data and set the `PGDATA` environment variable to the mount point. +1. Change PostgreSQL data volume permissions `sudo chown -R 70:70 /mnt/snapshot/or_postgresql-data/_data` +1. Start a temporary PostgreSQL instance to connect to the snapshot DB:Locate the PostgreSQL data volume in `ls /mnt/snapshot` generally called `or_postgresql-data and set the `PGDATA` environment variable to the mount point. ```shell sudo docker run -d \ --rm \