From f2685296c48420937d091624030c47d9ad20c3f5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michal=20Salo=C5=88?= Date: Fri, 17 Jul 2026 11:59:16 +0200 Subject: [PATCH 1/2] Fix errors in shared-storage docs --- apps/docs/content/references/import.mdx | 6 ++-- .../content/shared-storage/how-to/backup.mdx | 14 +++++++++- .../content/shared-storage/how-to/connect.mdx | 27 ++++++++++++++---- .../content/shared-storage/how-to/create.mdx | 17 +++++++++++ .../content/shared-storage/how-to/use.mdx | 16 ++++++++--- apps/docs/content/shared-storage/overview.mdx | 2 +- .../content/shared-storage/tech-details.mdx | 28 +++++++++++++------ .../src/components/content/shared-storage.mdx | 3 +- apps/docs/static/data.json | 4 +-- 9 files changed, 90 insertions(+), 27 deletions(-) diff --git a/apps/docs/content/references/import.mdx b/apps/docs/content/references/import.mdx index 84c9e721a..e2d9d5259 100644 --- a/apps/docs/content/references/import.mdx +++ b/apps/docs/content/references/import.mdx @@ -128,7 +128,7 @@ services: override: false # REQUIRED. Name of your other service - hostname: teststorage1 - type: shared-storage + type: shared-storage:ha ... ``` @@ -322,7 +322,7 @@ services: # A second, simpler service example - hostname: teststorage1 - type: shared-storage + type: shared-storage:ha ``` This example includes all possible configuration options for Zerops services. Not all options are required or applicable to every service type. The example shows two services in the same YAML file: a fully configured Node.js API service and a simpler static frontend service. @@ -696,6 +696,8 @@ services: The `mount:` parameter allows you to mount a shared storage (which should be created inside the project) to the service. +The `buildFromGit` requirement exists because an import without it only creates an empty service shell with nothing deployed yet, so there is nothing to mount the storage into. With `buildFromGit`, the import also builds and deploys the repository, and the mount is applied to the deployed service. If you import a service without `buildFromGit`, deploy your application first and then connect the shared storage [in the GUI](/shared-storage/how-to/connect). + ### Service Nginx Configuration The nginx configuration defines the nginx settings for the service. diff --git a/apps/docs/content/shared-storage/how-to/backup.mdx b/apps/docs/content/shared-storage/how-to/backup.mdx index 524e1dc22..2a7939b7d 100644 --- a/apps/docs/content/shared-storage/how-to/backup.mdx +++ b/apps/docs/content/shared-storage/how-to/backup.mdx @@ -22,7 +22,19 @@ Shared Storage backups are created using tar archival: ### Restoring Backups -To restore a Shared Storage backup: Download the backup file (`.tar.gz`) from the Zerops UI, extract the archive, and upload the contents back to your Shared Storage. +To restore a Shared Storage backup: + +1. Download the backup file (`.tar.gz`) from the Zerops UI. +2. Transfer the archive into a runtime service that has the shared storage [connected](/shared-storage/how-to/connect), for example over the [Zerops VPN](/references/networking/vpn). +3. Extract the archive into the mount directory from within the runtime container: + +```sh +tar -xzf backup.tar.gz -C /mnt/ +``` + +:::note +Do not restore by uploading files through the Filer UI. Uploads there are limited to 64MB per file, so extracting the archive directly into the mounted directory is the reliable path. +::: ## Best Practices diff --git a/apps/docs/content/shared-storage/how-to/connect.mdx b/apps/docs/content/shared-storage/how-to/connect.mdx index b1d3af6c5..ca29e4a50 100644 --- a/apps/docs/content/shared-storage/how-to/connect.mdx +++ b/apps/docs/content/shared-storage/how-to/connect.mdx @@ -30,14 +30,29 @@ For existing storage, go to the shared storage service detail page and select ** To disconnect storage, access the shared storage service detail page, select **Shared storage connections**, and toggle OFF the desired runtime service. -{/* -## Connect using zsc +:::note +Connecting and disconnecting a shared storage is applied live; the runtime containers are not restarted. +::: -Connect shared storage from the command line with the `zsc connectSharedStorage` command. Specify one or more storage names as parameters: +## Mount and unmount using zsc + +You can also mount and unmount a shared storage from within a runtime container using the `zsc` command: ```sh -zsc connectSharedStorage sharedstorage0 -zsc connectSharedStorage sharedDisk secondDisk +# Mount a shared storage +zsc shared-storage mount + +# Unmount a shared storage +zsc shared-storage unmount + +# Wait until the storage mount is ready +zsc shared-storage wait ``` -Run this command from within your runtime container via web terminal, SSH, or in your `zerops.yaml` file. For more details, see the [Zerops Setup Control documentation](/references/zsc).*/} \ No newline at end of file +Run these commands from your runtime container via web terminal or SSH, or as part of your `zerops.yaml` commands. For more details, see the [zsc reference](/references/zsc#shared-storage). + +:::caution +The shared storage must be mounted (through the GUI connection toggle or via `zsc shared-storage mount`) before your application can access the `/mnt/` directory. + +Before stopping or deleting a shared storage service, you must first disconnect (unmount) it from all runtime services. +::: \ No newline at end of file diff --git a/apps/docs/content/shared-storage/how-to/create.mdx b/apps/docs/content/shared-storage/how-to/create.mdx index 4ac9a20db..c6db14234 100644 --- a/apps/docs/content/shared-storage/how-to/create.mdx +++ b/apps/docs/content/shared-storage/how-to/create.mdx @@ -84,6 +84,23 @@ zCLI is the Zerops command-line tool. To create a new Shared Storage service via 2. Create a project description file 3. Create a project with a runtime and a Shared Storage service +### Import YAML + +The deployment mode is part of the service type: + +```yaml +services: + - # Shared storage service in highly available mode + hostname: teststorage + type: shared-storage:ha +``` + +Use `shared-storage:single` for the single container mode. + +:::note +You can also pin the underlying engine version directly with `seaweedfs:ha@3` or `seaweedfs:single@3`. +::: + ### Choose Your Runtime diff --git a/apps/docs/content/shared-storage/how-to/use.mdx b/apps/docs/content/shared-storage/how-to/use.mdx index cacf65cc4..96b6e9979 100644 --- a/apps/docs/content/shared-storage/how-to/use.mdx +++ b/apps/docs/content/shared-storage/how-to/use.mdx @@ -26,7 +26,7 @@ If you connect multiple runtimes, the content of the folder will be shared among - Multiple storage volumes can be mounted to a single service (e.g., `/mnt/files1`, `/mnt/files2`, etc.) - Shared storage mount is only available in runtime containers, not during build and prepare runtime phases -- All filesystem operations are automatically logged to runtime logs +- The mount process logs its output to the runtime service logs (tagged `zerops-mount-`) For technical details about mount behavior and filesystem capabilities, see the [Technical Details](/shared-storage/tech-details#mount-integration) page. @@ -34,11 +34,19 @@ For technical details about mount behavior and filesystem capabilities, see the Shared Storage is ideal for: -- **Persistent filesystem-based databases**: SQLite, Prometheus DB, etc. - **Configuration sharing**: Deploy configurations once and share across multiple services - Example: Deploy Apache Airflow configurations and DAG files once and share with all worker nodes -- **Alternative to object storage**: For applications that require filesystem semantics rather than object storage -- **Application data**: Store and serve images, documents, and other assets +- **Application data**: Store and serve user uploads, images, documents, and other assets (avoid huge trees of tiny files, see [performance considerations](#performance-considerations)) +- **Fallback to object storage**: For applications that strictly require filesystem semantics and cannot work with [object storage](/object-storage/overview). Be aware of the [limitations](/shared-storage/tech-details#performance-considerations) before choosing this path. + +:::caution Not suitable for databases +Do **not** run databases on Shared Storage. This includes SQLite, Prometheus TSDB, and other filesystem-based databases. + +- File locks are enforced **per mount only**: a lock held in one container does not prevent another container from writing to the same file, which leads to data corruption. +- The underlying storage engine is an append-only chunk store: every modification writes new chunks, so the frequent small writes typical for databases cause heavy write amplification. + +Use [managed databases](/postgresql/overview) instead. For storing database *backups*, prefer [object storage](/object-storage/overview): it is cheaper and better suited for write-once files. +::: ## Performance Considerations diff --git a/apps/docs/content/shared-storage/overview.mdx b/apps/docs/content/shared-storage/overview.mdx index f1cbb1fed..6da6c6dd8 100644 --- a/apps/docs/content/shared-storage/overview.mdx +++ b/apps/docs/content/shared-storage/overview.mdx @@ -13,7 +13,7 @@ import LargeCard from '@site/src/components/LargeCard'; Zerops provides a fully managed and scaled **Shared Storage** service, which can be mounted to your runtime services. It offers: - Persistent file sharing between containers of the same service or different services - Standard filesystem operations through a POSIX-compatible interface -- Built-in high-availability configuration +- Optional highly available mode for production workloads ## Documentation Sections diff --git a/apps/docs/content/shared-storage/tech-details.mdx b/apps/docs/content/shared-storage/tech-details.mdx index 0b4b07d4e..165891fab 100644 --- a/apps/docs/content/shared-storage/tech-details.mdx +++ b/apps/docs/content/shared-storage/tech-details.mdx @@ -14,15 +14,19 @@ Shared Storage consists of three main components: - **Volume Servers**: Store the actual file data - **Filer**: Provides a POSIX-compatible interface for file operations -An **automatic vacuum process** helps maintain optimal storage performance by reclaiming space from deleted files. This process is triggered when the size of deleted content exceeds 15% (reduced from the default 30%). +### Storage Engine Behavior + +SeaweedFS stores file data in append-only volumes. Files are split into chunks, and when a file is modified, new chunks are written while the old ones remain on disk until they are reclaimed. Frequent small modifications of existing files therefore cause significant write amplification, which is one of the reasons Shared Storage is [not suitable for databases](/shared-storage/how-to/use#use-cases). + +An **automatic vacuum process** reclaims the space occupied by deleted and superseded chunks. This process is triggered when the size of deleted content exceeds 15% (reduced from the SeaweedFS default of 30%). ### Mount Integration When connected to a runtime service: - Storage is mounted at `/mnt/` - Mount point is owned by the `zerops` user and group (no sudo required) -- All filesystem operations are logged to runtime logs (tagged as `zerops-mount-`) -- Mounting will overwrite any existing content in the mount directory +- The mount process logs its standard output and error to the runtime service logs (tagged as `zerops-mount-`); individual file operations are not logged +- Mounting removes any existing content of the mount directory - Shared storage mount is only available in runtime containers, not during build and prepare runtime phases ## Deployment Modes @@ -33,8 +37,8 @@ Zerops provides Shared Storage in two deployment modes: Recommended for production environments where data reliability is critical. -- **Architecture**: 2 volume servers with the master located on one of them -- **Data Durability**: Data and filer metadata are replicated 1:1 across nodes +- **Architecture**: Two containers, each running its own filer and volume server; the master server runs on the first container +- **Data Durability**: Data and filer metadata are replicated 1:1 across both containers - **Fault Tolerance**: - If a node fails, an automatic repair process begins - A new container replaces the failed one @@ -59,21 +63,27 @@ Shared Storage supports standard POSIX filesystem operations: - Create, read, update, and delete files and directories - Set permissions (with some limitations) -- Standard file locking operations +- File locking (per mount only, see the warning below) - Hard and symbolic links - Directory listing and traversal For a complete list of supported features, see the [SeaweedFS FUSE documentation](https://github.com/seaweedfs/seaweedfs/wiki/FUSE-Mount#supported-features). +:::caution File locks are per-mount only +File locks (`flock`, POSIX locks) are enforced only within a single container's mount. A process in another container can freely write to a file you hold locked. Do not rely on file locking for coordination between containers, and do not run [lock-dependent workloads such as databases](/shared-storage/how-to/use#use-cases) on Shared Storage. +::: + ## Resource Constraints ### Storage Limits - Maximum storage space: 60GB (can be increased via support request) -- Maximum file size: Unlimited within the 60GB total storage constraint +- Maximum file size: No fixed limit through the mount (files are stored in chunks), up to the available storage space - Maximum upload size via Filer UI: 64MB -### Memory Usage +### Mount Process Memory Usage + +The mount process runs inside each connected runtime container and consumes its memory: - Base memory consumption: ~60MB when idle - Peak memory usage: ~150MB under higher filesystem loads @@ -87,7 +97,7 @@ For a complete list of supported features, see the [SeaweedFS FUSE documentation ## Auto Scaling Configuration -Zerops scales Shared Storage services automatically by raising or lowering the hardware resources of each database container. +Zerops scales Shared Storage services automatically by raising or lowering the hardware resources of each container. Vertical auto scaling has the following default configuration: diff --git a/apps/docs/src/components/content/shared-storage.mdx b/apps/docs/src/components/content/shared-storage.mdx index 2bc3865c8..2b2062c1f 100644 --- a/apps/docs/src/components/content/shared-storage.mdx +++ b/apps/docs/src/components/content/shared-storage.mdx @@ -13,8 +13,7 @@ project: services: - # Shared storage service hostname: teststorage - type: shared-storage - mode: NON_HA + type: shared-storage:single - # {{serviceDisplay}} service hostname: app diff --git a/apps/docs/static/data.json b/apps/docs/static/data.json index fc9ba4a52..9dfbb9876 100644 --- a/apps/docs/static/data.json +++ b/apps/docs/static/data.json @@ -321,11 +321,11 @@ "readable": ["25.3"] }, "seaweedfs": { - "import": [["seaweedfs@3"]], + "import": [["seaweedfs:single@3", "seaweedfs:ha@3"]], "readable": ["3"] }, "sharedstorage": { - "import": [["shared-storage"]] + "import": [["shared-storage:single", "shared-storage:ha"]] }, "objectstorage": { "import": [["object-storage"]] From 6b2a35532c158fd7c4577dd845afd812c41996af Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michal=20Salo=C5=88?= Date: Fri, 17 Jul 2026 12:19:08 +0200 Subject: [PATCH 2/2] Refactor Shared storage section --- apps/docs/content/features/backup.mdx | 2 +- .../content/shared-storage/how-to/backup.mdx | 47 ------ .../content/shared-storage/how-to/connect.mdx | 42 ++++-- .../content/shared-storage/how-to/create.mdx | 57 +++---- .../content/shared-storage/how-to/manage.mdx | 64 ++++++-- .../content/shared-storage/how-to/use.mdx | 64 -------- apps/docs/content/shared-storage/overview.mdx | 141 ++++++++++++------ .../content/shared-storage/tech-details.mdx | 66 ++++---- apps/docs/sidebars.js | 14 +- 9 files changed, 230 insertions(+), 267 deletions(-) delete mode 100644 apps/docs/content/shared-storage/how-to/backup.mdx delete mode 100644 apps/docs/content/shared-storage/how-to/use.mdx diff --git a/apps/docs/content/features/backup.mdx b/apps/docs/content/features/backup.mdx index e8abbe8dd..0a2331c63 100644 --- a/apps/docs/content/features/backup.mdx +++ b/apps/docs/content/features/backup.mdx @@ -7,7 +7,7 @@ Zerops provides an automated, secure backup system for supported services. This ## Supported Services -Zerops provides automated backup functionality for the following services. For specific backup format details and restore instructions, visit each service's documentation: [MariaDB](/mariadb/how-to/backup), [PostgreSQL](/postgresql/how-to/manage#backups), [Qdrant](/qdrant/overview), [Elasticsearch](/elasticsearch/overview), [NATS](/nats/overview), [Meilisearch](/meilisearch/overview), and [Shared Storage](/shared-storage/how-to/backup). +Zerops provides automated backup functionality for the following services. For specific backup format details and restore instructions, visit each service's documentation: [MariaDB](/mariadb/how-to/backup), [PostgreSQL](/postgresql/how-to/manage#backups), [Qdrant](/qdrant/overview), [Elasticsearch](/elasticsearch/overview), [NATS](/nats/overview), [Meilisearch](/meilisearch/overview), and [Shared Storage](/shared-storage/how-to/manage#backups). ## Managing Backups in the UI diff --git a/apps/docs/content/shared-storage/how-to/backup.mdx b/apps/docs/content/shared-storage/how-to/backup.mdx deleted file mode 100644 index 2a7939b7d..000000000 --- a/apps/docs/content/shared-storage/how-to/backup.mdx +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: Backup Shared Storage -description: Learn how to backup and restore your Shared Storage data in Zerops. ---- - -Zerops provides automated data backup for Shared Storage services with full encryption and flexible management options. - -For general backup information including configuration, scheduling, and management options, see the [Zerops Backups](/features/backup) documentation, which covers: -- Backup scheduling and retention policies -- Tagging system and storage quotas -- Manual backup creation and CLI tools -- Security and encryption details - -## Shared Storage Backup Format - -Shared Storage backups are created using tar archival: - -- **Format**: `.tar.gz` (archive of directory contents) -- **Tooling**: `tar` -- **Content**: Compresses the entire shared volume (`/mnt/`) -- **Storage**: Encrypted and stored in isolated object storage - -### Restoring Backups - -To restore a Shared Storage backup: - -1. Download the backup file (`.tar.gz`) from the Zerops UI. -2. Transfer the archive into a runtime service that has the shared storage [connected](/shared-storage/how-to/connect), for example over the [Zerops VPN](/references/networking/vpn). -3. Extract the archive into the mount directory from within the runtime container: - -```sh -tar -xzf backup.tar.gz -C /mnt/ -``` - -:::note -Do not restore by uploading files through the Filer UI. Uploads there are limited to 64MB per file, so extracting the archive directly into the mounted directory is the reliable path. -::: - -## Best Practices - -For Shared Storage backups: -- Regularly clean up unnecessary files from your Shared Storage to reduce backup size -- Create manual backups with protected tags before major file operations or migrations -- Consider adjusting your backup frequency for optimal storage usage -- Test your restore process periodically to ensure data integrity - -For additional best practices and troubleshooting, refer to the [main backup documentation](/features/backup). \ No newline at end of file diff --git a/apps/docs/content/shared-storage/how-to/connect.mdx b/apps/docs/content/shared-storage/how-to/connect.mdx index ca29e4a50..faf05110b 100644 --- a/apps/docs/content/shared-storage/how-to/connect.mdx +++ b/apps/docs/content/shared-storage/how-to/connect.mdx @@ -1,34 +1,50 @@ --- -title: Connect shared storage to a runtime service -description: Learn how to connect shared storage to other services in Zerops. +title: Connect & mount shared storage +description: Learn how to connect shared storage to runtime services in Zerops and how the mount behaves. --- -import GroupCards from '@site/src/components/GroupCards'; +This page covers how to connect an existing shared storage to runtime services, how to disconnect it, and how the mount behaves inside your containers. -This page covers how to connect an existing shared storage to runtime services and how to disconnect services when needed. +## How the mount behaves -## In Zerops GUI +Once a shared storage is connected to a runtime service, Zerops mounts it at `/mnt/` in the runtime containers. For example, `/mnt/teststorage` for a `teststorage` shared storage: -### Connect a new shared storage +
+ Mounted shared storage in the file browser +
-When creating a new shared storage service, you can directly select which runtime services it should be connected to. See [Create Shared Storage](/shared-storage/how-to/create) for details about the creation process. +- The content is shared among **all containers** of the connected runtime service; if you connect multiple runtime services, they all see the same content +- Multiple storages can be mounted to a single service (e.g. `/mnt/files1`, `/mnt/files2`) +- The mount point is owned by the `zerops` user and group (no sudo required) +- Mounting removes any existing content of the mount directory +- The mount is only available in runtime containers, not during build and prepare runtime phases +- The mount process logs its standard output and error to the runtime service logs (tagged `zerops-mount-`); individual file operations are not logged -### Connect an existing shared storage +:::note +The `df` command may show incorrect or misleading information for shared storage mounts. Refer to the Zerops GUI for accurate storage metrics. +::: + +## Connect in Zerops GUI + +When creating a new shared storage service, you can directly select which runtime services it should be connected to. See [Create shared storage](/shared-storage/how-to/create) for the creation process. For existing storage, go to the shared storage service detail page and select **Shared storage connections**. Toggle ON any runtime services you wish to connect to this storage.
Connecting shared storage during creation
-## Disconnect a shared storage in Zerops GUI - -To disconnect storage, access the shared storage service detail page, select **Shared storage connections**, and toggle OFF the desired runtime service. +To disconnect storage, toggle OFF the desired runtime service in the same place. :::note Connecting and disconnecting a shared storage is applied live; the runtime containers are not restarted. @@ -55,4 +71,4 @@ Run these commands from your runtime container via web terminal or SSH, or as pa The shared storage must be mounted (through the GUI connection toggle or via `zsc shared-storage mount`) before your application can access the `/mnt/` directory. Before stopping or deleting a shared storage service, you must first disconnect (unmount) it from all runtime services. -::: \ No newline at end of file +::: diff --git a/apps/docs/content/shared-storage/how-to/create.mdx b/apps/docs/content/shared-storage/how-to/create.mdx index c6db14234..921aafdca 100644 --- a/apps/docs/content/shared-storage/how-to/create.mdx +++ b/apps/docs/content/shared-storage/how-to/create.mdx @@ -6,11 +6,9 @@ description: Learn how to create shared storage which you can use with your othe import GroupCards from '@site/src/components/GroupCards'; import Video from '@site/src/components/Video'; -Shared Storage provides persistent file storage that can be mounted as a POSIX-compatible filesystem to your runtime services. Built on [SeaweedFS ↗](https://github.com/seaweedfs/seaweedfs), it enables reliable data persistence and sharing across services in your infrastructure. - ## Create Using Zerops GUI -First, set up a project in Zerops GUI and add a runtime service. Then go to the project dashboard page and choose **Add new service** in the left menu in the **Services** block. Then add a new Shared Storage service: +Go to your project dashboard and choose **Add new service** in the left menu in the **Services** block. Then add a new Shared Storage service: