Add Yezzey submodule to gpcontrib by leborchuk · Pull Request #1752 · apache/cloudberry

leborchuk · 2026-05-18T14:56:48Z

Yezzey is an open-source extension for Apache Cloudberry and Greenplum 6 that transparently offloads Append-Only (AO/AOCO) table data to S3-compatible object storage. Inspired by Snowflake and AnyBlob, it extends the storage manager (smgr) so reads and writes go to S3 instead of local disk, keeping the user interface unchanged. A companion YProxy service acts as an I/O scheduler, managing connection pooling and request prioritization to prevent S3 throttling. Data is PGP-encrypted during upload. Benchmarks show only 10–43% query slowdown versus local storage, far outperforming PXF, making it ideal for cost-effective cold-data tiering.

The main feature of Yezzey - you don't need to change tables or code; just launch yezzey_define_offload_policy and move the data to S3. In this way, you can unload your cluster using the available disk space.

Currently, it is widely used on Greenplum 6 instances, and the goal is to provide users with the same interface in Cloudberry, so they can seamlessly migrate to Cloudberry.

We placed Yezzey as a submodule, as we believe one day we will replace all outdated solutions like AO/AOCO/Yezzey with PAX. However, that has not happened yet and we still need Yezzey.

Fixes #ISSUE_Number

What does this PR do?

Type of Change

Bug fix (non-breaking change)
New feature (non-breaking change)
Breaking change (fix or feature with breaking changes)
Documentation update

Breaking Changes

Test Plan

Unit tests added/updated
Integration tests added/updated
Passed make installcheck
Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

Followed contribution guide
Added/updated documentation
Reviewed code for security implications
Requested review from cloudberry committers

Additional Context

CI Skip Instructions

Yezzey is an open-source extension for Apache Cloudberry and Greenplum 6 that transparently offloads Append-Only (AO/AOCO) table data to S3-compatible object storage. Inspired by Snowflake and AnyBlob, it extends the storage manager (smgr) so reads and writes go to S3 instead of local disk, keeping the user interface unchanged. A companion YProxy service acts as an I/O scheduler, managing connection pooling and request prioritization to prevent S3 throttling. Data is PGP-encrypted during upload. Benchmarks show only 10–43% query slowdown versus local storage, far outperforming PXF, making it ideal for cost-effective cold-data tiering. The main feature of Yezzey - you don't need to change tables or code; just launch yezzey_define_offload_policy and move the data to S3. In this way, you can unload your cluster using the available disk space. Currently, it is widely used on Greenplum 6 instances, and the goal is to provide users with the same interface in Cloudberry, so they can seamlessly migrate to Cloudberry. We placed Yezzey as a submodule, as we believe one day we will replace all outdated solutions like AO/AOCO/Yezzey with PAX. However, that has not happened yet and we still need Yezzey.

leborchuk · 2026-05-18T14:59:59Z

It's the part of our roadmap, we discussed it #868

See the item Support Compute/Storage decouple by introducing Yezzey

tuhaihe · 2026-05-19T07:40:07Z

We also need to add the yezzey license info to the licenses dir and LICENSE, like other submodules. FYI.

pom.xml.

tuhaihe · 2026-05-19T07:45:45Z

For managing the new submodule, we can introduce it in the same way discussed here: #1084 (review)

leborchuk · 2026-05-19T12:20:19Z

We also need to add the yezzey license info to the licenses dir and LICENSE, like other submodules. FYI.

pom.xml.

fixed

leborchuk · 2026-05-19T13:24:17Z

For managing the new submodule, we can introduce it in the same way discussed here: #1084 (review)

Yes, I did as was described. The issue is that there is no stored tag information and the tag is an ephemeral entity shown only in the git submodule output. See

xifos@xifos-dev-jammy:~/git/cloudberry-leborchuk$ git submodule status
 0da57b85cf23e48d0e515f58c65a25425dbde012 contrib/pax_storage/src/cpp/contrib/googlebench (v1.9.2-3-g0da57b8)
 52204f78f94d7512df1f0f3bea1d47437a2c3a58 contrib/pax_storage/src/cpp/contrib/googletest (release-1.8.0-3536-g52204f78)
 3a58301067bbc03da89ae5a51b3e05b7da719d38 contrib/pax_storage/src/cpp/contrib/tabulate (v1.3-51-g3a58301)
 61c03f62b370b685b7994830b570a88d05ba15ab dependency/yyjson (0.10.0-20-g61c03f6)
 a09ea700d32bab83325aff9ff34d0582e50e3997 gpcontrib/gpcloud/test/googletest (release-1.8.0-2358-ga09ea700)
 0d88f66a5fd0dba82681eef5929529cb153cb325 gpcontrib/yezzey (1.8.8)

Here yezzey linked with 1.8.8 tag but the link info is still commit hash. If I delete tag git submodule just shows commit hash without tag info

reshke · 2026-05-20T14:56:54Z

Lgtm

tuhaihe · 2026-05-21T10:09:49Z

For managing the new submodule, we can introduce it in the same way discussed here: #1084 (review)

Yes, I did as was described. The issue is that there is no stored tag information and the tag is an ephemeral entity shown only in the git submodule output. See
xifos@xifos-dev-jammy:~/git/cloudberry-leborchuk$ git submodule status
 0da57b85cf23e48d0e515f58c65a25425dbde012 contrib/pax_storage/src/cpp/contrib/googlebench (v1.9.2-3-g0da57b8)
 52204f78f94d7512df1f0f3bea1d47437a2c3a58 contrib/pax_storage/src/cpp/contrib/googletest (release-1.8.0-3536-g52204f78)
 3a58301067bbc03da89ae5a51b3e05b7da719d38 contrib/pax_storage/src/cpp/contrib/tabulate (v1.3-51-g3a58301)
 61c03f62b370b685b7994830b570a88d05ba15ab dependency/yyjson (0.10.0-20-g61c03f6)
 a09ea700d32bab83325aff9ff34d0582e50e3997 gpcontrib/gpcloud/test/googletest (release-1.8.0-2358-ga09ea700)
 0d88f66a5fd0dba82681eef5929529cb153cb325 gpcontrib/yezzey (1.8.8)
Here yezzey linked with 1.8.8 tag but the link info is still commit hash. If I delete tag git submodule just shows commit hash without tag info

Hi, you're right. It looks good now. Here is my cmd list:

cd cloudberry/
git submodule add https://github.com/open-gpdb/yezzey.git gpcontrib/yezzey
git submodule status
cd gpcontrib/yezzey/
git fetch --tags
git tag --list
git checkout tags/1.8.8
cd ..
git add gpcontrib/yezzey/
git commit -m "Add yezzey to v1.8.8. xxxxx"

Never mind. We can add the version number info in the commit history when upgrading it next time.

tuhaihe · 2026-05-25T10:42:57Z

 fi

+#
+# yezzey


Hi, does yezzey need some extra dependencies for building? If so, we also need to add the pre-check for them when running configure --with-yezzey.

tuhaihe · 2026-05-25T10:43:35Z

 	recurse_targets += gp_stats_collector
 endif
+ifeq "$(with_yezzey)" "yes"
+        recurse_targets += yezzey


Suggested change

recurse_targets += yezzey

recurse_targets += yezzey

tuhaihe · 2026-05-25T10:47:28Z

+    services:
+      # Define the MinIO service container
+      minio:
+        image: lazybit/minio  # Use a specific MinIO image tag


Do we need to specify a tag for this image?

tuhaihe · 2026-05-25T10:48:35Z

+
+      - name: Install MinIO Client (mc)
+        run: |
+          set -ex pipefail


Suggested change

set -ex pipefail

set -exo pipefail

tuhaihe · 2026-05-25T10:55:02Z

+    name: Build and Test Yezzey Cloudberry
+    runs-on: ubuntu-latest
+    container:
+      image: apache/incubator-cloudberry:cbdb-build-ubuntu22.04-latest


Could we add the Rocky 8/9 docker image to the test matrix?

leborchuk marked this pull request as ready for review May 19, 2026 07:29

Copilot AI review requested due to automatic review settings May 19, 2026 07:29

Copilot started reviewing on behalf of leborchuk May 19, 2026 07:29 View session

This comment was marked as off-topic.

Sign in to view

Add yezzey license info

56a96bc

tuhaihe reviewed May 21, 2026

View reviewed changes

Comment thread configure Outdated

Leonid Borchuk and others added 2 commits May 21, 2026 13:20

Run autoconf to sync configure file

d379c9f

Merge branch 'main' into AddYezzey

47ad0c2

tuhaihe reviewed May 25, 2026

View reviewed changes

Conversation

leborchuk commented May 18, 2026

What does this PR do?

Type of Change

Breaking Changes

Test Plan

Impact

Checklist

Additional Context

CI Skip Instructions

Uh oh!

leborchuk commented May 18, 2026

Uh oh!

tuhaihe commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tuhaihe commented May 19, 2026

Uh oh!

This comment was marked as off-topic.

Uh oh!

leborchuk commented May 19, 2026

Uh oh!

leborchuk commented May 19, 2026

Uh oh!

reshke commented May 20, 2026

Uh oh!

Uh oh!

tuhaihe commented May 21, 2026

Uh oh!

tuhaihe May 25, 2026

Choose a reason for hiding this comment

Uh oh!

tuhaihe May 25, 2026

Choose a reason for hiding this comment

Uh oh!

tuhaihe May 25, 2026

Choose a reason for hiding this comment

Uh oh!

tuhaihe May 25, 2026

Choose a reason for hiding this comment

Uh oh!

tuhaihe May 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tuhaihe commented May 19, 2026 •

edited

Loading