From 0c5baff24bcf4ad530e603cb1c4359855a39390c Mon Sep 17 00:00:00 2001 From: Christian Winter Date: Wed, 22 Jul 2026 17:52:07 +0200 Subject: [PATCH 1/2] Add information about general purpose compression and update layout of compression info system tables --- content/compatibility/system_table.md | 75 +++++++++++++++++++-------- content/references/objects/tables.md | 20 +++++++ 2 files changed, 74 insertions(+), 21 deletions(-) diff --git a/content/compatibility/system_table.md b/content/compatibility/system_table.md index 3ebe9c46..fd8a56ce 100644 --- a/content/compatibility/system_table.md +++ b/content/compatibility/system_table.md @@ -105,35 +105,42 @@ This system table contains information about how tables and columns are compress Note that CedarDB can use different compression schemes within the same column and that this table currently only includes statistics on cold data. For more information on cold and hot data, see [this blog post](https://cedardb.com/blog/colibri/). -| Column | Type | Description | -|---------------------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| oid | Integer | The [Object Identifier](https://www.postgresql.org/docs/current/datatype-oid.html) of the table. | -| tablename | Text | The name of the table. | -| attributename | Text | The name of the attribute. | -| datatype | Text | The type of the attribute. | -| encoding | Text | The encoding scheme used for compression. | -| compressedvaluesize | Text | The maximum number of bytes required to encode a compressed value. For instance, if a dictionary has at most 256 keys, each value can be encoded using just one byte. | -| compressedsize | Bigint | The size of the compressed data in bytes. | -| uncompressedsize | Bigint | The size of the uncompressed data in bytes. For strings, this includes additional meta data to be able to query the data. | -| tuplecount | Bigint | The number of compressed tuples. | +| Column | Type | Description | +| ------------------ | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| oid | Integer | The [Object Identifier](https://www.postgresql.org/docs/current/datatype-oid.html) of the table. | +| tablename | Text | The name of the table. | +| attributename | Text | The name of the attribute. | +| datatype | Text | The type of the attribute. | +| encoding | Text | The lightweight encoding scheme used for this value, e.g. dictionary or frame-of-reference encoding. | +| generalcompression | Text | The general-purpose compression method applied on top of the lightweight encoding. Currently either `none` or `zstd` (see [compression option](/docs/references/objects/tables#options)). | +| encodedvaluewidth | Text | The maximum number of bytes required to encode a value using the lightweight encoding. For instance, if a dictionary has at most 256 keys, each value can be encoded using just one byte. | +| uncompressedsize | Bigint | The size of the uncompressed data in bytes. For strings, this includes additional meta data to be able to query the data. | +| encodedsize | Bigint | The size of the data in bytes after the lightweight encoding, before any general-purpose compression is applied. | +| compressedsize | Bigint | The size of the data on disk in bytes, after both the lightweight encoding and the general-purpose compression have been applied. | +| tuplecount | Bigint | The number of encoded tuples. | This is an excerpt of the output for [TPCH](https://www.tpc.org/tpch/) with scale factor 1: ```text -oid tablename attributename datatype encoding compressedvaluesize compressedsize uncompressedsize tuplecount +oid tablename attributename datatype encoding generalcompression encodedvaluewidth uncompressedsize encodedsize compressedsize tuplecount ----- -268435460 partsupp ps_comment text simple dictionary fourbyte 104230496 107284124 768830 -268435460 partsupp ps_supplycost numeric truncate fourbyte 3075344 6150640 768830 -268435460 partsupp ps_availqty integer truncate twobyte 1537680 3075320 768830 -268435460 partsupp ps_suppkey integer truncate twobyte 1537680 3075320 768830 -268435460 partsupp tid bigint increment zerobytes 0 6150640 768830 -268435460 partsupp ps_partkey integer frameofreference twobyte 1537680 3075320 768830 -268435462 orders tid bigint increment zerobytes 0 12000000 1500000 -268435462 orders o_clerk char sortedstring dictionary twobyte 3114096 46495400 1500000 -268435462 orders o_comment text simple dictionary fourbyte 90643872 96751711 1500000 +268435460 partsupp ps_comment text simple dictionary zstd fourbyte 106775896 103736736 27070464 765159 +268435460 partsupp ps_comment text simple dictionary zstd twobyte 4855777 4647024 1196032 34841 +268435460 partsupp ps_supplycost numeric truncate zstd fourbyte 6400000 3200016 2142208 800000 +268435460 partsupp ps_availqty int32 truncate zstd twobyte 3200000 1600032 1474560 800000 +268435460 partsupp ps_suppkey int32 truncate zstd twobyte 3200000 1600032 618496 800000 +268435460 partsupp ps_partkey int32 frameofreferencetinyblocks zstd eightbyte 3200000 891312 20480 800000 +268435460 partsupp tid bigint increment none zerobytes 6400000 0 0 800000 +268435462 orders o_clerk char sortedstring dictionary zstd twobyte 46484672 3114096 2265088 1500000 +268435462 orders o_comment text simple dictionary zstd fourbyte 96751727 90643888 29347840 1500000 +268435462 orders tid bigint increment none zerobytes 12000000 0 0 1500000 ... ``` +Note the two rows for `ps_comment`: CedarDB split this column across two differently-encoded blocks during loading, illustrating that a single column can use different encoding and compression schemes at once for different parts of its data. +Also note that `generalcompression` is only `zstd` when applying it reduces the on-disk size by a large enough margin. +Otherwise CedarDB skips the extra compression pass and stores the lightweight-encoded data as is (`generalcompression = none`, `compressedsize = encodedsize`), as is the case for `tid` here. + ## System Views System views provide convenient access to system information. @@ -182,6 +189,32 @@ The views instead use more human-readable symbolic names. #### cedardb_compression_infos This system view gives an easier usable representation of the compression ratio of the entries in the `cedardb_compression_info` system table. +It groups by `oid`, `tablename`, `attributename`, `datatype`, `encoding`, and `generalcompression`, and adds the following ratio columns: + +| Column | Type | Description | +| ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------- | +| encoding_ratio | Numeric | `uncompressedsize / encodedsize`: the compression ratio achieved by the lightweight encoding alone. | +| compression_ratio | Numeric | `encodedsize / compressedsize`: the additional compression ratio achieved by the general-purpose codec (e.g. zstd). | +| total_ratio | Numeric | `uncompressedsize / compressedsize`: the combined compression ratio. | + +This is an excerpt of the output for the same TPCH scale factor 1 tables shown above: + +```text +oid tablename attributename datatype encoding generalcompression encoding_ratio compression_ratio total_ratio tuplecount +----- +268435460 partsupp ps_comment text simple dictionary zstd 1.03 3.83 3.95 800000 +268435460 partsupp ps_supplycost numeric truncate zstd 2.00 1.49 2.99 800000 +268435460 partsupp ps_availqty int32 truncate zstd 2.00 1.09 2.17 800000 +268435460 partsupp ps_suppkey int32 truncate zstd 2.00 2.59 5.17 800000 +268435460 partsupp ps_partkey int32 frameofreferencetinyblocks zstd 3.59 43.52 156.25 800000 +268435460 partsupp tid bigint increment none NULL NULL NULL 800000 +268435462 orders o_clerk char sortedstring dictionary zstd 14.93 1.37 20.52 1500000 +268435462 orders o_comment text simple dictionary zstd 1.07 3.09 3.30 1500000 +268435462 orders tid bigint increment none NULL NULL NULL 1500000 +... +``` + +Note that `ps_comment`'s two rows in `cedardb_compression_info` are collapsed into a single row here, and that ratios are `NULL` when a column has an `encodedsize` or `compressedsize` of `0` (e.g. `increment`-encoded columns, which need no storage at all). ## Information Schema diff --git a/content/references/objects/tables.md b/content/references/objects/tables.md index eab9a5e1..8a2de27d 100644 --- a/content/references/objects/tables.md +++ b/content/references/objects/tables.md @@ -115,6 +115,15 @@ Create a table that stores all compressed data on a remote server previously cre CREATE TABLE remote_species (...) WITH (server = remote_storage); ``` +Set the compression method used for cold on-disk data, in addition to CedarDB's lightweight +encoding (see the `cedardb_compression_info` [system table](/docs/compatibility/system_table) for per-column +compression statistics). Supported values are `zstd` (the default) and `none`: + +```sql +CREATE TABLE species (...) WITH (compression = zstd); +CREATE TABLE species_uncompressed (...) WITH (compression = none); +``` + ### Identity Columns Identity columns automatically generate unique integer values: @@ -326,6 +335,17 @@ Drop a constraint and cascade to dependent constraints: ALTER TABLE child_table DROP CONSTRAINT fk_constraint CASCADE; ``` +### Storage Options + +Change the general-purpose compression codec used for on-disk column data. +Supported values are `zstd` (the default) and `none`. +This takes effect the next time affected data is written to disk: + +```sql +ALTER TABLE species SET (compression = zstd); +ALTER TABLE species SET (compression = none); +``` + ### Ownership ```sql From 710a5a19b58da2bfa46c08c4271b237259822f04 Mon Sep 17 00:00:00 2001 From: Christian Winter Date: Wed, 22 Jul 2026 18:58:12 +0200 Subject: [PATCH 2/2] Update pricing links to direct to get started instead --- content/community_edition.md | 2 +- layouts/partials/footer.html | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/community_edition.md b/content/community_edition.md index c04e7081..9e1fc95b 100644 --- a/content/community_edition.md +++ b/content/community_edition.md @@ -38,7 +38,7 @@ The **CedarDB Community Edition** It is **the right choice** for everything from solo projects to small-scale production workloads. No strings attached! {{% callout type="info" %}} -Looking for advanced features or enterprise support? Check out the **Enterprise Edition** or visit our [pricing page](https://cedardb.com/pricing/). +Looking for advanced features or enterprise support? Check out the **Enterprise Edition** or visit our [get started page](https://cedardb.com/get-started/). {{% /callout %}} | | Community Edition | Enterprise Edition | diff --git a/layouts/partials/footer.html b/layouts/partials/footer.html index 227a1a44..b5e7d374 100644 --- a/layouts/partials/footer.html +++ b/layouts/partials/footer.html @@ -11,7 +11,7 @@

Company

- Pricing + Get Started About Us Blog