Add information about general purpose compression

content/community_edition.md

@@ -38,7 +38,7 @@ The **CedarDB Community Edition**
 It is **the right choice** for everything from solo projects to small-scale production workloads. No strings attached!
 
 {{% callout type="info" %}}
-Looking for advanced features or enterprise support? Check out the **Enterprise Edition** or visit our [pricing page](https://cedardb.com/pricing/).
+Looking for advanced features or enterprise support? Check out the **Enterprise Edition** or visit our [get started page](https://cedardb.com/get-started/).
 {{% /callout %}}
 
 |                              | Community Edition                                                                                                 | Enterprise Edition        |

content/compatibility/system_table.md

@@ -105,35 +105,42 @@
 Note that CedarDB can use different compression schemes within the same column and that this table currently only includes statistics on cold data.
 For more information on cold and hot data, see [this blog post](https://cedardb.com/blog/colibri/).
 
-| Column              | Type    | Description                                                                                                                                                           |
-|---------------------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| oid                 | Integer | The [Object Identifier](https://www.postgresql.org/docs/current/datatype-oid.html)  of the table.                                                                     |
-| tablename           | Text    | The name of the table.                                                                                                                                                |
-| attributename       | Text    | The name of the attribute.                                                                                                                                            |
-| datatype            | Text    | The type of the attribute.                                                                                                                                            |
-| encoding            | Text    | The encoding scheme used for compression.                                                                                                                             |
-| compressedvaluesize | Text    | The maximum number of bytes required to encode a compressed value. For instance, if a dictionary has at most 256 keys, each value can be encoded using just one byte. |
-| compressedsize      | Bigint  | The size of the compressed data in bytes.                                                                                                                             |
-| uncompressedsize    | Bigint  | The size of the uncompressed data in bytes. For strings, this includes additional meta data to be able to query the data.                                             |
-| tuplecount          | Bigint  | The number of compressed tuples.                                                                                                                                      |
+| Column             | Type    | Description                                                                                                                                                                               |
+| ------------------ | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| oid                | Integer | The [Object Identifier](https://www.postgresql.org/docs/current/datatype-oid.html) of the table.                                                                                          |
+| tablename          | Text    | The name of the table.                                                                                                                                                                    |
+| attributename      | Text    | The name of the attribute.                                                                                                                                                                |
+| datatype           | Text    | The type of the attribute.                                                                                                                                                                |
+| encoding           | Text    | The lightweight encoding scheme used for this value, e.g. dictionary or frame-of-reference encoding.                                                                                      |
+| generalcompression | Text    | The general-purpose compression method applied on top of the lightweight encoding. Currently either `none` or `zstd` (see [compression option](/docs/references/objects/tables#options)). |
+| encodedvaluewidth  | Text    | The maximum number of bytes required to encode a value using the lightweight encoding. For instance, if a dictionary has at most 256 keys, each value can be encoded using just one byte. |
+| uncompressedsize   | Bigint  | The size of the uncompressed data in bytes. For strings, this includes additional meta data to be able to query the data.                                                                 |
+| encodedsize        | Bigint  | The size of the data in bytes after the lightweight encoding, before any general-purpose compression is applied.                                                                          |
+| compressedsize     | Bigint  | The size of the data on disk in bytes, after both the lightweight encoding and the general-purpose compression have been applied.                                                         |
+| tuplecount         | Bigint  | The number of encoded tuples.                                                                                                                                                             |
 
 This is an excerpt of the output for [TPCH](https://www.tpc.org/tpch/) with scale factor 1:
 
 ```text
-oid         tablename   attributename   datatype    encoding                compressedvaluesize compressedsize  uncompressedsize    tuplecount
+oid         tablename   attributename   datatype    encoding                    generalcompression  encodedvaluewidth   uncompressedsize    encodedsize     compressedsize  tuplecount
 -----
-268435460   partsupp    ps_comment      text        simple dictionary       fourbyte            104230496       107284124           768830
-268435460   partsupp    ps_supplycost   numeric     truncate                fourbyte            3075344         6150640             768830
-268435460   partsupp    ps_availqty     integer     truncate                twobyte             1537680         3075320             768830
-268435460   partsupp    ps_suppkey      integer     truncate                twobyte             1537680         3075320             768830
-268435460   partsupp    tid             bigint      increment               zerobytes           0               6150640             768830
-268435460   partsupp    ps_partkey      integer     frameofreference        twobyte             1537680         3075320             768830
-268435462   orders      tid             bigint      increment               zerobytes           0               12000000            1500000
-268435462   orders      o_clerk         char        sortedstring dictionary twobyte             3114096         46495400            1500000
-268435462   orders      o_comment       text        simple dictionary       fourbyte            90643872        96751711            1500000
+268435460   partsupp    ps_comment      text        simple dictionary          zstd                fourbyte            106775896           103736736       27070464        765159
+268435460   partsupp    ps_comment      text        simple dictionary          zstd                twobyte             4855777             4647024         1196032         34841
+268435460   partsupp    ps_supplycost   numeric     truncate                    zstd                fourbyte            6400000             3200016         2142208         800000
+268435460   partsupp    ps_availqty     int32       truncate                    zstd                twobyte             3200000             1600032         1474560         800000
+268435460   partsupp    ps_suppkey      int32       truncate                    zstd                twobyte             3200000             1600032         618496          800000
+268435460   partsupp    ps_partkey      int32       frameofreferencetinyblocks  zstd                eightbyte           3200000             891312          20480           800000
+268435460   partsupp    tid             bigint      increment                   none                zerobytes           6400000             0               0               800000
+268435462   orders      o_clerk         char        sortedstring dictionary     zstd                twobyte             46484672            3114096         2265088         1500000
+268435462   orders      o_comment       text        simple dictionary          zstd                fourbyte            96751727            90643888        29347840        1500000
+268435462   orders      tid             bigint      increment                   none                zerobytes           12000000            0               0               1500000
 ...
 ```
 
+Note the two rows for `ps_comment`: CedarDB split this column across two differently-encoded blocks during loading, illustrating that a single column can use different encoding and compression schemes at once for different parts of its data.
+Also note that `generalcompression` is only `zstd` when applying it reduces the on-disk size by a large enough margin.
+Otherwise CedarDB skips the extra compression pass and stores the lightweight-encoded data as is (`generalcompression = none`, `compressedsize = encodedsize`), as is the case for `tid` here.
+
 ## System Views
 
 System views provide convenient access to system information.
@@ -182,6 +189,32 @@
 #### cedardb_compression_infos
 
 This system view gives an easier usable representation of the compression ratio of the entries in the `cedardb_compression_info` system table.
+It groups by `oid`, `tablename`, `attributename`, `datatype`, `encoding`, and `generalcompression`, and adds the following ratio columns:
+
+| Column            | Type    | Description                                                                                                         |
+| ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------- |
+| encoding_ratio    | Numeric | `uncompressedsize / encodedsize`: the compression ratio achieved by the lightweight encoding alone.                 |
+| compression_ratio | Numeric | `encodedsize / compressedsize`: the additional compression ratio achieved by the general-purpose codec (e.g. zstd). |
+| total_ratio       | Numeric | `uncompressedsize / compressedsize`: the combined compression ratio.                                                |
+
+This is an excerpt of the output for the same TPCH scale factor 1 tables shown above:
+
+```text
+oid         tablename   attributename   datatype    encoding                    generalcompression  encoding_ratio  compression_ratio  total_ratio  tuplecount
+-----
+268435460   partsupp    ps_comment      text        simple dictionary          zstd                1.03            3.83                3.95         800000
+268435460   partsupp    ps_supplycost   numeric     truncate                    zstd                2.00            1.49                2.99         800000
+268435460   partsupp    ps_availqty     int32       truncate                    zstd                2.00            1.09                2.17         800000
+268435460   partsupp    ps_suppkey      int32       truncate                    zstd                2.00            2.59                5.17         800000
+268435460   partsupp    ps_partkey      int32       frameofreferencetinyblocks  zstd                3.59            43.52               156.25       800000
+268435460   partsupp    tid             bigint      increment                   none                NULL            NULL                NULL         800000
+268435462   orders      o_clerk         char        sortedstring dictionary     zstd                14.93           1.37                20.52        1500000
+268435462   orders      o_comment       text        simple dictionary          zstd                1.07            3.09                3.30         1500000
+268435462   orders      tid             bigint      increment                   none                NULL            NULL                NULL         1500000
+...
+```
+
+Note that `ps_comment`'s two rows in `cedardb_compression_info` are collapsed into a single row here, and that ratios are `NULL` when a column has an `encodedsize` or `compressedsize` of `0` (e.g. `increment`-encoded columns, which need no storage at all).
 
 ## Information Schema
 

content/references/objects/tables.md

@@ -115,6 +115,15 @@ Create a table that stores all compressed data on a remote server previously cre
 CREATE TABLE remote_species (...) WITH (server = remote_storage);
 ```
 
+Set the compression method used for cold on-disk data, in addition to CedarDB's lightweight
+encoding (see the `cedardb_compression_info` [system table](/docs/compatibility/system_table) for per-column
+compression statistics). Supported values are `zstd` (the default) and `none`:
+
+```sql
+CREATE TABLE species (...) WITH (compression = zstd);
+CREATE TABLE species_uncompressed (...) WITH (compression = none);
+```
+
 ### Identity Columns
 
 Identity columns automatically generate unique integer values:
@@ -326,6 +335,17 @@ Drop a constraint and cascade to dependent constraints:
 ALTER TABLE child_table DROP CONSTRAINT fk_constraint CASCADE;
 ```
 
+### Storage Options
+
+Change the general-purpose compression codec used for on-disk column data.
+Supported values are `zstd` (the default) and `none`.
+This takes effect the next time affected data is written to disk:
+
+```sql
+ALTER TABLE species SET (compression = zstd);
+ALTER TABLE species SET (compression = none);
+```
+
 ### Ownership
 
 ```sql

layouts/partials/footer.html

@@ -11,7 +11,7 @@
     <div class="cedar-footer__links">
       <div>
         <h2>Company</h2>
-        <a href="https://cedardb.com/pricing/">Pricing</a>
+        <a href="https://cedardb.com/get-started/">Get Started</a>
         <a href="https://cedardb.com/about/">About Us</a>
         <a href="https://cedardb.com/blog/">Blog</a>
       </div>