fix(0071): apply corrected rollup MV DDL to ch-prod-01 + runbook/docs#64
Merged
Conversation
Investigated renaming the shadowing rollup bucket alias `AS timestamp` to `ts_bucket` as a permanent hardening. Rejected: a TO-table MV maps its SELECT output to the target columns BY NAME, so `ts_bucket` is refused (Code: 8 THERE_IS_NO_COLUMN), reproduced live on CH 26.3.10.60 via prices-clickhouse/tests/rollup_chain_it.rs. The bucket key is structurally forced to `AS timestamp`; qualifying the source as `t.timestamp` is the only fix, exactly as 0059 shipped. Bank the finding as comments in rollups.sql / preroll.sql so the dead-end isn't re-attempted (comment-only; both rollup integration tests stay green). Convert 0071 to a directory task and add the operator runbook (notes/G-prod-reapply-runbook.md) with an empty-DB fast path. No schema behaviour change; no prod deploy.
…oof step
The §6 verify query had 'price_ohlcv_15m FINAL AS r' — CH requires the alias
before FINAL ('AS r FINAL'), else Code: 62 parse error (hit live). Also add a
SHOW CREATE definition check after re-create: on an empty DB, listing 6 MV
names does not prove the DROP took (buggy set is also 6), so confirm the live
definition reads 'AS t FINAL' + 't.timestamp'.
…an up The live MV filters WHERE t.timestamp >= now() - INTERVAL 2 HOUR, so a smoke fixture with an epoch timestamp (1700000000) is excluded by the refresh and _15m stays empty (reads as 'stuck'/blank). Replace with now()-relative timestamps, flag the window, and make smoke-row cleanup an explicit always-run step.
…d-01 Empty-DB path verified live (CH 26.3.10.60): six prices.mv_ohlcv_* dropped and re-created from corrected rollups.sql; SHOW CREATE confirms qualified t.timestamp; refresh healthy; no residue, no restart. All ACs met (recompute N/A on empty DB). Provenance in notes/G-prod-reapply-runbook.md.
karczuRF
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Re-applies the corrected refreshable-MV rollup chain to live
ch-prod-01(the buggy DDL had been live since 0051) and closes task 0071. The OHLC bug: the bucket keyAS timestampshadows the sourcetimestamp, soargMin(open)/argMax(close)/argMax(close_usd)tie-break to an arbitrary row instead of the true first/last by time — fixed by source-qualifyingt.timestamp.prices.mv_ohlcv_*dropped + re-created from correctedrollups.sql.SHOW CREATE mv_ohlcv_1m_to_15mconfirmsFROM price_ohlcv_1m AS t FINALwithargMin/argMax(…, t.timestamp)(no baretimestamp);system.view_refresheshealthy. No restart, no data residue,prices.*-scoped — BEdefault.*untouched. Recompute step N/A (no data → no mis-rolled buckets).notes/G-prod-reapply-runbook.md) with empty-DB fast path; hardened from the live run — three footguns fixed: re-streaming without the DROP is a no-op (IF NOT EXISTS), the verify query needsAS r FINAL(alias beforeFINAL), and the live smoke test must use in-window timestamps (now() - INTERVAL 2 HOURfilter).rollups.sql/preroll.sqlnow document why the bucket alias can't be renamed: aTO-table MV routes results to the target by column name (so the bucket must betimestamp), which is separately what shadows the source — sot.-qualification is the only fix. Ats_bucketrename was tried and rejected withCode: 8 THERE_IS_NO_COLUMN, reproduced live viaprices-clickhouse/tests/rollup_chain_it.rs. Comment-only — no DDL behaviour change; rollup integration tests green.active/ → archive/), all ACs checked, provenance recorded.Notes
cargo check/test --workspace(rollup integration tests are#[ignore]d — no CH on the runner).