Skip to content

[MaterializedView] Add enableMaterializedViewRewrite per-query option to control MV rewrite#18665

Merged
xiangfu0 merged 1 commit into
apache:masterfrom
hongkunxu:feat/sse_mv_query_rewrite
Jun 6, 2026
Merged

[MaterializedView] Add enableMaterializedViewRewrite per-query option to control MV rewrite#18665
xiangfu0 merged 1 commit into
apache:masterfrom
hongkunxu:feat/sse_mv_query_rewrite

Conversation

@hongkunxu
Copy link
Copy Markdown
Contributor

@hongkunxu hongkunxu commented Jun 3, 2026

Summary

Adds a new per-query dimension for controlling materialized-view (MV) query rewrite.

Previously MV query rewrite could be controlled in two ways:
(1) a broker startup config (pinot.broker.query.enable.materialized.view.rewrite)
(2) a per-MV switch in the MV table config.

This PR adds a third, query-level dimension: a session variable enableMaterializedViewRewrite (default true).

Motivation

  1. Bugfix: queries issued by the materialized-view minion task must not attempt MV query rewrite at all. Beyond the risk of being rewritten to hit the MV itself, running rewrite on these queries is pure wasted work — it should be short-circuited early to avoid the needless overhead. The minion executor now sets enableMaterializedViewRewrite=false so its materialization query skips rewrite entirely and always reads the base table.
  2. User control: users need the ability to turn the whole MV query rewrite off per query. Defaulting to true keeps full backward compatibility — once an operator enables rewrite on the broker, users get it automatically with no extra session variable to set — while still giving them an off switch for testing or when an MV is misbehaving.

Usage

SET enableMaterializedViewRewrite = 'false';

@hongkunxu hongkunxu force-pushed the feat/sse_mv_query_rewrite branch from e8ea1e8 to 63023a8 Compare June 3, 2026 10:17
@hongkunxu hongkunxu marked this pull request as ready for review June 3, 2026 10:21
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jun 3, 2026

Codecov Report

❌ Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 64.50%. Comparing base (cfc8063) to head (5c647dd).
⚠️ Report is 27 commits behind head on master.

Files with missing lines Patch % Lines
...ew/executor/GrpcMaterializedViewQueryExecutor.java 83.33% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18665      +/-   ##
============================================
+ Coverage     64.45%   64.50%   +0.04%     
  Complexity     1291     1291              
============================================
  Files          3365     3372       +7     
  Lines        208066   208596     +530     
  Branches      32482    32577      +95     
============================================
+ Hits         134108   134553     +445     
- Misses        63171    63248      +77     
- Partials      10787    10795       +8     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-21 64.50% <94.11%> (+0.04%) ⬆️
temurin 64.50% <94.11%> (+0.04%) ⬆️
unittests 64.50% <94.11%> (+0.04%) ⬆️
unittests1 56.92% <100.00%> (+0.10%) ⬆️
unittests2 37.13% <76.47%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@xiangfu0 xiangfu0 requested review from Copilot June 3, 2026 22:01
@xiangfu0 xiangfu0 added configuration Config changes (addition/deletion/change in behavior) query Related to query processing materialized-view labels Jun 3, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new per-query/session query option (enableMaterializedViewRewrite, default true) to control whether the broker is allowed to perform materialized-view (MV) query rewrite, and uses it to ensure MV minion materialization queries never get rewritten back onto an MV.

Changes:

  • Introduces enableMaterializedViewRewrite as a user-facing query option key (default-on, opt-out).
  • Adds broker-side gating to bypass MV rewrite entirely when the option is set to false.
  • Updates the MV minion gRPC executor to inject SET enableMaterializedViewRewrite='false' into its materialization queries, plus adds unit tests for parsing and broker behavior.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java Adds the new query option key constant and documents intended semantics.
pinot-common/src/main/java/org/apache/pinot/common/utils/config/QueryOptionsUtils.java Adds isMaterializedViewRewriteEnabled(...) helper with default-true behavior for back-compat.
pinot-common/src/test/java/org/apache/pinot/common/utils/config/QueryOptionsUtilsTest.java Adds unit test covering default-true and explicit-disable semantics.
pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandler.java Gates MV rewrite application on the new per-query option.
pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/BaseSingleStageBrokerRequestHandlerTest.java Adds regression test asserting rewrite engine is not consulted and no MV swap occurs when disabled.
pinot-materialized-view/src/main/java/org/apache/pinot/materializedview/executor/GrpcMaterializedViewQueryExecutor.java Injects SET enableMaterializedViewRewrite='false' into MV materialization queries sent via gRPC.
pinot-materialized-view/src/test/java/org/apache/pinot/materializedview/executor/GrpcMaterializedViewQueryExecutorTest.java Adds test ensuring the injected SET parses and resolves to “rewrite disabled”.

@xiangfu0 xiangfu0 force-pushed the feat/sse_mv_query_rewrite branch from 63023a8 to 3b167a9 Compare June 6, 2026 00:23
Introduce a user-facing per-query option `enableMaterializedViewRewrite`
(default true) that gates broker-side materialized-view rewrite. The MV
minion query executor forces `enableMaterializedViewRewrite=false` for its
materialization query via gRPC request metadata, which the broker applies as
an override after parsing the query. This makes the disable unconditional: it
cannot be defeated by an `enableMaterializedViewRewrite` option already
present in the (user-authored) materialization SQL, since a SQL `SET` prefix
would otherwise lose to a later duplicate and SQL options outrank request
options. The materialization query therefore always reads the base table and
is never rewritten back onto an MV (its own or a sibling), avoiding the
circular-rewrite hazard. Disabling only forgoes an optimization and never
changes results. Distinct from the broker-internal materializedViewRewrite
marker, which remains stripped from user input.

Signed-off-by: Hongkun Xu <xuhongkun666@163.com>
Co-authored-by: Xiang Fu <xiangfu.1024@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@xiangfu0 xiangfu0 force-pushed the feat/sse_mv_query_rewrite branch from 3b167a9 to 5c647dd Compare June 6, 2026 00:29
@xiangfu0 xiangfu0 merged commit 678c99c into apache:master Jun 6, 2026
11 checks passed
@xiangfu0 xiangfu0 deleted the feat/sse_mv_query_rewrite branch June 6, 2026 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

configuration Config changes (addition/deletion/change in behavior) materialized-view query Related to query processing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants