Skip to content

Fix flaky pipe auto split IT#17756

Merged
jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix-pipe-auto-split-it
May 27, 2026
Merged

Fix flaky pipe auto split IT#17756
jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix-pipe-auto-split-it

Conversation

@Caideyipi
Copy link
Copy Markdown
Collaborator

Motivation

IoTDBPipeAutoSplitIT assumes that auto split always leaves both history and realtime pipes visible immediately after creation. In practice, the history snapshot pipe can finish and be auto-dropped very quickly when there is no historical data, so SHOW PIPE may only return the realtime pipe and the test fails with expected 2 but was 1.

Modifications

  • Relax the auto split assertion to require the realtime pipe and reject the original unsplit pipe.
  • Allow the history pipe to be absent after automatic completion/drop.
  • Keep validating the history pipe's sink option when that history pipe is still visible.

Tests

  • Passed: git diff --check -- integration-test/src/test/java/org/apache/iotdb/pipe/it/dual/treemodel/auto/basic/IoTDBPipeAutoSplitIT.java
  • Not run successfully locally: IoTDBPipeAutoSplitIT#testSingleEnv. Maven/JVM failed during compilation on this machine due Windows page file/native memory exhaustion.

@jt2594838 jt2594838 merged commit bc150ff into apache:master May 27, 2026
41 checks passed
@jt2594838 jt2594838 deleted the fix-pipe-auto-split-it branch May 27, 2026 07:49
Copy link
Copy Markdown
Member

@luoluoyuyu luoluoyuyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review 总结

IT 修复思路正确:history pipe 在 snapshot 传完后可能被 auto-drop,原先用固定 size==2 + 顺序断言过于脆弱。assertAutoSplitResult 改为只要求存在 _realtime、不存在未拆分原名、且列表中仅有 _history/_realtime 子 pipe,更贴近真实行为。

建议合入(见行内 1 条 nit)。

// The history pipe may have already been auto-dropped after snapshot transfer completes.
Assert.assertTrue(
showPipeResult.stream().anyMatch(i -> Objects.equals(i.id, pipeName + "_realtime")));
Assert.assertFalse(showPipeResult.stream().anyMatch(i -> Objects.equals(i.id, pipeName)));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Assert.assertFalse(showPipeResult.stream().anyMatch(...)) 可简写为 Assert.assertTrue(showPipeResult.stream().noneMatch(i -> Objects.equals(i.id, pipeName))),语义相同。

另:若未来存在第三个pipeName 前缀相关但非 _history/_realtime 的 pipe(例如手动创建的 a2b_custom),allMatch 仍会通过。若需更严,可改为:

final long related = showPipeResult.stream().filter(i -> i.getId().startsWith(pipeName)).count();
Assert.assertEquals(2, related); // or 1 if history already dropped

当前实现已足够修 flake,非阻塞。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants