Run Nx integration tests sequentially to prevent shared-sandbox race condition#380
Merged
rhoerr merged 1 commit intoMay 27, 2026
Conversation
When Nx runs multiple affected projects in parallel (default --parallel=3), each project's PHPUnit bootstrap runs concurrently inside the same Warden php-fpm container, sharing the same filesystem. This causes two classes of race condition: 1. Both bootstraps call deployTestModules.php simultaneously, each removing and re-copying to app/code/Magento/TestModule*/ at the same time. 2. Both bootstraps call setup:install, which creates a sandbox directory under dev/tests/integration/tmp/ keyed by a SHA-256 hash of the install config. Since both targets share the same phpunit.xml.dist and install-config-mysql.php, they produce an identical sandbox path. The second process to arrive fails with "mkdir(): File exists", which the PHPUnit error handler converts to a fatal exception that crashes the bootstrap before any tests run. Setting --parallel=1 makes Nx execute test targets sequentially, eliminating the shared-state conflicts. The tradeoff is longer wall-clock time when multiple packages are affected, but the tests were failing entirely without this change so reliability wins over speed.
4 tasks
rhoerr
approved these changes
May 27, 2026
rhoerr
left a comment
Contributor
There was a problem hiding this comment.
Thank you. Seems plausible and safe. Merging to try it on mage-os/mageos-magento2#213 .
ddevallan
added a commit
to ddevallan/mageos-magento2
that referenced
this pull request
May 27, 2026
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rhoerr
pushed a commit
to mage-os/mageos-magento2
that referenced
this pull request
May 31, 2026
…(cron path) (#213) * Replace basic_get polling with basic_consume push for AMQP consumers Magento already uses basic_consume (push-based) when running a consumer in daemon mode via Queue::subscribe(). However, when --max-messages is set — the path taken by every cron-spawned consumer via ConsumersRunner — CallbackInvoker falls into a dequeue() loop that calls basic_get() once per message, requiring one full network round-trip per message. RabbitMQ explicitly discourages basic_get for continuous consumption. This change adds Queue::subscribeWithLimit(), structurally identical to the existing Queue::subscribe() but cancelling the consumer via basic_cancel() once $maxMessages have been processed. CallbackInvoker now routes AMQP queues with a message limit through this method, making the cron path consistent with the CLI daemon path. The dequeue() polling loop is preserved unchanged for MySQL and STOMP backends. The consumers_wait_for_messages config is respected: value 1 blocks indefinitely (matching original behaviour), value 0 uses a 1-second idle timeout to exit promptly on an empty queue. Benchmark (5,000 messages, Docker/local RabbitMQ ~0.1ms RTT): basic_get before: ~0.72s (~6,900 msg/s) basic_consume after: ~0.30s (~16,500 msg/s) — 2.4x faster At a typical production RTT of 2ms the improvement is ~100x: 5,000 x 2ms = 10s before vs ~50 round-trips x 2ms = 0.1s after. * Fix coding-standard warnings and pre-existing test fixture mismatch - Expand empty closures `function () {}` to multi-line form to satisfy the Magento2 coding standard (closing brace must be on its own line) in QueueTest.php and CallbackInvokerTest.php - Update TRemoteService.txt fixture to use nullable type `?int $typeId` matching the TRepositoryInterface source; the old fixture used `int` which caused RemoteServiceGeneratorTest::testGenerate #1 to fail * Clean up stale test modules before deploying to prevent mkdir collision When integration tests run against a warm CI cache, test module directories from a previous run may already exist in app/code/Magento. The deployTestModules script's mkdir() call then fails with "File exists", which the PHPUnit error handler converts to a fatal exception, crashing the bootstrap before any tests run. Mirroring the existing shutdown-time deleteTestModules() cleanup upfront ensures a clean slate on every run regardless of cache state. * Trigger CI to pick up --parallel=1 fix from mage-os/github-actions#380 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Trigger CI Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When Nx runs multiple affected projects in parallel (default
--parallel=3), each project's PHPUnit bootstrap runs concurrently inside the same Wardenphp-fpmcontainer, sharing the same bind-mounted filesystem. This produces two classes of race condition that crash the integration test bootstrap before any tests execute:1.
deployTestModules.phpconflictsBoth bootstraps call
deployTestModules.phpsimultaneously. Each process removes and re-copies toapp/code/Magento/TestModule*/at the same time — one process deletes directories the other just created, or twomkdir()calls collide on the same path, triggering a"File exists"PHP warning that the integration test error handler converts into a fatalPHPUnit\Framework\Exception.2. Shared sandbox directory conflict
Both bootstraps call
setup:installviaMagento\TestFramework\Application::install(), which creates a sandbox directory under:Because both targets share the same
phpunit.xml.distandinstall-config-mysql.php, they compute an identical sandbox path. The second process to callmkdir()on that path fails with"File exists", again converted to a fatal bootstrap exception.Fix
Add
--parallel=1to thenx affectedcommand so test targets execute sequentially:Trade-off
Sequential execution increases wall-clock time when multiple packages are affected simultaneously. However:
A longer-term fix would give each test target a distinct install config (different DB name), producing unique sandbox paths and eliminating the shared-state problem while restoring parallelism.
Related
Follows #379 (cache key fix, already merged).
🤖 Generated with Claude Code