Skip to content

made txs shard handling stricter#50

Merged
pompon0 merged 24 commits into
mainfrom
gprusak-queue
Jun 16, 2026
Merged

made txs shard handling stricter#50
pompon0 merged 24 commits into
mainfrom
gprusak-queue

Conversation

@pompon0

@pompon0 pompon0 commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Txs in each shard will be sent sequentially, which is consistent with what autobahn expects.

  • I have separated the txs sharding from endpoints - they are separate concepts now
  • each endpoint will allow sending up to tasksPerEndpoint transactions in parallel, as before
  • endpoint for each shard is sticky for backward compatible behavior
  • the number of shards increases by default from len(endpoints) to tasksPerEndpoint * len(endpoints)
  • introduced a new config field NumShards which allows configuring the number of shards independently from the number of endpoints.
  • since there will be significantly more shards, we cannot afford each of them to have a large queue any more, so the queue capacity is shared now. I have introduced a QueuePool for that purpose - it maintains a collection of queues with a shared capacity, sending/receiving from each queue is gated separately for efficient goroutine management.
  • I have updated the utils package by copying over (part of) the latest version of sei-chain/sei-tendermint/libs/utils

@cursor

cursor Bot commented Jun 15, 2026

Copy link
Copy Markdown

PR Summary

High Risk
This refactors the core transaction send pipeline (queueing, sharding defaults, and closed- vs open-loop rate limiting), so throughput, backpressure, and latency behavior can change materially on real load runs.

Overview
Stricter sharding for load generation: txs are routed to N shards (default endpoints × tasksPerEndpoint, overridable via numShards) instead of one queue per endpoint. Each shard drains sequentially through a dedicated goroutine; endpoint assignment stays shardIndex % len(endpoints) while each endpoint still runs up to tasksPerEndpoint parallel RPC senders via the new ethClient.

The old per-endpoint Worker (buffered channel + optional skip-rate-limit) is replaced by QueuePool (one shared backlog cap across all shards), ShardedSender shard loops that limiter.Wait before RPC, and ethClient for HTTP/WS sends, metrics, and inclusion registration.

Open-loop: when the scheduler owns the arrival clock, main passes an unlimited sender limiter so TPS is not double-gated; prewarm always self-paces on the shared limiter. The open-loop scheduler now uses golang.org/x/sync/semaphore instead of a custom helper.

Plumbing: concurrency moves to utils/scope (with utils/service as thin aliases); utils gains RWMutex, OrPanic, and related helpers synced from sei-tendermint; queue-depth metrics aggregate ShardStats from registered ShardedSender instances.

Reviewed by Cursor Bugbot for commit 25d018e. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread sender/sharded_sender.go
Comment thread sender/queue.go
Comment thread utils/testonly.go
@pompon0 pompon0 requested a review from bdchatham June 15, 2026 13:04
Comment thread sender/queue.go
@pompon0 pompon0 requested a review from wen-coding June 15, 2026 13:42
Comment thread sender/sharded_sender.go Outdated
Comment thread sender/eth_client.go Outdated
attribute.String("endpoint", c.cfg.Endpoint),
attribute.String("scenario", tx.Scenario.Name),
))
utils.SendOrDrop(c.receipts, tx)

@wen-coding wen-coding Jun 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is SendOrDrop, under load we may drop some txs from receipt tracking. The tx is still submitted, but we'd never observe an on-chain failure for the dropped ones — so TrackReceipts is effectively a sample, not full coverage. Is that correct?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is. Also, afaiu in case of autobahn it will not be available at all since receipt store is not ready, right?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Autobahn the receipt store currently works for EVM tx (although there are many performance and safety issues the storage team is fixing), it doesn't work for Cosmos tx at all.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anyway, after rebase receipt requests are gone entirely in favor of observing the finalized blocks.

Comment thread config/config.go
Comment thread sender/eth_client.go Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ead64e6. Configure here.

Comment thread utils/testonly.go
@pompon0 pompon0 enabled auto-merge (squash) June 16, 2026 13:55
@pompon0 pompon0 merged commit 7c4bfe8 into main Jun 16, 2026
4 checks passed
@pompon0 pompon0 deleted the gprusak-queue branch June 16, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants