Harden security, reliability, and CI from adversarial review#2
Conversation
Fixes every issue in .roast/REPORT-latest.md (6 High, 10 Medium, 5 Low). API security (High 1, 2): - Refuse non-loopback bind unless HYPERDATA_API_KEY is set (Bearer / X-API-Key auth on all non-health routes) or HYPERDATA_UNSAFE_PUBLIC_API=1 explicitly accepts the risk - CORS wildcard only on loopback; HYPERDATA_CORS_ORIGINS allowlist otherwise - Per-IP REST rate limiting (300 req/min sliding window) - WebSocket: bounded per-client send queues with one writer task per client (drop-and-count instead of unbounded task fan-out), inbound message size/rate limits, bad-message disconnect threshold, heartbeat loop errors logged instead of swallowed Reliability (High 3, 4, 5; Med 8, 9): - Hub startup contract: component failures recorded in status.failed_components, surfaced in /v1/health and an honest degraded log line; position_scanner/market_data report 'starting' until their first cycle succeeds instead of a blind 'ready' - Status-update loop body wrapped so one bad component can't kill the staleness watchdog; HLP z-score alerts debounced to once a minute - Explicit aiohttp timeouts on every external HTTP call (market data, position scanner, HL price poll, Telegram/Discord webhooks) - Exchange payload parse guards: malformed Binance/OKX/Bybit/HL records and orderbook levels are dropped individually and counted (parse_errors in liquidation stats) instead of crashing connection loops - Per-venue orderflow freshness (venue_freshness) so a dead venue can't hide behind the combined feed; hub warns when one venue goes silent Data integrity (Med 1, 3, 4, 5): - Liquidation dedup buckets on exchange event time and the cascade bypass is venue-scoped (also fixes bypass keys that never matched) - Corrupted SQLite files are quarantined to data/corrupted/ with a timestamp instead of deleted - schema_version table with versioned migrations; only duplicate-column errors are treated as already-applied - Wallet addresses validated (0x + 40 hex), normalized to lowercase, and capped at 50k with least-recently-seen expiry Strategy safety (Med 2, 6, 7): - Smart-money ranking requires 10+ trades and $50k+ volume, exposes a sample-size confidence score, and clears stale tiers on requalification - Paper trader: balance-checked position adds, weighted-average entry price, invalid-signal rejection — balance can never go negative - LLM agent: strict first-line-only action parsing (ambiguous output is rejected, never substring-matched), persistent worker thread, and an evals-per-hour budget (LLM_MAX_EVALS_PER_HOUR) Tests and CI (High 6, Low 1-5): - test_position_scanner.py rewritten against the SQLite address store and re-enabled in CI (all network mocked) - New tests/test_hardening.py: bind guard, auth/CORS/rate-limit middleware, WS abuse limits, dedup/cascade, malformed payloads, degraded startup, paper-trader invariants, LLM parsing, quarantine, schema version, alert redaction, per-venue freshness (43 tests) - CI: lint blocking (repo lint debt cleaned), actions pinned by SHA, requirements.lock used for installs, advisory pip-audit step - Dead copy-trading/smart-money handlers deleted from api_server - README/API docs foreground the exposure model; alert logs no longer persist full payloads (wallet intelligence redacted) - run_dashboard menu input no longer mutates executor internals
There was a problem hiding this comment.
What this PR does: Hardens security, reliability, and CI based on an adversarial codebase review, adding API bind guards, auth, rate limits, WS backpressure, parse guards, quarantine, schema versioning, strategy safety, and updating CI to pinned commit SHAs and blocking ruff.
Risk areas:
- The
setup-pythonstep in the CI workflow is missing thewith:block that specifiespython-version, so the Python version matrix will be ignored and all jobs will use the runner's default Python; this undermines cross-version testing.
Verdict:⚠️ Minor concerns
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
| uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 |
There was a problem hiding this comment.
🟠 P1 (High): The setup-python action is invoked without a with: block specifying python-version: ${{ matrix.python-version }}, so the Python version matrix is not applied. This means all jobs will use the runner's default Python and cross-version testing is lost. Add with: python-version: ${{ matrix.python-version }} to restore matrix testing.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b0875b3da1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
| uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 |
There was a problem hiding this comment.
Restore matrix Python selection
This step no longer passes python-version to actions/setup-python; the action docs state that without this input it falls back to .python-version or the runner PATH, and this repo has no .python-version. The matrix value is now only in the display name, so the two jobs won't actually exercise 3.12 and 3.13 and version-specific failures can slip through CI.
Useful? React with 👍 / 👎.
| await self.feed.emit(event) | ||
| except Exception: | ||
| logger.debug("[bybit] failed to parse liquidation: %s", d) | ||
| except (KeyError, TypeError, ValueError): |
There was a problem hiding this comment.
Keep malformed Bybit payloads from reconnecting
Narrowing this handler to KeyError, TypeError, and ValueError leaves AttributeError uncaught, but the lines above call d.get(...) without first proving data["data"] is a dict. If Bybit sends an allLiquidation message with a non-dict data shape during schema drift or an error payload, _on_message raises out to the websocket loop and reconnects instead of recording a parse error, which is exactly the crash/reconnect path this hardening is meant to avoid.
Useful? React with 👍 / 👎.
Co-Messi
left a comment
There was a problem hiding this comment.
Testing single comment 0
| await self.feed.emit(event) | ||
| except Exception: | ||
| logger.debug("[bybit] failed to parse liquidation: %s", d) | ||
| except (KeyError, TypeError, ValueError): |
Co-Messi
left a comment
There was a problem hiding this comment.
Testing single comment 1
| self._init_tables() | ||
| except sqlite3.DatabaseError: | ||
| logger.warning("Database corrupted at %s — recreating", self.db_path) | ||
| if conn is not None: |
Co-Messi
left a comment
There was a problem hiding this comment.
Testing single comment 2
| # Log that an alert fired, not its payload — alert bodies can contain | ||
| # wallet addresses and position intelligence that must not sit in | ||
| # rotating plaintext logs. | ||
| first_line = message.strip().splitlines()[0] if message.strip() else "" |
Co-Messi
left a comment
There was a problem hiding this comment.
Testing single comment 3
| """ | ||
| lines = text.split("\n", 1) | ||
| action_word = lines[0].strip().upper() | ||
| action_word = lines[0].strip().upper().strip(".!:*# ") |
Co-Messi
left a comment
There was a problem hiding this comment.
Testing single comment 4
| pip install pytest ruff | ||
| pip install -r requirements.lock | ||
| pip install -e . --no-deps | ||
| pip install pytest pytest-asyncio ruff pip-audit |
Co-Messi
left a comment
There was a problem hiding this comment.
🛡️ HyperData-Terminal Full Adversarial & Security Review
This PR successfully resolves many critical security issues (6 High, 10 Medium, 5 Low). However, a deep adversarial review of the newly introduced code has uncovered several remaining vulnerabilities across the websocket ingestion feeds, database quarantine logic, alert redactions, LLM parsing, and CI/CD supply chains.
See inline comments for details, code analogies, and recommended remediations.
| await self.feed.emit(event) | ||
| except Exception: | ||
| logger.debug("[bybit] failed to parse liquidation: %s", d) | ||
| except (KeyError, TypeError, ValueError): |
There was a problem hiding this comment.
[Adversarial Finding - High]: Unhandled AttributeError can crash Bybit Ingestion Feed.
Unlike OKXConnection, BybitConnection._on_message does not catch AttributeError in its except block. If Bybit streams a message where symbol or side fields are absent or null (parsed as None), the subsequent call to normalize_symbol(symbol_raw, "bybit") will execute None.upper(), throwing an AttributeError.
Since this exception is unhandled, it will propagate up and crash the websocket connection message loop, forcing a socket disconnect and reconnect loop.
Analogy: A security gate that locks itself shut and triggers a full power cycle whenever someone tries to swipe a blank card instead of just rejecting the card.
Fix: Catch AttributeError in the except tuple:
except (KeyError, TypeError, ValueError, AttributeError):
self.feed.record_parse_error("bybit", d)| self._init_tables() | ||
| except sqlite3.DatabaseError: | ||
| logger.warning("Database corrupted at %s — recreating", self.db_path) | ||
| if conn is not None: |
There was a problem hiding this comment.
[Adversarial Finding - Medium]: Database Quarantine False Positives on Lock Contention.
sqlite3.DatabaseError is the base class for all sqlite errors, including sqlite3.OperationalError (e.g. "database is locked" or "disk I/O error").
If a concurrent process (like a live dashboard view or an automated verification run) holds a lock on the database during hub startup, the integrity check or connection setup will fail with OperationalError: database is locked. The code will catch this as a corruption error, quarantine the perfectly healthy database, and start a fresh empty database.
Analogy: If your front door is stuck shut for a minute, you assume the house has collapsed, move your belongings to a temporary shed, and build a new house from scratch.
Fix: Check the exception details. Do not quarantine if the error message indicates lock contention or a busy state:
except sqlite3.DatabaseError as exc:
if conn is not None:
conn.close()
if "lock" in str(exc).lower() or "busy" in str(exc).lower():
logger.error("Database is locked/busy — failing startup rather than quarantining")
raise| # Log that an alert fired, not its payload — alert bodies can contain | ||
| # wallet addresses and position intelligence that must not sit in | ||
| # rotating plaintext logs. | ||
| first_line = message.strip().splitlines()[0] if message.strip() else "" |
There was a problem hiding this comment.
[Adversarial Finding - Medium]: Plaintext Wallet/Position Leaks in Alert Log Redaction.
The code redacts alert logs by writing only the first_line of the message to console/logs. However, if the alert format is updated or if the alert message has the wallet address or position intelligence on the first line (e.g., Whale alert: 0x... has a large position), this logic will fail. The plaintext wallet intelligence will be written directly into rotation logs.
Analogy: Shredding a letter but only keeping the first line because you assume the first line never contains private secrets. If the writer started with their address, your shredder is useless.
Fix: Use regex to explicitly strip or mask EVM addresses (0x[a-fA-F0-9]{40}) from the logged first line, rather than assuming it's safe:
import re
redacted_line = re.sub(r'0x[a-fA-F0-9]{40}', '0x...[REDACTED]', first_line)| """ | ||
| lines = text.split("\n", 1) | ||
| action_word = lines[0].strip().upper() | ||
| action_word = lines[0].strip().upper().strip(".!:*# ") |
There was a problem hiding this comment.
[Adversarial Finding - Low/Robustness]: Split-by-newline is vulnerable to blank lines in LLM responses.
lines = text.split("\n", 1) splits the output of the LLM by the first newline. If the LLM responds with a leading blank line or formatting greeting before outputting BUY/SELL/HOLD, lines[0] will be empty. This will cause the parser to reject a completely valid decision.
Analogy: A robot mail sorter that throws away letters if there is a blank line at the top of the envelope before the address.
Fix: Split and filter out empty lines, then take the first non-empty line:
all_lines = [l.strip() for l in text.splitlines() if l.strip()]
if not all_lines:
return None
action_word = all_lines[0].upper().strip(".!:*# ")
reason = "\n".join(all_lines[1:]) if len(all_lines) > 1 else ""| pip install pytest ruff | ||
| pip install -r requirements.lock | ||
| pip install -e . --no-deps | ||
| pip install pytest pytest-asyncio ruff pip-audit |
There was a problem hiding this comment.
[Supply Chain - Medium]: Unpinned test dependencies in CI workflow.
Installing pytest, pytest-asyncio, ruff, and pip-audit without version pinning in the CI workflow exposes the pipeline to non-deterministic test environments and package hijacking attacks. If a malicious version of any of these is published to PyPI, it will be executed automatically in your CI workflow.
Analogy: Inviting random contractors off the street into your workshop without checking their credentials or tools.
Fix: Pin the versions in the workflow file directly or include them in requirements.lock:
pip install pytest==8.3.4 pytest-asyncio==0.25.2 ruff==0.9.1 pip-audit==2.7.3| [tool.ruff.lint] | ||
| select = ["E", "F", "W", "I"] | ||
|
|
||
| [tool.ruff.lint.per-file-ignores] |
There was a problem hiding this comment.
[Packaging Security - Medium]: Packaging of raw config* directory.
(Reviewer note: Commenting here on the ruff addition since the setuptools section is unchanged in this diff)
Line 20 of pyproject.toml includes config* in the packaging build: include = ["src*", "config*"]. This causes any local config templates, settings, or accidental secrets stored in the config/ directory to be packaged into the distributed .whl and .tar.gz archive artifacts upon building, exposing internal file structures to downstream installers.
Analogy: Packing your private notes and house diagrams into a box of free books that you leave on the sidewalk.
Fix: Restrict finding to src only in pyproject.toml line 20:
include = ["src*"]
Co-Messi
left a comment
There was a problem hiding this comment.
Adversarial re-review of PR #2. The PR fixes the headline issues but ships several critical regressions and dead-code paths that defeat the stated protections. 22 findings below — CRIT/HIGH should land before merge. Verified each by tracing the relevant code path; line numbers reference the fix-roast-report branch.
Top issues:
!forceOrder@arrBinance stream silently produces zero events (liquidation_feed.py:165)- Cascade bypass self-perpetuates; size rounded to $100 = 100x false-dedup window (api_server.py:442, 482)
- Paper trader close path lets
balancego negative (paper_trader.py:208) - Cascade example strategy is silently non-functional (liquidation_cascade.py:47)
- WebSocket upgrade has no Origin/CSRF check - cross-site scraping on loopback (api_server.py:686)
- Telegram bot token + Discord webhook URL leak in
logger.exceptiontracebacks (alerts.py:384) LLM_MAX_EVALS_PER_HOURbudget consumed before the LLM call - flaky provider exhausts it in 20 min (llm_agent.py:80)evaluate()blocks asyncio for up to 20s per call - paper trader stalls on every slow LLM response (llm_agent.py:111)- pip-audit has
continue-on-error: true- known CVEs do not block CI; no Dependabot (ci.yml:37)
Please address CRIT + HIGH before merging. I'll re-review once patches land.
Additional findings (lines not in PR diff hunks — listed for reference):
-
src/strategies/examples/liquidation_cascade.py:47 -- [CRIT] Cascade example strategy is silently non-functional....
-
src/strategies/paper_trader.py:259 -- [HIGH] In-memory state mutated before SQLite persist; state diverges on DB error....
-
src/data_layer/liquidation_feed.py:144 -- [HIGH] No max retry / circuit breaker on WS reconnect -- pinned CPU on dead endpoint....
-
src/data_layer/position_scanner.py:59 -- [MED]
asyncio.gatherwithoutreturn_exceptions=True-- one bad endpoint aborts price + meta updates together....
| @@ -159,19 +163,25 @@ def __init__(self, feed: LiquidationFeed): | |||
|
|
|||
| async def _on_message(self, data: Any) -> None: | |||
| if isinstance(data, dict) and data.get("e") == "forceOrder": | |||
There was a problem hiding this comment.
[CRIT] Binance !forceOrder@arr stream produces zero events.
Handler gates on isinstance(data, dict) and data.get('e') == 'forceOrder'. !forceOrder@arr wraps each frame in a JSON array [ {...}, {...} ], so the array payload fails the type check and the early return skips BOTH the parse block AND the record_parse_error increment. Result: liquidations.get_stats(60)['by_exchange'] has no binance key, the PR's own counter stays at 0, no warning fires. Cascade alerts that key off coverage['binance'] = sampled look like Binance is feeding when it isn't.
Fix: iterate data when it's a list -- for raw in (data if isinstance(data, list) else [data]): -- before the per-record try/except.
| bypass_key = f"{ev.symbol}_{ev.side}" | ||
| bypass_key = f"{ev.symbol}_{ev.side}_{ev.exchange}" | ||
| if bypass_key in self._cascade_bypass and now < self._cascade_bypass[bypass_key]: | ||
| return False |
There was a problem hiding this comment.
[CRIT] Dedup silently falls back to local clock on missing event time.
ev_ts = ev.timestamp if ev.timestamp > 0 else now violates the PR's 'buckets on exchange event time' claim. When any connector's or 0 fallback fires (Binance/Bybit/OKX/HL parse guards), the dedup hash uses local receive time. Two timestamp=0 events within 3s and matching symbol/side/exchange collide and are silently deduped.
Fix: skip dedup entirely when ev.timestamp <= 0 (treat as 'cannot safely hash'). Never substitute local clock for missing exchange time.
| bypass_key = f"{ev.symbol}_{ev.side}_{ev.exchange}" | ||
| if bypass_key in self._cascade_bypass and now < self._cascade_bypass[bypass_key]: | ||
| return False | ||
|
|
There was a problem hiding this comment.
[CRIT] size_usd rounded to nearest $100 = 100x false-dedup window.
round(99950, -2) == round(100049, -2) == round(100050, -2) == 100000. Two distinct liquidations with sizes $99,950 vs $100,050 sharing symbol/side/exchange within 3s dedupe as the same event. In a 50+ liq/min cascade this is near-certain false-dedup. Combined with cascade bypass feedback loop (line 482), a single cascade both over-dedups legitimate events AND over-broadcasts replayed duplicates.
Fix: drop size_rounded from the hash, or use round(ev.size_usd, -1) (nearest $10), or include the full size.
| self._cascade_bypass[bypass_key] = now + self._CASCADE_BYPASS_DURATION | ||
| # Bypass dedup only for this exchange's stream: cascades on one | ||
| # venue say nothing about duplicates on another. | ||
| self._cascade_bypass[key] = now + self._CASCADE_BYPASS_DURATION |
There was a problem hiding this comment.
[CRIT] Cascade bypass self-perpetuates via feedback loop.
self._cascade_bypass[key] = now + self._CASCADE_BYPASS_DURATION is re-set on every call where len(entries) >= 3 (the if entries >= 3: branch above). On WS replay, duplicates pass _is_duplicate_liq, get appended to _cascade_tracker[key], and re-trigger cascade detection -- extending the bypass to now + 30 again. A single 5-min replay floods the broadcast with 10x duplicates and the bypass NEVER expires while traffic continues. cascade_bypass dict has no size or time cap.
Fix: track first_bypass_at per key; refuse to extend past first_bypass_at + 2*CASCADE_BYPASS_DURATION (60s absolute cap). Also bound _cascade_tracker to N entries per key.
| @@ -178,8 +208,19 @@ def _execute_trade(self, strategy_name: str, signal: Signal) -> None: | |||
| self.balance += pos["size_usd"] + pnl | |||
There was a problem hiding this comment.
[CRIT] Close path allows balance to go negative.
self.balance += pos['size_usd'] + pnl -- no floor. Open a short at price=100, size=$1000. Price moves to 10000. pnl = 1000 * (-99) = -99000. balance += 1000 - 99000 = -98000. Violates the PR claim 'balance can never go negative'. A long close with >100% loss has the same bug.
Fix: self.balance = max(0.0, self.balance + pos['size_usd'] + pnl) -- or enforce maintenance margin: reject close if pos['size_usd'] + pnl < 0 and force liquidate remainder at 0.
| p.unlink() | ||
| dest = quarantine_dir / f"{p.name}.{stamp}" | ||
| try: | ||
| p.rename(dest) |
There was a problem hiding this comment.
[HIGH] Quarantine fallback can brick the system on a stuck DB.
p.rename(dest) may fail with OSError (cross-device EXDEV, EBUSY if held open, EACCES, ENOSPC). The catch logs and falls back to p.unlink(). If unlink also fails (Windows file handle, read-only mount), the corrupted DB remains. Then line 80+: sqlite3.connect(str(self.db_path), ...) re-opens the same corrupted file -> raises sqlite3.DatabaseError again. The except at line 59 does NOT catch this -- it propagates out of __init__ uncaught, crashing the process on startup. The 'last resort so we can still start' comment is misleading: no last-resort success path.
Fix: wrap recovery sqlite3.connect in its own try/except. On failure, open :memory: and log fatal. Use shutil.move (handles cross-device) before unlink.
| now = time.time() | ||
| if abs(hlp_zscore) > 2.0 and now - _last_hlp_alert_check > 60.0: | ||
| _last_hlp_alert_check = now | ||
| asyncio.create_task( |
There was a problem hiding this comment.
[HIGH] Fire-and-forget task + dead HLP-zscore alert body.
Line 550: asyncio.create_task(self.alerts._check_hlp_zscore(self.hlp.get_stats())) -- task handle discarded. Under PyPy/debug asyncio, GC of unreferenced tasks is a real failure mode. Worse: alerts._check_hlp_zscore (lines 174-183) is a no-op -- body is pass under # HLP Z-score alerts -- DISABLED. Every status tick where |z|>2 for >60s creates and drops a coroutine that does nothing. Status log claims an alert path is in use; it is not. Cascade alerts (line 152-160) and large_liq (145-148) are also disabled bodies.
Fix: either delete the create_task block + the dead _check_* methods, OR re-enable the alert send and store the task on self._tasks so stop() awaits it.
| if bypass_key in self._cascade_bypass and now < self._cascade_bypass[bypass_key]: | ||
| return False | ||
|
|
||
| ev_ts = ev.timestamp if ev.timestamp > 0 else now |
There was a problem hiding this comment.
[HIGH] Dedup safety depends on parse-layer divide-by-1000 convention -- undocumented invariant.
The dedup hash int(ev_ts // self._DEDUP_WINDOW) buckets on seconds. All 4 parse guards divide by 1000.0 (Binance 178, Bybit 228, OKX 270, HL 340). But this normalization is NOT enforced at the dedup layer. If a future connector adds ev_ts = d.get('time', 0) without /1000.0, a single event from that connector would hash to a different bucket than the same event from a properly-normalized connector, silently defeating cross-venue dedup.
Fix: sanity guard in _is_duplicate_liq: if not (1e9 < ev_ts < 1e11): return False # not safely dedupable. Log warning with offending exchange.
| - name: Dependency vulnerability audit | ||
| # Advisory: a newly-published CVE in a pinned dep should be visible | ||
| # without failing unrelated PRs. Review failures in the job log. | ||
| continue-on-error: true |
There was a problem hiding this comment.
[HIGH] pip-audit has continue-on-error: true -- known CVEs do not block CI. No Dependabot config.
Line 37: continue-on-error: true on the pip-audit -r requirements.lock step. PR description admits this is 'advisory'. In practice, a known CVE in any pinned dep (aiohttp, httpx, etc.) can land in main with green CI. No follow-up step parses output and fails the build. No scheduled weekly run. No Dependabot/Renovate config (.github/dependabot.yml returns 404). Lockfile is the source of truth and is never re-pinned by CI.
Fix: drop continue-on-error: true + add --strict, OR add a follow-up step that fails the job if any vuln has a fix and severity >= medium. Add Dependabot for pip and for github-actions SHAs.
|
|
||
| # Every outbound request gets an explicit deadline: a hung exchange endpoint | ||
| # must fail the refresh cycle, not stall the hub's market-refresh loop forever. | ||
| HTTP_TIMEOUT = aiohttp.ClientTimeout(total=10) |
There was a problem hiding this comment.
[MED] Hyperliquid info API uses ClientTimeout(total=10) only -- no connect/sock_connect split.
With total=10 and no connect= or sock_connect=, a slow TLS handshake (HL throttling the IP after a rate spike) can consume the entire 10s budget, leaving zero for the response body -- a hung endpoint masquerades as a 10s partial-read error. Under sustained HL throttling, both market_refresh and position_scan polls' 10s budgets collapse in lockstep. Same hole at position_scanner.py:20.
Fix: aiohttp.ClientTimeout(total=10, connect=3, sock_connect=3, sock_read=5). Apply to position_scanner.HTTP_TIMEOUT too.
Follow-up: 4 findings from PR #2 adversarial reviewLines not in diff hunks — GitHub API only allows inline review comments on changed lines. Listed here in full. 1.
|
CI:
- Restore 'python-version: ${{ matrix.python-version }}' on setup-python
(the matrix was display-only after the rewrite)
- Pin test tooling versions in the workflow; make pip-audit blocking
- Bump aiohttp 3.13.3 -> 3.14.1 in requirements.lock (4 published CVEs)
- Add .github/dependabot.yml (pip + github-actions, weekly)
API server:
- WebSocket upgrades with a browser Origin header are rejected unless the
origin is allowlisted (browser WS is not gated by the same-origin policy,
so any webpage could read the loopback stream)
- Split /v1/live (minimal, unauthenticated) from /v1/health (detailed,
requires the API key when one is set — the payload is operational recon)
- Health/liveness paths exempt from per-IP rate limiting (LB/monitor
probes share a NAT IP)
- Dedup: events without a plausible epoch-seconds timestamp are never
deduped (no local-clock fallback, no ms-scale hashing); the hash uses
the exact size instead of round(-2) so distinct ~$100k events cannot
collapse; cascade bypass has an absolute 2x-duration cap so replayed
duplicates cannot extend it forever; cascade tracker bounded per key
- /v1/health reports degraded when position scanner or market data loops
are in 'error' (previously only startup failures counted)
Feeds:
- Binance forceOrder handler accepts array frames (@arr batching) instead
of silently dropping them
- Bybit handler accepts v5 list payloads and short keys (p/v/S/s/T),
catches AttributeError, and drops malformed records individually
- Exchange WS reconnect escalates to a single ERROR after 20 consecutive
failures and quiets the log spam; still retries at max backoff (a hard
stop would permanently kill the feed across a venue outage)
- Split connect/sock_read timeouts on all external HTTP deadlines so a
slow TLS handshake can't consume the whole budget
- Position scanner price/meta updates use return_exceptions so one dead
endpoint doesn't discard the other's result
Strategies:
- Paper trader: close credit floored at zero balance (a >100% adverse move
on a short could previously drive balance negative); trades persist to
SQLite BEFORE mutating the books, so a DB error can no longer diverge
the portfolio from the audit trail
- LLM agent: evaluate() is now async and awaited by the paper trader (a
slow LLM no longer blocks the event loop); transport-level failures
refund their budget slot (a down provider can't exhaust the hourly
budget; parse failures still consume theirs since tokens were billed);
leading blank lines in responses are tolerated
- Cascade example strategy actually fires: getattr on the stats dict
always returned 0 (and the key name was wrong) — now reads
stats['long_volume_usd']
Persistence & alerts:
- 'database is locked/busy' raises instead of quarantining a healthy DB
held by another process; quarantine uses shutil.move and falls back to
an in-memory store if the corrupted file can't be moved or removed
- Alert send failures log the exception type only (aiohttp error strings
can embed the bot-token URL); wallet addresses masked on the logged line
- Removed dead HLP z-score alert scheduling (the alert body is disabled)
Declined with rationale (see PR discussion): config/ stays in packaging
(settings.py is a runtime import, contains no secrets); smart-money tier
persistence is not wired (save_wallet/load_wallets have no callers), so
the restart-resurrection scenario cannot occur.
18 new tests cover the above; 166 passing total.
Review triage — all findings addressed in 0961b0dFixed (19):
Declined (3), with rationale:
18 new tests cover the fixes; 166 passing, lint + audit clean. |
Summary
Fixes every issue in the adversarial codebase review (
.roast/REPORT-latest.md): 6 High, 10 Medium, 5 Low, plus the CI/supply-chain findings.API security (High 1, 2)
HYPERDATA_API_HOST=0.0.0.0is now refused unlessHYPERDATA_API_KEYis set (auth enforced on all non-health routes viaAuthorization: BearerorX-API-Key) orHYPERDATA_UNSAFE_PUBLIC_API=1explicitly accepts the risk.HYPERDATA_CORS_ORIGINSallowlist, otherwise no CORS headers.ensure_futurefan-out). Inbound message size cap, rate cap, and bad-message disconnect threshold. Heartbeat loop logs failures instead ofpass.Reliability (High 3–5; Med 8, 9)
status.failed_components, force/v1/healthtodegraded, and the "all components online" log is only emitted when it's true.position_scanner/market_datareportstarting→connected/errorfrom their actual loop outcomes instead of a blindready.parse_errorscounters (exposed in liquidation stats) — no more crash/reconnect loops on schema drift.venue_freshness(), surfaced in/v1/health) so a dead venue can't hide behind the combined feed; the status loop body is isolated so one badget_stats()can't kill the staleness watchdog.Data integrity (Med 1, 3–5)
data/corrupted/with a timestamp instead of deleted.schema_versiontable + versioned migrations; onlyduplicate columnis treated as already-applied, anything else raises.0x+ 40 hex), lowercased, capped at 50k with LRU-style expiry.Strategy safety (Med 2, 6, 7)
confidenceper wallet plus ranking-criteria caveats in stats, and clears stale smart/dumb tiers when a wallet stops qualifying.LLM_MAX_EVALS_PER_HOURbudget.Tests & CI (High 6, Low 1–5)
tests/test_position_scanner.pyrewritten against the SQLite address store (all network mocked) and re-enabled in CI.tests/test_hardening.py(43 tests): bind guard, auth/CORS/rate-limit middleware over a live test server, WS abuse limits, dedup/cascade, malformed payloads, degraded hub startup, paper-trader invariants, LLM parsing, DB quarantine, schema versioning, alert redaction, per-venue freshness.requirements.lock, advisorypip-auditstep added.api_server.py; README/API docs foreground the exposure model; alert logs redact payloads (wallet intelligence no longer sits in rotating logs); dashboard menu input no longer mutates executor internals.Test plan
python -m pytest tests/ -q— 148 passed, 2 skipped (live-marked)ruff check src/ tests/— cleanpy_compile run_api.py run_dashboard.py+ import smoke checks — pass