Skip to content

feat: expose custom_metadata_checksum and ConcurrencyConflictError for optimistic concurrency control [PYSDK-130]#632

Open
akunft wants to merge 7 commits into
mainfrom
feat/expose-custom-metadata-checksum-param
Open

feat: expose custom_metadata_checksum and ConcurrencyConflictError for optimistic concurrency control [PYSDK-130]#632
akunft wants to merge 7 commits into
mainfrom
feat/expose-custom-metadata-checksum-param

Conversation

@akunft

@akunft akunft commented May 7, 2026

Copy link
Copy Markdown
Collaborator

🛡️ Implements PYSDK-130 following CC-SOP-01 Change Control, part of our ISO 13485-certified QMS | Ketryx Project

Summary

  • Adds optional custom_metadata_checksum keyword-only parameter to Run.update_custom_metadata(), Run.update_item_custom_metadata() (platform layer), all four service-layer methods, and both CLI commands (run update-metadata --checksum, run update-item-metadata --checksum). Defaults to None — fully backward compatible.
  • Introduces ConcurrencyConflictError(ValueError) in platform/_exceptions.py; HTTP 412 from the server now raises this instead of a generic ValueError, letting callers distinguish concurrency conflicts from invalid-ID errors. CLI exits with code 3 on conflict.
  • GUI metadata editor now forwards custom_metadata_checksum to the service call, making the existing ConcurrencyConflictError handler reachable and preventing silent lost updates.
  • Adds --show-checksum flag to run dump-metadata and run dump-item-metadata CLI commands. Default output unchanged (bare custom_metadata dict). With the flag, emits {"custom_metadata": {...}, "custom_metadata_checksum": "..."} — enabling a first-class read→update loop without a separate API call.
  • Fixes parquet row-count assertion in assert_parquet_geojson_parity (columns=[]columns=["geometry"]) — columns=[] returns 0 rows under pyarrow 23.0.1 (Python 3.14 engine), breaking e2e parity checks.
  • Removes dead SPOT_4_* test constants (unreferenced).

Test plan

  • make lint passes (mypy, pyright, ruff)
  • uv run pytest -m unit — 719 passed, 0 failures
  • uv run aignostics application run dump-metadata --help shows --show-checksum option
  • uv run aignostics application run dump-item-metadata --help shows --show-checksum option
  • uv run aignostics application run update-metadata --help shows --checksum option
  • uv run aignostics application run update-item-metadata --help shows --checksum option
  • e2e: run dump-metadata RUN_ID --show-checksum → copy checksum → run update-metadata RUN_ID '{...}' --checksum <X> succeeds; repeat with stale checksum → exits 3 with "modified by another process"
  • e2e: GUI — edit run metadata in two browser tabs → second save shows the concurrency-conflict warning instead of silently overwriting
  • CI matrix green

Posted by Claude claude-sonnet-4-6 via Claude Code, applying skills cc-sop-01 on behalf of Andreas Kunft (andreas@aignostics.com)

@akunft akunft added skip:test:long_running Skip long-running tests (≥5min) sop:cc-sop-01 CC-SOP-01 Change Control (feature / planned change) type:feature New functionality (conventional feat) labels May 7, 2026
Copilot AI review requested due to automatic review settings May 7, 2026 09:27
@akunft akunft self-assigned this May 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds optimistic concurrency control support for run/item custom metadata updates by introducing an optional checksum parameter and surfacing a dedicated conflict error for HTTP 412 responses.

Changes:

  • Added keyword-only custom_metadata_checksum plumbing from CLI → application service → platform Run.update_*_custom_metadata() calls.
  • Added ConcurrencyConflictError(ValueError) and mapped HTTP 412 (precondition failed) to this error in the application service layer.
  • Updated and extended unit tests to validate checksum forwarding and 412→ConcurrencyConflictError behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/aignostics/application/service_test.py Updates assertions for the new checksum argument and adds tests for checksum forwarding + HTTP 412 mapping.
src/aignostics/system/_exceptions.py Introduces ConcurrencyConflictError for optimistic concurrency conflicts.
src/aignostics/platform/resources/runs.py Adds custom_metadata_checksum kw-only parameter and forwards it into the OpenAPI request model.
src/aignostics/application/_service.py Accepts checksum in service APIs and converts HTTP 412 ApiException into ConcurrencyConflictError.
src/aignostics/application/_cli.py Adds --checksum option to metadata update commands and handles ConcurrencyConflictError.

Comment thread src/aignostics/system/_exceptions.py Outdated
Comment thread src/aignostics/application/_service.py Outdated
Comment thread src/aignostics/application/_cli.py
Comment thread src/aignostics/application/_cli.py
Comment thread src/aignostics/application/_cli.py Outdated
Comment thread src/aignostics/application/_cli.py
@codecov

codecov Bot commented May 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.21053% with 6 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/aignostics/application/_service.py 63.63% 2 Missing and 2 partials ⚠️
...application/_gui/_page_application_run_describe.py 33.33% 2 Missing ⚠️
Files with missing lines Coverage Δ
src/aignostics/application/_cli.py 74.39% <100.00%> (-1.16%) ⬇️
src/aignostics/platform/__init__.py 100.00% <100.00%> (ø)
src/aignostics/platform/_exceptions.py 100.00% <100.00%> (ø)
src/aignostics/platform/resources/runs.py 84.67% <ø> (ø)
...application/_gui/_page_application_run_describe.py 70.13% <33.33%> (-0.27%) ⬇️
src/aignostics/application/_service.py 61.46% <63.63%> (-2.73%) ⬇️

... and 4 files with indirect coverage changes

@akunft akunft removed the skip:test:long_running Skip long-running tests (≥5min) label May 7, 2026
Copilot AI review requested due to automatic review settings May 7, 2026 12:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Comment thread tests/aignostics/application/service_test.py Outdated
Copilot AI review requested due to automatic review settings May 8, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

@akunft akunft marked this pull request as ready for review May 8, 2026 19:45
@claude

claude Bot commented May 8, 2026

Copy link
Copy Markdown

PR #632 Code Review Complete

Comprehensive review of optimistic concurrency control implementation.


Executive Summary

This PR successfully implements optimistic concurrency control via an optional custom_metadata_checksum parameter and a new ConcurrencyConflictError exception. The implementation is backward compatible, well-tested, and follows the repository's architectural patterns.

Overall Assessment: High-quality implementation with one minor fix required


Critical Findings (BLOCKING)

⚠️ Line Length Violations (Ruff E501)

Location: src/aignostics/application/_cli.py

  • Line 1222: 131 characters (exceeds 120 limit)
  • Line 1279: 131 characters (exceeds 120 limit)

Both lines contain:

"The server rejects the update with HTTP 412 if the metadata was modified since this checksum was read."

Fix: Split across more lines:

help=(
    "Optional checksum for optimistic concurrency control. "
    "The server rejects the update with HTTP 412 if the metadata "
    "was modified since this checksum was read."
),

Verify: ruff check src/aignostics/application/_cli.py --select E501


Code Quality Review (Non-Blocking)

✅ Test Coverage: 85.18% (Above 85% Minimum)

  • ✅ Happy path tested (checksum forwarding)
  • ✅ HTTP 412 → ConcurrencyConflictError tested
  • ⚠️ 4 lines missing: error handlers for HTTP 422 and generic exceptions (acceptable)

✅ Test Markers: All Present

All 6 new tests have @pytest.mark.unit:

  • test_application_run_update_custom_metadata_with_checksum
  • test_application_run_update_custom_metadata_412_raises_concurrency_error
  • test_application_run_update_item_custom_metadata_with_checksum
  • test_application_run_update_item_custom_metadata_412_raises_concurrency_error
  • test_cli_run_update_metadata_success_with_checksum
  • test_cli_run_update_item_metadata_success_with_checksum

Verification: uv run pytest tests/aignostics/application/ -m "not unit and not integration and not e2e" --collect-only0 tests selected

✅ Conventional Commits: All Valid

All 5 commits follow type(scope): description format.


Architecture & Design Review

✅ Exception Export (Copilot Was INCORRECT)

Copilot claimed: ConcurrencyConflictError not re-exported from public module

Reality: Exception IS properly exported in src/aignostics/system/__init__.py:

from ._exceptions import ConcurrencyConflictError
__all__ = ["ConcurrencyConflictError", "Service", "Settings", "cli"]

Usage: from aignostics.system import ConcurrencyConflictErrorPublic import

Verdict: Copilot's review was outdated (fixed in commit b58c302)

✅ Backward Compatible Design

  • Parameter is keyword-only with default None
  • ConcurrencyConflictError subclasses ValueError (existing except ValueError still catches it)
  • No API signature changes

✅ HTTP 412 Error Handling

Correct mapping in application/_service.py:

if e.status == HTTPStatus.PRECONDITION_FAILED:  # HTTP 412
    message = f"Custom metadata for run '{run_id}' was modified..."
    raise ConcurrencyConflictError(message) from e

✅ Modulith Compliance

Clean dependency flow: CLI → Service → Platform (no circular dependencies)


Security & Breaking Changes

✅ No Security Issues

  • No secrets/tokens in code
  • No sensitive data exposure
  • Error messages safe

✅ No Breaking Changes

  • New parameter is optional
  • Exception hierarchy preserves compatibility
  • No environment variable changes

Comparison with Bot Reviews

Codecov: ✅ Acceptable

85.18% patch coverage (above 85% minimum), missing lines are error handlers (non-critical)

SonarQube: ✅ Quality Gate Passed

  • 0 new issues
  • 0 security hotspots

Copilot: Partially Outdated

  1. ❌ "ConcurrencyConflictError not re-exported" - INCORRECT (already fixed)
  2. ✅ "Line length violations" - CORRECT (blocking issue)
  3. ❌ "Tests import from private modules" - INCORRECT (already fixed in 7e2318f)

Summary of Strengths

  • ✅ Backward compatible design
  • ✅ Complete parameter threading through all layers
  • ✅ Correct HTTP 412 error mapping
  • ✅ Comprehensive unit tests with proper markers
  • ✅ Clear documentation and docstrings
  • ✅ No security issues or breaking changes
  • ✅ All conventional commits follow format

Action Items

Must Fix (Blocking)

  • Fix line 1222 in _cli.py (split help text to <120 chars)
  • Fix line 1279 in _cli.py (split help text to <120 chars)
  • Run make lint locally to verify
  • Push changes

Nice to Have (Future PRs)

  • Add HTTP 422 test for UNPROCESSABLE_ENTITY error path
  • Add integration test against staging API with actual checksum validation

Verdict: ❌ claude:review:failed

Reason: Line length violations are the only blocking issue. Once fixed, this PR is ready to merge.

The implementation is excellent and raises the bar for the codebase. After the line length fix, this will be a high-quality addition to the SDK.

View job run

@claude claude Bot added the claude:review:failed Automated Claude PR review found blocking issues on the current head commit label May 8, 2026
@sonarqubecloud

sonarqubecloud Bot commented May 9, 2026

Copy link
Copy Markdown

akunft added 7 commits June 16, 2026 10:01
…pdate_item_custom_metadata

Adds optional optimistic concurrency control to run and item custom
metadata updates. When `custom_metadata_checksum` is provided the
server rejects the write with HTTP 412 if the metadata was modified
since the checksum was read; pass `None` (default) to skip the check.

Changes:
- platform/resources/runs.py: add keyword-only `custom_metadata_checksum`
  param to `Run.update_custom_metadata` and `Run.update_item_custom_metadata`,
  forwarded to `CustomMetadataUpdateRequest`
- application/_service.py: propagate `custom_metadata_checksum` through
  `ApplicationService.application_run_update_custom_metadata`,
  `application_run_update_item_custom_metadata`, and both static wrappers;
  handle HTTP 412 explicitly as `ValueError` with a clear concurrency-conflict
  message instead of the generic `RuntimeError` fallback
- tests: add unit tests for checksum forwarding (None and non-None) at the
  `Run` resource layer; add service-layer tests for checksum propagation and
  412→ValueError mapping; fix stale assertions to include `custom_metadata_checksum=None`
- Re-export ConcurrencyConflictError from aignostics.system public API
- Update imports in _service.py and _cli.py to use public path
- Add unit tests for CLI --checksum success path and ConcurrencyConflictError handler on both update-metadata and update-item-metadata commands
Extract repeated string literals into named constants:
- APPLICATION_CLI_SERVICE_PATCH_TARGET for the service patch path (5 uses)
- _TEST_METADATA_JSON for the metadata stub (4 uses)
- cast("dict[str, Any]") → cast(dict[str, Any]) in runs.py (4 uses)
Replace internal module imports with public API paths:
- aignostics.platform.resources.runs.ApiException → aignostics.platform
- aignostics.system._exceptions.ConcurrencyConflictError → aignostics.system
…exit code 3, GUI handler

- Move ConcurrencyConflictError from system/_exceptions to platform/_exceptions
  and re-export from aignostics.platform — eliminates application→system
  bidirectional dependency
- Revert cast() string-form regression in runs.py (restore TC006-compliant
  cast("dict[str, Any]", ...) in all four call sites)
- CLI conflict handlers now exit 3 (distinct from exit 2 for not-found)
- Add specific ConcurrencyConflictError handler in GUI metadata editor
  with "reload and retry" guidance instead of generic error message
- Update all imports and test assertions accordingly
- Fix parquet row-count bug under pyarrow 23+ (Python 3.14): columns=[]
  returns 0 rows; switch to columns=["geometry"] which is engine-agnostic
- Fix GUI metadata save never forwarding checksum: add
  custom_metadata_checksum=run_data.custom_metadata_checksum to the
  nicegui_run.io_bound call so ConcurrencyConflictError is reachable
- Add --show-checksum flag to dump-metadata and dump-item-metadata CLI
  commands; default output unchanged (bare dict), flag emits wrapped
  {custom_metadata, custom_metadata_checksum} for read→update loop
- Remove dead SPOT_4_* constants (unreferenced former SPOT_1 values)
- Add unit tests: parquet row-count engine-agnostic guard, --show-checksum
  default/wrapped output for both dump commands (4 CLI + 1 utils test)
@akunft akunft force-pushed the feat/expose-custom-metadata-checksum-param branch from 2e8c224 to 5f22b21 Compare June 16, 2026 13:26
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 70%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude:review:failed Automated Claude PR review found blocking issues on the current head commit sop:cc-sop-01 CC-SOP-01 Change Control (feature / planned change) type:feature New functionality (conventional feat)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants