Skip to content

psycopg load_from_records cannot adapt mappings for JSONB columns #663

Description

@cofin

Summary

PsycopgAsyncConfig advertises supports_native_arrow_import, but load_from_records() fails when a record contains Python mappings destined for PostgreSQL JSON/JSONB columns. The Arrow/COPY writer passes the mappings to psycopg without JSON adaptation and psycopg raises:

cannot adapt type 'dict' using placeholder '%t'

Confirmed downstream reproducer

Litestar Queues uses load_from_records() for its native enqueue_many() path. Queue records contain mappings in args_json, kwargs_json, metadata_json, and result_json. Against a real PostgreSQL instance, the psycopg variant fails while the same public enqueue_many(TaskRequest) scenario passes through asyncpg.

The existing downstream regression documentation independently identifies the same gap in src/tests/integration/backends/sqlspec/test_postgres_psycopg_autocommit_hot_paths.py: its test currently forces the universal execute_many() tier because the psycopg Arrow/COPY tier cannot adapt the native JSON values.

Minimal shape:

from sqlspec.adapters.psycopg import PsycopgAsyncConfig

# obtain a driver using the normal SQLSpec session lifecycle
await driver.load_from_records(
    "queue_tasks",
    [{"id": "...", "kwargs_json": {"value": 1}, "metadata_json": {}}],
)

Expected behavior

load_from_records() should adapt Python mappings for PostgreSQL JSON/JSONB destination columns (or normalize them into a COPY-compatible representation) when using psycopg. The native path should round-trip mappings consistently with ordinary parameterized INSERTs.

Acceptance criteria

  • psycopg sync and async load_from_records() accept mappings for JSON/JSONB columns
  • values round-trip as JSON objects
  • focused real-PostgreSQL coverage exercises multiple JSON-bearing columns
  • downstream consumers can rely on supports_native_arrow_import without adapter-specific fallback gates

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions