Fix blob column handling for binary data with embedded nulls#6
Merged
Conversation
Change Type.Blob from String to (Array Byte) so binary data with embedded null bytes is not truncated. Add SQLiteColumn_from_blob to return blob data as a length-aware Carp Array, and use the stored blob_len in sqlite3_bind_blob instead of strlen.
There was a problem hiding this comment.
Build & Tests
Build: compiles cleanly on ARM Linux.
Tests: all 26 tests pass, including the new blob round-trip test ([0b 1b 0b 2b 3b] with embedded nulls).
CI: passing on both macos-latest and ubuntu-latest.
Findings
None. The fix is correct and addresses the root cause at every layer:
- Type change (
sqlite3.carp:77):Type.Blobnow wraps(Array Byte)instead ofString— the right representation for binary data. - Blob retrieval (
sqlite3_helper.h:194–199): usessqlite3_column_blob+sqlite3_column_bytesinstead ofsqlite3_column_text. Allocates exactlylenbytes (no null terminator) and storesblob_lenin the column struct. - Blob binding (
sqlite3_helper.h:252): usesval.blob_leninstead ofstrlen(val.s)— the critical fix that prevented truncation on write. SQLiteColumn_from_blob(sqlite3_helper.h:70–76): creates a CarpArrayfrom the column'sspointer andblob_len. Ownership transfer is correct — theSQLiteColumnis consumed by value, so no double-free.SQLiteColumn_blob(sqlite3_helper.h:62–68): takes the CarpArrayby value, storesa.lenanda.data. Again, ownership is clean.
Minor note: blob_len is uninitialized for non-blob column types (int, float, text, null). This is fine since it's only read in the blob code path, but it adds sizeof(int) to every SQLiteColumn. Not a concern in practice.
Breaking change (Blob String → Blob (Array Byte)) is clearly documented in the PR description.
Verdict: merge
Correct fix for a real data-corruption bug, CI passing, well-tested.
hellerve
approved these changes
Jun 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
sql_blobcase inSQLiteColumn.to-carpwas usingfrom-text(backed bysqlite3_column_text/SQLiteColumn_from_str), which treats data as a null-terminated C string. Binary blobs containing embedded null bytes were silently truncated.Changes:
Type.Blobnow wraps(Array Byte)instead ofString, preserving exact blob lengthblob_lenfield toSQLiteColumnstruct to carry the byte count through the C layerSQLiteColumn_from_blobto convert blob data into a length-aware Carp Arraysqlite3_bind_blobcall to useblob_leninstead ofstrlen, which also truncated on bind[0b 1b 0b 2b 3b]) round-trips correctlyBreaking change:
Type.Blobnow holds(Array Byte)instead ofString. Code constructing or pattern-matching onBlobvalues needs updating.Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.