Read and parse a local file once on the chunked GPU read path (#3373)#3374
Open
brendancol wants to merge 2 commits into
Open
Read and parse a local file once on the chunked GPU read path (#3373)#3374brendancol wants to merge 2 commits into
brendancol wants to merge 2 commits into
Conversation
_read_geotiff_gpu_chunked read the source file off disk twice and parsed its IFDs twice at dask-graph-build time: the per-tile byte-count cap block did read_all + parse_header + parse_all_ifds and discarded the result, then the GDS qualification block repeated the same read and parse. IFD parsing is O(tile_count), so a 10k+ tile COG paid that cost twice (plus a second full-file read) per open_geotiff(gpu=True, chunks=...) call. Merge the two passes: read the bytes once and parse header/IFDs once before the cap loop, run the cap check on that single parse, then reuse the same raw/header/ifds for the GDS qualification probe. Preserves the existing contract: the per-tile cap still raises ValueError before any GDS work; a metadata-parse failure (or empty IFD list, or bad overview level) leaves raw/header/ifds as None / falls through to the CPU path instead of raising; the local-file-only guard and the _FileSource context-manager scoping (hardened for the free-threaded race) are unchanged. Graph-build path only, not a decode-throughput change. Adds 3 regression tests: a read_all/parse invocation counter proving the file is read and parsed once (it counted 2 each before the fix), the DoS per-tile cap still raising, and a synthetic parse failure routing to the CPU path without raising. Closes #3373
brendancol
commented
Jun 17, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
PR Review: Read and parse a local file once on the chunked GPU read path (#3373)
Blockers (must fix before merge)
None.
Suggestions (should fix, not blocking)
None.
Nits (optional improvements)
xrspatial/geotiff/_backends/gpu.py:1416: theif not ifds: raise ValueError("No IFDs found in TIFF file")inside the qualificationtryis now only reachable whenparse_all_ifdsreturns an empty list (a non-None, falsy value). That is the same path the old code took, and the broadexceptbelow routes it to the CPU fallback, so behavior is unchanged. A one-line comment noting that an empty list is intentionally routed to CPU would save the next reader the trace. Optional.
What looks good
- The over-cap
ValueErroris raised in the cap loop (line ~1400), which sits before and outside the qualificationtry/except, so the denial-of-service guard still propagates. Verified. - Parse failure, an empty IFD list, and a bad overview level all leave
raw/header/ifdsasNoneor fall through the broadexcept, routing to_read_geotiff_daskwithout raising, which matches the prior two-parse semantics. header.byte_orderandraware only dereferenced underif ifds is not None:, where a successful parse guarantees both are set.- The
_FileSourcecontext-manager scoping (hardened for the free-threaded race) is preserved: the read still happens inside thewithblock. - The invocation-counter test has teeth. It counts 2 reads/parses on the pre-fix code and 1 after, so the regression cannot silently return.
Checklist
- Behavior matches the prior implementation (cap raises, failures fall through)
- GPU dask path still returns a lazy cupy-backed array
- Cap / DoS guard preserved
- Edge cases (empty IFDs, bad overview, parse failure) covered
- No premature materialization or unnecessary copies (one fewer read, one fewer parse)
- Benchmark not needed (graph-build path, geotiff not in the benchmarked set)
- README feature matrix not applicable (no API change)
- Tests present and meaningful (487 passed, 3 skipped in tests/gpu/)
brendancol
commented
Jun 17, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
PR Review (follow-up): #3373
The single optional nit from the first pass is addressed in d5c96b0: the empty-IFD-list path now carries a comment explaining it falls through to the CPU reader.
No remaining findings. The fix preserves the cap/DoS raise, the parse-failure and empty-IFD fall-through to the CPU path, and the _FileSource scoping. The 3373 regression tests still pass (3 passed), and the wider tests/gpu/ suite was green (487 passed, 3 skipped).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3373.
_read_geotiff_gpu_chunkedread a local source file off disk twice and parsed its IFDs twice at dask-graph-build time. The per-tile byte-count cap block didread_all+parse_header+parse_all_ifdsand discarded the result; the GDS qualification block right after repeated the same read and parse. IFD parsing is O(tile_count), so a COG with 10k+ tiles paid that cost twice (plus a second full-file read) on everyopen_geotiff(gpu=True, chunks=...)call.Change
raw/header/ifdsfor the GDS qualification probe.Contract preserved:
ValueError(the denial-of-service guard) before any GDS work runs.raw/header/ifdsasNoneand falls through to the CPU path (_read_geotiff_dask) instead of raising, exactly as before._FileSourcecontext-manager scoping (hardened for the free-threaded race) are unchanged.This is graph-construction overhead only, not a decode-throughput change.
Backends
GPU read path (
gpu=True, chunks=...). The dask+cupy chunked read still returns a lazy cupy-backed array; numpy, cupy-eager, and dask+numpy paths are untouched.Tests
Added three regression tests in
xrspatial/geotiff/tests/gpu/test_reader.py:read_all/parse_header/parse_all_ifdsinvocation counter proving the local chunked GPU read parses once (it counted 2 each before the fix).tests/gpu/(487 passed, 3 skipped) andgolden_corpus/test_dask_gpu.pypass.No ASV: geotiff is not in the performance-labeler benchmarked set, so the
performancelabel does not trigger benchmarks here.