Remove diskann-platform; tidy diskann-disk instrumentation & reader modules#1205
Remove diskann-platform; tidy diskann-disk instrumentation & reader modules#1205suri-kumkaran wants to merge 1 commit into
diskann-platform; tidy diskann-disk instrumentation & reader modules#1205Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors the disk-index stack by removing the diskann-platform crate, moving platform I/O and perf/timing utilities into diskann-disk, and updating dependent crates (diskann-tools, diskann-benchmark, diskann-providers) to use the new module boundaries.
Changes:
- Removed
diskann-platformfrom the workspace and relocated its platform-specific I/O primitives intodiskann-disk::search::provider::aligned_file_reader::platform. - Consolidated instrumentation by moving
Timerintodiskann-disk::utils::instrumentationand decouplingdiskann-providersfrom perf instrumentation dependencies/features. - Reorganized
aligned_file_readerunderdiskann-disk::search::provider, and addedDiskVertexProviderFactory::from_disk_index_pathto simplify higher-level setup.
Reviewed changes
Copilot reviewed 55 out of 61 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| diskann-tools/src/utils/search_disk_index.rs | Updates imports to the relocated aligned reader traits and instrumentation paths. |
| diskann-tools/src/utils/build_pq.rs | Switches Timer usage to diskann-disk instrumentation and tidies provider imports. |
| diskann-tools/src/utils/build_disk_index.rs | Moves Timer dependency from providers to diskann-disk instrumentation. |
| diskann-tools/src/bin/compute_multivec_groundtruth.rs | Updates Timer import location. |
| diskann-tools/src/bin/compute_groundtruth.rs | Updates Timer import location. |
| diskann-tools/Cargo.toml | Drops diskann-providers/perf_test from the tools perf_test feature wiring. |
| diskann-providers/src/utils/mod.rs | Removes the providers-local Timer module export. |
| diskann-providers/src/model/pq/pq_construction.rs | Replaces Timer usage with std::time::Instant for simple timing. |
| diskann-providers/Cargo.toml | Removes opentelemetry/diskann-platform deps and drops the perf_test feature. |
| diskann-platform/src/win/thread_safe_handle.rs | Deleted as part of removing diskann-platform. |
| diskann-platform/src/win/ssd_io_context.rs | Deleted as part of removing diskann-platform. |
| diskann-platform/src/macos/ssd_io_context.rs | Deleted as part of removing diskann-platform. |
| diskann-platform/src/macos/mod.rs | Deleted as part of removing diskann-platform. |
| diskann-platform/src/linux/mod.rs | Deleted as part of removing diskann-platform. |
| diskann-platform/src/lib.rs | Deleted as part of removing diskann-platform. |
| diskann-platform/Cargo.toml | Deleted as part of removing diskann-platform. |
| diskann-disk/src/utils/mod.rs | Removes old utils::aligned_file_reader exports now that it moved under search/provider. |
| diskann-disk/src/utils/instrumentation/timer/windows.rs | New Windows perf/timing helpers moved under diskann-disk instrumentation. |
| diskann-disk/src/utils/instrumentation/timer/macos.rs | New macOS perf/timing helpers moved under diskann-disk instrumentation. |
| diskann-disk/src/utils/instrumentation/timer/linux.rs | New Linux perf/timing helpers moved under diskann-disk instrumentation. |
| diskann-disk/src/utils/instrumentation/timer.rs | Replaces diskann_platform::* with per-OS timer backends inside diskann-disk. |
| diskann-disk/src/utils/instrumentation/perf_logger.rs | Switches PerfLogger to use diskann-disk’s Timer. |
| diskann-disk/src/utils/instrumentation/mod.rs | Re-exports Timer from the new instrumentation module layout. |
| diskann-disk/src/utils/aligned_file_reader/mod.rs | Deleted; aligned file reader moved under search/provider. |
| diskann-disk/src/storage/quant/pq/pq_generation.rs | Moves Timer import to diskann-disk instrumentation. |
| diskann-disk/src/storage/quant/generator.rs | Moves Timer import to diskann-disk instrumentation. |
| diskann-disk/src/search/provider/mod.rs | Exposes the relocated aligned_file_reader module. |
| diskann-disk/src/search/provider/disk_vertex_provider.rs | Updates aligned reader imports to the new module location. |
| diskann-disk/src/search/provider/disk_vertex_provider_factory.rs | Adds from_disk_index_path and updates aligned reader factory imports. |
| diskann-disk/src/search/provider/disk_sector_graph.rs | Updates aligned reader imports to the new module location. |
| diskann-disk/src/search/provider/disk_provider.rs | Switches setup to DiskVertexProviderFactory::from_disk_index_path and updates imports. |
| diskann-disk/src/search/provider/cached_disk_vertex_provider.rs | Updates aligned reader trait import path. |
| diskann-disk/src/search/provider/aligned_file_reader/traits/mod.rs | Adjusts trait module structure and visibility. |
| diskann-disk/src/search/provider/aligned_file_reader/traits/aligned_reader_factory.rs | New aligned reader factory trait definition. |
| diskann-disk/src/search/provider/aligned_file_reader/traits/aligned_file_reader.rs | Updates imports to the new aligned read location. |
| diskann-disk/src/search/provider/aligned_file_reader/reader/windows.rs | Updates platform primitive imports to the new internal platform module. |
| diskann-disk/src/search/provider/aligned_file_reader/reader/storage_provider.rs | Updates imports to the new aligned reader module paths. |
| diskann-disk/src/search/provider/aligned_file_reader/reader/mod.rs | New module that selects concrete reader implementations by platform. |
| diskann-disk/src/search/provider/aligned_file_reader/reader/linux.rs | Updates IOContext import to the new platform module. |
| diskann-disk/src/search/provider/aligned_file_reader/platform/windows/ssd_io_context.rs | New Windows IO context type colocated with aligned reader platform primitives. |
| diskann-disk/src/search/provider/aligned_file_reader/platform/windows/mod.rs | Trims exports and rehomes IOContext and Win32 types for aligned readers. |
| diskann-disk/src/search/provider/aligned_file_reader/platform/windows/io_completion_port.rs | Fixes internal imports and modernizes logging formatting. |
| diskann-disk/src/search/provider/aligned_file_reader/platform/windows/file_io.rs | Fixes internal imports to the new platform module structure. |
| diskann-disk/src/search/provider/aligned_file_reader/platform/windows/file_handle.rs | Adds dead_code allowances and improves formatting in errors/logs. |
| diskann-disk/src/search/provider/aligned_file_reader/platform/mod.rs | New platform dispatch module for aligned reader primitives. |
| diskann-disk/src/search/provider/aligned_file_reader/platform/linux.rs | Documents/ensures Linux IOContext keeps the registered FD alive. |
| diskann-disk/src/search/provider/aligned_file_reader/mod.rs | New aligned file reader module root under search/provider. |
| diskann-disk/src/search/provider/aligned_file_reader/factory/virtual_storage.rs | Updates internal module paths after refactor. |
| diskann-disk/src/search/provider/aligned_file_reader/factory/mod.rs | New factory module root for aligned reader factories. |
| diskann-disk/src/search/provider/aligned_file_reader/factory/file.rs | Updates internal imports after module reorganization. |
| diskann-disk/src/search/provider/aligned_file_reader/aligned_read.rs | New typed alignment-checked IO read request type. |
| diskann-disk/src/build/builder/core.rs | Updates tests/imports to use relocated Timer and aligned reader factories. |
| diskann-disk/Cargo.toml | Removes diskann-platform, updates platform deps (io-uring/windows-sys), keeps perf feature wiring. |
| diskann-disk/benches/benchmarks/aligned_file_reader_bench.rs | Updates aligned reader import path for benchmarks. |
| diskann-disk/benches/benchmarks_iai/aligned_file_reader_bench_iai.rs | Updates aligned reader import path for IAI benchmarks. |
| diskann-benchmark/src/disk_index/search.rs | Uses DiskVertexProviderFactory::from_disk_index_path to reduce low-level coupling. |
| diskann-benchmark/Cargo.toml | Drops diskann-providers/perf_test from disk-index feature wiring. |
| Cargo.toml | Removes diskann-platform from workspace members and dependencies. |
| Cargo.lock | Removes diskann-platform package entries and reflects updated platform deps. |
| agents.md | Updates tier listing to remove diskann-platform. |
| .github/workflows/disk-benchmarks.yml | Updates path filters to stop triggering on the removed diskann-platform crate. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #[cfg(test)] | ||
| pub(crate) use factory::VirtualAlignedReaderFactory; |
| #[cfg(test)] | ||
| mod virtual_storage; | ||
| #[cfg(test)] | ||
| pub(crate) use virtual_storage::VirtualAlignedReaderFactory; |
Codecov Report❌ Patch coverage is
❌ Your patch status has failed because the patch coverage (7.14%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #1205 +/- ##
==========================================
+ Coverage 89.79% 90.82% +1.02%
==========================================
Files 488 488
Lines 93312 93319 +7
==========================================
+ Hits 83791 84757 +966
+ Misses 9521 8562 -959
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Folds the
diskann-platformcrate intodiskann-disk, decouplesdiskann-providersfromperf instrumentation, and reorganizes the affected
diskann-diskmodules. No behavior change;pure refactor.
Changes
Timerout ofdiskann-providers(its one timing use now usesstd::time::Instant); drop providers'diskann-platform,opentelemetrydeps andperf_testfeature.
diskann-platform: perf counters now live as a private detail ofTimer; the ioprimitives (io_uring / Windows IOCP / file handles) move next to their only consumer,
aligned_file_reader. Workspace, CI path filters, and docs updated.Timer+ platform perf counters now sit inutils/instrumentation/besidePerfLogger(perf → Timer → PerfLogger). Canonical path isutils::instrumentation::Timer.aligned_file_reader:utils/→search/provider/(its sole consumer area) andreorganized internally into
reader/,factory/,traits/,platform/.DiskVertexProviderFactory::from_disk_index_path, removingthe benchmark's coupling to the low-level reader factory.