Proposal: pluggable dense vector backend and optional Milvus support

[zc277584121]

Hi maintainers,

I was looking at SimpleMem's database layer and noticed that the current VectorStore path appears to combine dense vector search, full-text search, and structured metadata handling around LanceDB / Tantivy / SQL metadata.

Would you be open to a pluggable vector-storage boundary so users can keep the SimpleMem APIs while choosing a different dense vector backend, such as Milvus?

A possible scoped shape:

1. Keep the current LanceDB/Tantivy path as the default implementation.
2. Split a narrow dense-vector backend contract from the higher-level memory/search logic.
3. Add an optional Milvus backend for dense vector storage and retrieval.
4. Keep structured metadata and FTS behavior aligned with the current product semantics.
5. Support Milvus Lite for local development and Milvus server / Zilliz Cloud for larger deployments.
6. Add focused tests for insert/search/filter behavior and score ordering.

Would this kind of backend abstraction be a good fit for SimpleMem, or is the current storage design intended to stay tightly coupled to LanceDB/Tantivy?

[Jiaaqiliu]

Thanks for the thoughtful proposal — this is a reasonable direction.

You're reading the current design correctly: the storage layer is presently coupled to LanceDB (dense vectors + FTS) with structured metadata handled around it, rather than sitting behind a narrow, swappable dense-vector contract. There's nothing philosophically opposed to a pluggable backend; the current coupling is just how it grew, not a deliberate lock-in.

The scoped shape you outlined is sensible — in particular:

1. Keep the LanceDB/Tantivy path as the default.
2. Extract a small dense-vector backend interface (insert / search / filter / score-ordering) from the higher-level memory + hybrid-retrieval logic.
3. Add an optional Milvus backend (Milvus Lite for local, server/Zilliz for scale) behind that interface.
4. Keep structured metadata + FTS semantics aligned with today's behavior.

The main things I'd want to preserve are (a) the hybrid retrieval semantics (vector + keyword + structured filters must stay consistent across backends) and (b) score ordering parity, so results don't silently change for existing users. If you'd like to open a PR starting with just the interface extraction (no behavior change) plus the focused insert/search/filter/ordering tests you mentioned, that's a great first step and I'm happy to review. Thanks again.