feat: optimize native Joplin AI embedding pipeline and clustering quality

[Harsh16gupta]

This PR resolves the core clustering quality and vector space issue that I documented in my last weekly report:

"While testing... I ran into an issue when switching to Joplin's native AI search API... when using the native API, it keeps returning K=2 as the best cluster count. Because the native embeddings don't blend note titles the same way, the vector space ends up more clumped... Also, I'm still noticing some noise notes sitting right on the border between clusters... Another thing I noticed is that the cluster names generated right now aren't that good..."

When switching to Joplin's native AI search API (which uses models with 768 or 1536 dimensions), the clustering quality was poor because:

1. Native embedding vectors were being rejected by a hardcoded 384-dimension check.
2. Note chunks arrived out of order from pagination and were averaged equally, so body text/signatures drowned out the title and opening paragraph.
3. High-dimensional vectors were being reduced using fixed UMAP settings tuned for smaller models, causing points to clump or become noise.

---

Changes in this pr :

1. Flexible Vector Dimension Check: isValidEmbeddingVector now accepts 768D or 1536D native AI vectors instead of rejecting non-384D vectors.
2. Chunk Ordering by chunkIndex: Sorts native chunks in correct order so title and opening text always get top priority.
3. Weighted Chunk Pooling: Uses position decay (weightedAverageVectorsWithNorm) so lead paragraphs carry more weight than body boilerplate.
4. Outlier Filtering: Drops near-zero raw vectors (rawNorm < 1e-6) to prevent corrupted chunks from creating noise on cluster borders.
5. Adaptive UMAP Settings: Auto-scales UMAP dimensions and neighbor settings based on vector size and note count (createAdaptiveConfig).
6. Cache Safety: Stores modelId and dimension in cache metadata so cached local 384D vectors don't collide with native AI vectors.

---

Verification

• Local ONNX pipeline is completely untouched.
• All 185 tests pass (npm test).
• I have manually verified the results, and they are very good.

[Harsh16gupta]

Manually tested the pipeline using Joplin's Native AI search API and the cluster quality is now great!

The main problem was vector dimension validation: high-dimensional native vectors (768D/1536D) were getting rejected (it took a lot of time to debug because I never expected it to be caused by such a small validation check, so I spent a lot of time experimenting with different clustering parameters). After fixing dynamic dimension validation alongside chunk index sorting and adaptive UMAP scaling, Native AI clustering now forms clean, well-separated categories.