First live observations: Insight Cards & Entity Extraction (v10.55)

[doobidoo]

What this is

After enabling MCP_INSIGHT_CARDS_ENABLED=true and running a full maintain cycle today, here are the first-hand observations from using Insight Cards (#869) and Entity Extraction (#868) in a real memory corpus of ~9,300 memories.

Credit where it is due:

• 🔗 Entity Extraction — implemented by @doobidoo in PR feat(reasoning): entity extraction and memory-entity linking (Phase 2, #732) #868
• 💡 Insight Cards — implemented by @doobidoo in PR feat(consolidation): Insight Cards — automated pattern/trend/gap detection (Phase 3, #732) #869
• 🏷️ tag_match AND/OR filtering — contributed by @filhocf in PR [Feature]: tags filter that narrows down results (aka AND filter) #889

---

The maintain cycle output

With all 6 steps running (MCP_INSIGHT_CARDS_ENABLED=true, consolidation enabled):

Step 1 — Cleanup:           0 duplicates
Step 2 — Conflicts:         0 semantic conflicts
Step 3 — Stale:             8,843 memories > 30 days
Step 4 — Quality:           9,273 scored, avg 0.504
Step 5 — Entity extraction: 500 scanned → 1,104 entity links stored
Step 6 — Insight cards:     189 generated, 6 net-new stored
                            (103 pattern / 17 trend / 69 gap)
                            Elapsed: 3.7 seconds

The deduplication on Step 6 is working well — 183 previously-stored cards correctly skipped, only 6 genuinely new ones written.

---

Key observations from the first 106 insight cards

1. conflict:unresolved gap — known false positive

The gap detector flagged: "Tag 'conflict:unresolved' has 140 memories but no decisions recorded."

This is a false positive. The tag is applied by the session consolidation hook as a status marker on session summaries ("this session had unresolved items"). It is not a knowledge domain requiring decision documentation.

Takeaway for future improvement: the InsightGenerator gap detector would benefit from a configurable exclusion list of tags that are metadata/status markers rather than knowledge domains (e.g. conflict:unresolved, automated, __test__).

2. radar:2026-04-02 — real data quality issue surfaced

64 LinkedIn posts harvested by an agent in April had memory_type=NULL. The insight trend detector tried to sort them and hit:

TypeError: '<' not supported between instances of 'str' and 'NoneType'

This exposed a pre-existing data quality gap and a bug in the InsightGenerator.

Fix shipped as v10.55.2: Three dict.get("key", default) calls in insights.py replaced with or "" / or []. The gotcha: dict.get does not fall back to default when the key exists with a None value. All 64 memories also bulk-updated to memory_type="reference".

3. ci tag — signal/noise problem made visible

The gap detector surfaced: "Tag 'ci' has 2,476 memories but no decisions recorded."

On inspection: 2,276 of those are automated CI run observation dumps. The insight card is technically correct but practically noise. The ci tag does have 39 decisions — they're just drowned out.

Recommendation: use memory_type=observation + tag temporary for automated CI dumps so the 7-day retention policy auto-expires them. Reserve decision/learning types for actionable CI outcomes.

4. Architecture trend reversal — real gap caught

The trend detector noticed the architecture tag recently shifted from decision+pattern → learning+observation. This was accurate — formal architecture decision memories had stopped being recorded as the codebase matured.

Action taken: stored an explicit architecture direction decision memory documenting the current state (modular server layer, Strategy Pattern for storage, HTTP as primary interface, Hybrid sync separation of concerns).

---

How to enable these features

Add to your .env:

# Insight Cards (Step 6 of maintain cycle)
MCP_INSIGHT_CARDS_ENABLED=true

# Entity extraction runs as Step 5 automatically when consolidation is enabled
MCP_CONSOLIDATION_ENABLED=true

Then trigger a maintain cycle via MCP tool:

memory_quality(action="maintain", dry_run=False)

Or via HTTP API:

curl -sk https://localhost:8000/api/memories/quality \
  -H 'X-API-Key: <your-key>' \
  -d '{"action": "maintain"}'

---

Open questions / future improvements

1. InsightGenerator tag exclusion list — some tags are status metadata, not knowledge domains; gap detection should be skippable per-tag
2. CI observation retention policy — should automated run dumps use temporary tag by default to self-expire?
3. Insight card acknowledgement — should users be able to mark a card as "acknowledged / won't fix" to prevent it re-generating on the next cycle?

Happy to hear how others are finding these features on larger corpora. 🧠

[doobidoo]

Addendum — Milvus backend credit

Worth mentioning here: the maintain cycle (Steps 1–6 including entity extraction and insight cards) working on Milvus backends depends on the consolidation protocol having a functional delete_memory proxy — which @henry201605 contributed in PR #872. Without that fix, Steps 3–6 would fail silently on Milvus.

@henry201605 is also actively testing the new v10.55 features on Milvus (as evidenced by PR #898). Would be great to hear your observations on how entity extraction and insight cards behave on a Milvus corpus! 🐋

[doobidoo]

Status update on the three open questions (v10.57.3):

1. InsightGenerator tag exclusion list — not yet implemented. The gap detector still fires on status/metadata tags like conflict:unresolved and automated. A configurable MCP_INSIGHT_EXCLUDE_TAGS=conflict:unresolved,automated,__test__ would be the clean solution — or a heuristic that skips tags where >90% of memories share a single memory_type. PR welcome if anyone wants to take this on.

2. CI observation retention policy — the recommendation in the post stands: use memory_type=observation + tag temporary for automated run dumps so the 7-day retention policy auto-expires them. Not enforced automatically yet; this requires a convention change on the write side.

3. Insight card acknowledgement — no "acknowledged / won't fix" flag yet. For now, the workaround is deleting the unwanted card; it will re-generate on the next cycle unless the underlying tag pattern changes. A memory_type=insight + tag acknowledged filter in the InsightGenerator would solve this cleanly.

---

Curious to hear from others running on different corpus sizes — particularly the entity extraction yield (links per 500 scanned) and the insight card survival rate (net-new vs. generated). On this 9,300-memory corpus the ratio was 6/189 (3.2%); I would expect that to be higher on a younger corpus where fewer cards have been stored previously. @henry201605 — how are Steps 5 and 6 looking on your Milvus corpus?

[filhocf]

Great to see the features in action on a real corpus! Here are our numbers for comparison — smaller scale but same stack.

Our Setup

• Backend: sqlite-vec (single user, 3 machines synced via OneDrive)
• Corpus: ~2,700 memories (grew from ~1,800 after enabling harvest with Kiro CLI support)
• Embedding model: all-MiniLM-L6-v2 (default)
• Version: v10.57.3 (fork with Kiro CLI parser + pt_BR locale patterns)

Observations

Entity Extraction:

• Works well on structured memories (decisions, conventions)
• @mentions and #tags extraction is precise — no false positives in our corpus
• We use it primarily for cross-referencing project names and tool names across sessions

Insight Cards:

• The pattern detection surfaces useful trends (e.g., recurring bugs in the same subsystem)
• Gap detection identified areas where we had decisions but no follow-up implementation — actionable
• With ~2,700 memories the maintain cycle completes in <10s

tag_match (PR #904):

• Essential for our workflow — we use tag_match="all" to scope searches to specific project + type combinations (e.g., tags=["mir-sistema", "decision"])
• The AND filtering reduced noise significantly vs the previous OR-only behavior

Harvest + locale patterns (our fork):

• With pt_BR patterns enabled: 75 candidates per session vs 9 with English-only — 8x improvement
• Design discussions opened for upstream: Design: Kiro CLI session format support for memory_harvest #927 (Kiro CLI parser) and Design: Multilingual pattern plugins for harvest extractor #928 (locale plugins)

Thanks for the credit on tag_match — and for shipping + maintaining the entity extraction and insight cards code. The v10.55→v10.57 arc is solid.

[doobidoo]

Thank you for these detailed observations — they directly shaped v10.58.0 (released today, May 16 2026)!

All three issues you raised are now addressed:

1. Noisy system tags in gap detection — New MCP_INSIGHT_EXCLUDE_TAGS env var lets you specify project-specific tags (e.g. MCP_INSIGHT_EXCLUDE_TAGS=ci,radar) to exclude from gap detection, in addition to the built-in exclusion set.

2. Status-marker tags with mostly automated memories — New automated-type heuristic: gap detection is skipped for tags where >90% of memories have an automated memory_type (session, auto-generated, temporary). This catches noise without manual exclusion lists.

3. Acknowledged cards being regenerated — Tagging an insight card with acknowledged now materialises a permanent sentinel memory. The sentinel hash is independent of source memories, so the card is never regenerated even after the original is deleted.

Details: PR #939 | CHANGELOG | Release v10.58.0