feat(extensions): non-numeric fact substrate (migration 0021) + calc-overlay precedence fix#888
Merged
Merged
Conversation
The graph Fact node has modeled non-numeric facts since inception; the OLTP facts table was numeric-only (value FLOAT NOT NULL). Close the asymmetry: value becomes nullable; add string_value, fact_type, value_type, content_type, and decimals with a numeric-XOR-nonnumeric CHECK. Element gains item_type (value domain, orthogonal to element_type). fact_sets admits the 'disclosure' type for standing text-block binding sets, and structure.py names the TEXT_BLOCK_CAPS subset. TenantOps fan-out to public + every tenant schema; NOT NULL columns backfill via a temporary DEFAULT then drop it so migrated schemas match freshly provisioned ones.
…item_type Stop hardcoding the graph Fact's non-numeric slots: pass fact_type, value_type, and content_type through from the OLTP row, COALESCE string_value into the value column, and fall back to the legacy '-2' decimals only for numeric rows (Nonnumeric carries NULL, matching XBRL). Unit nodes now derive from the facts actually present (numeric only — nonNumeric facts carry no unitRef) with the static unit_usd row retained for fact-less graphs, and FACT_HAS_UNIT reads facts.unit instead of pinning every fact to USD. Element staging maps item_type through and derives is_textblock/is_numeric/is_integer/is_shares from it, with NULL item_type keeping the legacy numeric defaults — existing graph output is byte-identical.
…readers Invert the calc-DAG overlay precedence: a structure's own calculation arcs are its footing spec, so a disclosure note that decomposes a global calc parent (Revenues, OperatingExpenses, ...) now foots against its own members instead of the statement-level children absent from its FactSet (which reported the rollup skipped or falsely failed). The merge lives in calc_dag.py as merge_calculations — local wins per parent, global as fallback, pure. With local-wins the all-children- absent skip guard becomes semantically correct for locally-arced parents; locked by tests. Also guard every numeric fact reader for the new non-numeric arm: rule variable binding, SumEquals aggregation, period-balance loading, statement rendering, and the get_statement fact query all filter to numeric facts so a text-block fact neither crashes float paths nor counts as present/zero in footing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First half of the text-block-facts increment (report-engine M3): the OLTP fact model gains the non-numeric value path the graph Fact node has always had, and materialize passes it through instead of hardcoding.
facts.valuenullable +string_value/fact_type/value_type/content_type/decimalswith a numeric-XOR-nonnumeric CHECK;elements.item_type;fact_setsadmits'disclosure';TEXT_BLOCK_CAPSnamed next to the CAP vocabulary.FACT_HAS_UNITderive fromfacts.unit(numeric facts only) instead of pinning USD; Elementitem_type/is_textblockmapped through. Existing graph output is byte-identical (verified on the showcase demo: same fact_type/decimals/value_type, singleunit_usd, 1:1 unit edges, legacy element flags).calc_dag.merge_calculations), so a disclosure note decomposing a global calc parent foots against its own members. Global remains the fallback.No writer behavior changes in this PR — the authoring surface (Document→text-block bind, picker, envelope rendering) follows in the second PR.
Testing
tests/operations/+tests/taxonomy/+tests/migrations/without maxfail: 2887 passed.