fix(asset-export-basic): preload domain cache to fix truncated hierarchy paths (CSA-444)#2561
Open
bladata1990 wants to merge 3 commits into
Open
fix(asset-export-basic): preload domain cache to fix truncated hierarchy paths (CSA-444)#2561bladata1990 wants to merge 3 commits into
bladata1990 wants to merge 3 commits into
Conversation
…chy paths (CSA-444) MeshExporter.export() lazy-loaded domains in GUID order, so mid-tier domains looked up before their parent were cached with truncated identities, producing 1-level-short hierarchy paths in the export. Preload the domain cache up-front (level-order, parents-before-children), matching asset-import and cube-assets-builder. Adds ExportDomainHierarchyTest covering a 3-level domain hierarchy. merge.yml: temporarily build snapshot images on push to this branch (to be reverted before PR).
…snapshot) Prevents an unrelated package build failure from cancelling the asset-export-basic snapshot image build. Temporary; revert before PR.
Reverts the push-trigger branch entry and fail-fast:false added for snapshot image testing. merge.yml now matches main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
JPMC's CSA Asset Export produced truncated domain hierarchy paths for mid-tier domains — e.g. a Data Product under
Risk Management and Compliance > Wholesale Credit Riskexported asWholesale Credit Risk@<product>instead ofRisk Management and Compliance@Wholesale Credit Risk@<product>. The tenant data was correct; the bug was in the exporter.Linear: CSA-444
Root cause
MeshExporter.export()never calledctx.dataDomainCache.preload(). Without it, domains were lazy-loaded one-at-a-time (getByGuid → lookupById) in GUID order, not parent-before-child order. When a mid-tier domain was looked up before its parent was cached,getIdentityForAssetresolved the parent identity viagetIdentity(parentGuid, bypassReadLock=true)— which is non-recursive and does not lookup-on-miss — so it returned blank and the domain was cached with a 1-level-short identity. Every product/sub-domain referencing it then inherited the truncated path.Only
DataDomainCache.refreshCache()loads domains in level-order (roots first, then children sorted by parent QN), and it runs only insidepreload(). Sibling packages already call it (asset-import/AbstractBaseImporter.kt,cube-assets-builder/Importer.kt);asset-export-basicwas the only mesh-touching package missing it.The fix
One line in
MeshExporter.export(), before the export loop:This loads the domain cache up-front in parents-before-children order.
preload()is@Synchronizedand idempotent; the cost is one bulk paged scan instead of N individualsize=1lookups (net cheaper).Tests
Added
ExportDomainHierarchyTest— builds a 3-level domain hierarchy (grandparent → parent → child) with a data product under the deepest child and asserts the export resolves full level-order identities (not truncated).Verification on a live tenant
Built a snapshot image (
ghcr.io/atlanhq/csa-asset-export-basic:7.2.4-SNAPSHOT), deployed toapps-typedef.atlan.com, created a 3-level fixture (D3 → SD1 → CSA444-LeafwithCSA444-Product), and ran the export.Test workflow run: https://apps-typedef.atlan.com/api/orchestration/workflows/default/csa-asset-export-basic-csa444-mctgz
Result in
products-export.csv:CSA444-Product(DataProduct)dataDomainD3@SD1@CSA444-Leaf✅ full 3-levelCSA444-Leaf(DataDomain)parentDomainD3@SD1✅SD1(DataDomain)parentDomainD3✅Mid-tier domains now resolve to their full root-to-leaf identity. Before the fix,
CSA444-Productwould have shownSD1@CSA444-Leaf/CSA444-Leaf.Follow-up
atlanhq/marketplace-packagesneeds itscsa-asset-export-basicimage bumped to a released version once this lands and a tagged build is published (tracked under CSA-444).