Skip to content

docs: define MinerU practices roadmap - #146

Merged
konard merged 7 commits into
mainfrom
issue-143-614fca5161bb
Jul 27, 2026
Merged

docs: define MinerU practices roadmap#146
konard merged 7 commits into
mainfrom
issue-143-614fca5161bb

Conversation

@konard

@konard konard commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • audits MinerU 3.4.4 at pinned commit 79d6d8d against the current Rust and JavaScript web-capture implementations
  • turns the issue's broad "all best practices" request into 20 finite, testable practices with current-state mappings and acceptance gates
  • proposes a clean-room, Unlicense-compatible CaptureDocument v1 architecture and a seven-phase delivery roadmap
  • preserves the issue, PR, release, repository, CI, and related-work metadata used by the research
  • keeps Cargo.lock synchronized with Rust package version 0.3.33 after merging the latest main

Root cause and scope

MinerU parses local documents with OCR/layout/model pipelines, while web-capture acquires rendered URLs. The foundational gap is therefore not a missing parser dependency; it is the absence of a shared typed document model between acquisition and serialization.

This PR is intentionally contract-first, matching the issue's documentation and enhancement labels. It does not copy MinerU code or artifacts, add an ML runtime, or expose an untested parser API. Follow-up implementation starts with shared Rust/JavaScript schema fixtures and golden tests before adding optional document or OCR backends.

Evidence

  • Reproduction: issue Use all best practices from https://github.com/opendatalab/mineru but in Rust/JavaScript and using Unlicense (Public Domain) #143 has a title but no body, comments, screenshots, or acceptance test.
  • Automated gates defined: each BP-01 through BP-20 entry includes a verification gate; the acceptance matrix makes the research scope finite.
  • Provenance: online sources are pinned and compact API snapshots are archived under docs/case-studies/issue-143/data/.
  • Licensing: the plan treats MinerU's custom license, models, datasets, and fixtures as separate review boundaries and keeps original project code Unlicense-compatible.
  • CI diagnosis: Rust workflow run 30238377295 failed at rust/src/main.rs:211; the failure was reproduced locally with Rust 1.97.1. The equivalent compatibility fix landed on main in fix(js): remove deprecated glob dependency #145 and is included here by merge.

Validation

  • Prettier 3.7.4 check for the Markdown documents
  • local Markdown relative-link check
  • parse all archived JSON with Node.js
  • node scripts/check-js-rust-parity.mjs
  • npm run lint (0 errors; 30 pre-existing warnings)
  • npm run format:check
  • npm run check:duplication
  • npm test -- --runInBand --testPathIgnorePatterns=tests/e2e/docker.test.js (55 suites and 493 tests passed; 4 suites/42 Docker tests skipped)
  • Node 22 targeted API integration suite after merging main (18 passed)
  • cargo +1.97.1 fmt --all -- --check
  • cargo +1.97.1 clippy --all-targets --all-features -- -D warnings
  • cargo +1.97.1 test --all-features (267 executable tests and 2 doc tests passed)

Fixes #143

Adding .gitkeep for PR creation (default mode).
This file will be removed when the task is complete.

Issue: #143
@konard konard self-assigned this Jul 27, 2026
@konard konard changed the title [WIP] Use all best practices from https://github.com/opendatalab/mineru but in Rust/JavaScript and using Unlicense (Public Domain) docs: define MinerU practices roadmap Jul 27, 2026
@konard
konard marked this pull request as ready for review July 27, 2026 05:22
@konard

konard commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator Author

Working session summary

Implemented and finalized PR #146.

  • Added a pinned MinerU 3.4.4 audit with 20 testable practices, licensing boundaries, CaptureDocument v1, and a seven-phase Rust/JavaScript roadmap.
  • Preserved research sources and API snapshots.
  • Merged the latest main and synchronized Rust 0.3.33 lock metadata.
  • Updated the PR title/description and marked it ready for review.
  • Local validation: 493 JavaScript tests and 269 Rust tests passed.
  • Final SHA fa24ce4: JavaScript, Rust, and parity CI all pass.
  • PR is cleanly mergeable; working tree is clean.

Main document: README.md


This summary was automatically extracted from the AI working session output.

@konard

konard commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Model: GPT-5.6 Sol
  • Provider: OpenAI
  • Public pricing estimate: $28.419403

📊 Context and tokens usage:

  • 499.4K / 200K (250%) input tokens, 56.6K / 128K (44%) output tokens

Total: (499.4K + 20.9M cached) input tokens, 56.6K output tokens, $28.419403 cost

🤖 Models used:

  • Tool: OpenAI Codex
  • Requested: gpt-5.6-sol
  • Thinking level: medium (~15999 tokens)
  • Model: GPT 5.6 Sol (gpt-5.6-sol)

📎 Log file uploaded as Gist (7868KB)


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
konard merged commit f176d76 into main Jul 27, 2026
18 checks passed
@konard

konard commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator Author

🎉 Auto-merged

This pull request has been automatically merged by hive-mind.

  • All CI checks have passed

Auto-merged by hive-mind with --auto-merge flag

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use all best practices from https://github.com/opendatalab/mineru but in Rust/JavaScript and using Unlicense (Public Domain)

1 participant