Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
- name: Set up .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
dotnet-version: '10.0.x'

- name: Determine runtime identifier
id: rid
Expand Down Expand Up @@ -83,7 +83,7 @@ jobs:
python-version: '3.11'
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
dotnet-version: '10.0.x'
- run: pip install build hatchling twine
- name: Build per-platform engine wheels
shell: bash
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
python-version: '3.11'
- uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
dotnet-version: '10.0.x'
- run: pip install build hatchling
- name: Build per-platform engine wheels
shell: bash
Expand Down
21 changes: 14 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

## Project Overview

Python-Redlines generates `.docx` redline/tracked-changes documents by comparing two Word files. A pure-Python wrapper drives compiled C# (.NET 8) engine binaries; the Python layer handles platform detection, binary extraction, temp file management, and subprocess execution.
Python-Redlines generates `.docx` redline/tracked-changes documents by comparing two Word files. A pure-Python wrapper drives compiled C# (.NET 10) engine binaries; the Python layer handles platform detection, binary extraction, temp file management, and subprocess execution.

Two comparison engines are available:
- **XmlPowerToolsEngine** — wraps Open-XML-PowerTools WmlComparer (original engine)
- **DocxodusEngine** — wraps Docxodus, a modernized .NET 8.0 fork with better move detection
- **DocxodusEngine** — wraps Docxodus, a modernized .NET 10.0 fork with better move detection.
Takes `engine="wmlcomparer"` (default) or `engine="docxdiff"` to pick the comparison algorithm.

## Monorepo structure — three published packages

Expand All @@ -34,7 +35,7 @@ shared pytest/coverage config.
# Initialize the Docxodus submodule (required before building its engine)
git submodule update --init --recursive

# Build engine binaries for one or more platforms (requires .NET 8.0 SDK).
# Build engine binaries for one or more platforms (requires .NET 10.0 SDK).
# RIDs: linux-x64 linux-arm64 win-x64 win-arm64 osx-x64 osx-arm64
python build_differ.py linux-x64
python build_differ.py --all
Expand Down Expand Up @@ -63,9 +64,10 @@ python -m build --wheel packages/docxodus # needs an archive in _binaries/
package is missing, with the `pip install` command to fix it.

Both engines expose `run_redline(author_tag, original, modified, **kwargs)`.
`DocxodusEngine` overrides `_build_command()` to translate kwargs (e.g. `detect_moves`,
`detail_threshold`) into CLI flags. `XmlPowerToolsEngine` uses the legacy
4-positional-arg format and ignores kwargs.
`DocxodusEngine` overrides `_build_command()` to translate kwargs (e.g. `engine`,
`detect_moves`, `detail_threshold`) into CLI flags, and raises `ValueError` when a
WmlComparer-only kwarg is combined with `engine="docxdiff"`. `XmlPowerToolsEngine` uses
the legacy 4-positional-arg format and ignores kwargs.

2. **Binary packages** ship one platform archive under
`src/<pkg>/_binaries/<rid>.tar.gz` (or `.zip` for Windows). The archive is
Expand All @@ -79,6 +81,10 @@ python -m build --wheel packages/docxodus # needs an archive in _binaries/
`build_differ.py` compiles an engine for a given RID with `dotnet publish` and
writes a single flat archive into the corresponding binary package's `_binaries/`.

The two engines target different frameworks — `csproj` is `net8.0`, `docxodus/tools/redline`
is `net10.0` — so `build_differ.py`'s `ENGINES` list carries a per-engine `tfm`. The .NET 10
SDK builds both; `global.json` pins it.

## Build & release flow

- A binary-package wheel must contain **exactly one** platform archive. Each
Expand All @@ -104,4 +110,5 @@ integration test validates exactly 9 revisions on the fixture documents.
## Stdout Format Differences

- **XmlPowerToolsEngine**: `"Revisions found: 9"`
- **DocxodusEngine**: `"Redline complete: 9 revision(s) found"`
- **DocxodusEngine**, default / `engine="wmlcomparer"`: `"Redline complete: 9 revision(s) found"`
- **DocxodusEngine**, `engine="docxdiff"`: `"Redline complete: 11 revision(s) found"`
41 changes: 35 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ on it freely: two documents in, one redline out.
## Quick Start

The **default engine is [Docxodus](https://github.com/JSv4/Docxodus)** — a modernized,
actively-maintained .NET 8 comparison engine (detailed below). Install it and you're
actively-maintained .NET 10 comparison engine (detailed below). Install it and you're
running; the engine binary is prebuilt and embedded in the wheel, so there is **no .NET
SDK to install and nothing to compile**:

Expand Down Expand Up @@ -45,7 +45,7 @@ recommended choice; `XmlPowerToolsEngine` remains available as a legacy option.

### `DocxodusEngine` — Default (Recommended)

**[Docxodus](https://github.com/JSv4/Docxodus)** is a modernized .NET 8.0 fork of Open-XML-PowerTools with
**[Docxodus](https://github.com/JSv4/Docxodus)** is a modernized .NET 10.0 fork of Open-XML-PowerTools with
significant improvements:

- **Move detection** — identifies content that was moved rather than deleted and re-inserted
Expand Down Expand Up @@ -125,6 +125,7 @@ redline_bytes, stdout, stderr = engine.run_redline(

| Setting | Type | Default | Description |
|---|---|---|---|
| `engine` | str | `"wmlcomparer"` | Comparison algorithm: `"wmlcomparer"` or `"docxdiff"` |
| `detail_threshold` | float | 0.0 | Comparison granularity (0.0–1.0, lower = more detailed) |
| `case_insensitive` | bool | False | Ignore case differences |
| `detect_moves` | bool | False | Enable move detection |
Expand All @@ -135,13 +136,35 @@ redline_bytes, stdout, stderr = engine.run_redline(
| `conflate_spaces` | bool | True | Treat breaking/non-breaking spaces the same |
| `date_time` | str | now | Custom ISO 8601 timestamp for revisions |

> **Warning:** Move detection can cause Word to display "unreadable content" warnings due to a known
> **Warning:** (WmlComparer only) Move detection can cause Word to display "unreadable content" warnings due to a known
> ID collision bug. When using `detect_moves=True`, always set `simplify_move_markup=True` as well.
> This converts move markup to regular del/ins (loses green move styling but ensures Word compatibility).

> **Note:** These settings are only available on `DocxodusEngine`. `XmlPowerToolsEngine` ignores
> extra keyword arguments.

### Choosing an engine

`DocxodusEngine` wraps two comparison algorithms in one binary. `wmlcomparer` is the default
and is the lineage inherited from Open-XML-PowerTools. `docxdiff` is Docxodus's newer
structure-aware IR engine, which produces finer-grained markup — on the same pair of documents
it reports 11 revisions where `wmlcomparer` reports 9.

```python
engine.run_redline("Reviewer", original, modified, engine="docxdiff")
```

`docxdiff` does not implement `detail_threshold`, `simplify_move_markup`, or
`detect_format_changes`. Passing any of them alongside `engine="docxdiff"` raises `ValueError`
rather than silently ignoring them. It does honour `detect_moves`, `case_insensitive`,
`conflate_spaces`, `move_similarity_threshold`, `move_minimum_word_count`, and `date_time`.

Move markup differs between the two. `docxdiff` renders moves natively and rejects
`simplify_move_markup`, so the Word-compatibility mitigation described in the warning above is
unavailable there; whether Word's ID-collision warning affects DocxDiff's native move markup is
untested. If you need moves lowered to plain del/ins for maximum Word compatibility, use
`engine="wmlcomparer"` with `simplify_move_markup=True`.

## Architecture Overview

Both engines follow the same pattern: a Python wrapper class invokes a self-contained C# binary via subprocess.
Expand Down Expand Up @@ -184,19 +207,25 @@ extracts the platform archive once into the user cache directory, and runs it.

### Stdout Differences

The two engines produce slightly different stdout messages:
The engines produce slightly different stdout messages:

| Engine | Example stdout |
|---|---|
| `XmlPowerToolsEngine` | `Revisions found: 9` |
| `DocxodusEngine` | `Redline complete: 9 revision(s) found` |
| `DocxodusEngine` (default / `engine="wmlcomparer"`) | `Redline complete: 9 revision(s) found` |
| `DocxodusEngine` (`engine="docxdiff"`) | `Redline complete: 11 revision(s) found` |

The revision counts differ between the two Docxodus engines because the algorithms differ,
not because either is wrong.

## Development

### Prerequisites

- Python 3.9+
- .NET 8.0 SDK (only for building the engine binaries locally)
- .NET 10.0 SDK (only for building the engine binaries locally). Install with
`apt install dotnet-sdk-10.0`, or `curl -sSL https://dot.net/v1/dotnet-install.sh | bash -s -- --channel 10.0`.
A .NET 8 SDK can no longer build the Docxodus engine, which targets `net10.0`.

### Setup

Expand Down
4 changes: 3 additions & 1 deletion build_differ.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
"name": "ooxmlpowertools",
"csproj": os.path.join("csproj"),
"csproj_file": os.path.join("csproj", "redlines.csproj"),
"tfm": "net8.0",
"binaries_dir": os.path.join(
"packages", "ooxmlpowertools", "src",
"python_redlines_ooxmlpowertools", "_binaries",
Expand All @@ -34,6 +35,7 @@
"name": "docxodus",
"csproj": os.path.join("docxodus", "tools", "redline"),
"csproj_file": os.path.join("docxodus", "tools", "redline", "redline.csproj"),
"tfm": "net10.0",
"binaries_dir": os.path.join(
"packages", "docxodus", "src",
"python_redlines_docxodus", "_binaries",
Expand Down Expand Up @@ -88,7 +90,7 @@ def build_engine_for_rid(engine, rid):
print(f"[{engine['name']}] Building {rid} ...")
run_command(f"dotnet publish {csproj} -c Release -r {rid} --self-contained")

publish_dir = os.path.join(csproj, "bin", "Release", "net8.0", rid, "publish")
publish_dir = os.path.join(csproj, "bin", "Release", engine["tfm"], rid, "publish")
if not os.path.isdir(publish_dir):
raise RuntimeError(f"Expected publish output not found: {publish_dir}")

Expand Down
4 changes: 3 additions & 1 deletion docs/developer-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
## Prerequisites

- Python 3.9 or higher
- .NET 8.0 SDK (to compile the C# engine binaries)
- .NET 10.0 SDK (to compile the C# engine binaries). The Docxodus engine targets
`net10.0`; the Open-XML-PowerTools engine still targets `net8.0` and the .NET 10 SDK
builds both.
- `git` (the Docxodus engine is a submodule)

## Repository layout
Expand Down
2 changes: 1 addition & 1 deletion docxodus
Submodule docxodus updated 334 files
2 changes: 1 addition & 1 deletion global.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"sdk": {
"version": "8.0.0",
"version": "10.0.0",
"rollForward": "latestFeature"
}
}
3 changes: 3 additions & 0 deletions packages/core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ redline_bytes, stdout, stderr = engine.run_redline(
)
```

`DocxodusEngine` accepts `engine="wmlcomparer"` (default) or `engine="docxdiff"` to select the
comparison algorithm. See the [project README](https://github.com/JSv4/Python-Redlines#choosing-an-engine).

If an engine's companion package is not installed, instantiating the engine
raises `EngineNotInstalledError` with the `pip install` command to fix it.

Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/python_redlines/__about__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: 2024-present U.N. Owen <void@some.where>
#
# SPDX-License-Identifier: MIT
__version__ = "0.2.1"
__version__ = "0.3.0"
47 changes: 44 additions & 3 deletions packages/core/src/python_redlines/engines.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,14 @@ def run_redline(self, author_tag: str, original: Union[str, bytes, Path], modifi
(as ``str`` or ``pathlib.Path``). Returns the redline output as bytes.

Additional keyword arguments are passed to _build_command() for engine-specific options.
DocxodusEngine supports: detail_threshold, case_insensitive, detect_moves,
DocxodusEngine supports: engine, detail_threshold, case_insensitive, detect_moves,
simplify_move_markup, move_similarity_threshold, move_minimum_word_count,
detect_format_changes, conflate_spaces, date_time.

DocxodusEngine's engine kwarg selects the comparison algorithm: 'wmlcomparer'
(the default) or 'docxdiff'. The docxdiff engine ignores detail_threshold,
simplify_move_markup, and detect_format_changes, so passing them alongside
engine='docxdiff' raises ValueError rather than silently changing nothing.
"""
temp_files = []
try:
Expand Down Expand Up @@ -194,6 +199,13 @@ class DocxodusEngine(BaseEngine):
BINARY_BASE_NAME = 'redline'
EXTRA_NAME = 'docxodus'

# Comparison engines accepted by the redline CLI's --engine flag.
ENGINES = ('wmlcomparer', 'docxdiff')

# DocxCompare.ToDocxDiffSettings drops these on the docxdiff branch, and the CLI
# accepts them there without complaint, so reject them before we shell out.
_WMLCOMPARER_ONLY = ('detail_threshold', 'simplify_move_markup', 'detect_format_changes')

# Boolean flags (default False — presence enables)
_BOOL_FLAGS = [
('case_insensitive', '--case-insensitive'),
Expand All @@ -215,8 +227,33 @@ class DocxodusEngine(BaseEngine):
('date_time', '--date-time'),
]

@staticmethod
def _validate_kwargs(kwargs):
@classmethod
def _normalize_engine(cls, kwargs):
"""The chosen engine, lowercased and stripped, or None if the caller didn't pick one."""
if 'engine' not in kwargs:
return None

engine = kwargs['engine']
if not isinstance(engine, str):
raise ValueError(f"engine must be a string, got {engine!r}")

normalized = engine.strip().lower()
if normalized not in cls.ENGINES:
raise ValueError(
f"engine must be one of {', '.join(cls.ENGINES)}, got {engine!r}"
)
return normalized

@classmethod
def _validate_kwargs(cls, kwargs):
if cls._normalize_engine(kwargs) == 'docxdiff':
for name in cls._WMLCOMPARER_ONLY:
if name in kwargs:
raise ValueError(
f"{name} is not supported by the 'docxdiff' engine "
f"(WmlComparer-only). Remove it or use engine='wmlcomparer'."
)

if 'detail_threshold' in kwargs:
val = kwargs['detail_threshold']
if not isinstance(val, (int, float)) or val < 0.0 or val > 1.0:
Expand All @@ -234,10 +271,14 @@ def _validate_kwargs(kwargs):

def _build_command(self, author_tag, original_path, modified_path, target_path, **kwargs):
self._validate_kwargs(kwargs)
engine = self._normalize_engine(kwargs)

cmd = [self.extracted_binaries_path, original_path, modified_path, target_path,
f'--author={author_tag}']

if engine is not None:
cmd.append(f'--engine={engine}')

for kwarg, flag in self._BOOL_FLAGS:
if kwargs.get(kwarg):
cmd.append(flag)
Expand Down
Loading
Loading