add module publishing and templating system

[flybayer]

Greptile Summary

This PR introduces a full module authoring, compilation, and publishing pipeline — converting the repo from hand-maintained module.yml files to a structured definition.yml format with a YAML composition system (includes, merges, templates) and a CLI toolchain that validates, compiles, tags, and publishes versioned module definitions to the Ravion API.

• Compiler & schema: authoring-schema.ts validates definition files, compiler.ts resolves $include/$merge/$template directives recursively with cycle detection, and module-schema.ts validates the compiled output against the canonical module shape.
• Release pipeline: release.ts diffs local compiled configs against the remote inventory, tags.ts creates annotated git tags per module version, and publish.ts creates/patches remote definitions and versions via HTTP — all wired together in the GitHub Actions workflow.
• Migration guardrail: guardrails.ts blocks re-introduction of the old module.yml format by scanning the repo for YAML files that look like legacy definitions.

Confidence Score: 3/5

Safe to merge for current module definitions (all stable versions), but the pre-release version sorting bug is a latent defect that would silently write wrong definition files if pre-release versions are ever published alongside their stable counterparts.

The compareSemver function strips pre-release suffixes via parseInt and then falls back to lexicographic comparison, which ranks "1.0.0-alpha" above "1.0.0" — the inverse of correct semver precedence. The authoring schema explicitly permits pre-release versions, so this path is reachable without any schema changes. Incorrect version selection in selectLatestVersion would propagate into generated definition.yml files without any warning.

tools/ravion-modules/src/generate-definitions.ts — specifically the compareSemver and selectLatestVersion functions.

Important Files Changed

Filename	Overview
tools/ravion-modules/src/generate-definitions.ts	Generates/normalizes module definitions from remote inventory; contains a semver comparison bug that can select pre-release versions over stable releases.
tools/ravion-modules/src/compiler.ts	YAML composition compiler with include/merge/template directives; cycle detection is correct; $with tokens in partial strings pass through silently without error.
tools/ravion-modules/src/publish.ts	Handles dry-run and live publishing of module definitions/versions to Ravion API; token validation is permissive — missing RAVION_API_TOKEN silently skips auth header.
tools/ravion-modules/src/release.ts	Computes publish state (unknown/unpublished/published/conflict) by comparing local and remote configs via stable-stringified JSON; logic is sound.
tools/ravion-modules/src/tags.ts	Plans and creates annotated git tags per module version; correctly handles annotated vs lightweight tags when parsing existing refs.
.github/workflows/module-definitions.yml	CI/CD workflow for validation and publishing; npm test implicitly builds TypeScript (tsc first), so dist/ is available for subsequent CLI steps.
tools/ravion-modules/src/authoring-schema.ts	Validates definition.yml authoring format; semver pattern correctly accepts pre-release versions, but selectLatestVersion sorting is inconsistent with this.
tools/ravion-modules/src/cli.ts	Top-level CLI dispatcher; straightforward command routing, errors surface naturally via uncaught exception exit.
tools/ravion-modules/src/guardrails.ts	Prevents legacy module.yml files from being re-introduced; heuristic detection via YAML shape looks correct.
tools/ravion-modules/src/module-schema.ts	Validates compiled module config structure; correctly catches duplicate input IDs and unknown root sections.

Sequence Diagram

sequenceDiagram
    participant GH as GitHub Actions
    participant CLI as ravion-modules CLI
    participant FS as Filesystem (definition.yml)
    participant API as Ravion API
    participant Git as git

    GH->>CLI: compile (validate job)
    CLI->>FS: findDefinitionFiles()
    FS-->>CLI: definition.yml paths
    CLI->>FS: parseAuthoringDefinitionFile + resolveValue
    CLI-->>GH: CompiledDefinition[]

    GH->>CLI: tags --api --create (publish job)
    CLI->>API: listModuleDefinitions / listModuleVersions
    API-->>CLI: RemoteModuleInventory
    CLI->>CLI: getReleaseStatuses → validateReleaseStatuses
    CLI->>CLI: planTags (filter unpublished)
    CLI->>Git: "git tag -a type@version"
    Git-->>CLI: ok
    GH->>Git: git push origin --tags

    GH->>CLI: publish --apply (publish job)
    CLI->>API: listModuleDefinitions / listModuleVersions
    API-->>CLI: RemoteModuleInventory
    CLI->>CLI: validateReleaseStatuses (conflict check)
    CLI->>API: createModuleDefinition (if new)
    CLI->>API: patchModuleDefinition (if metadata changed)
    CLI->>API: createModuleVersion (if unpublished)
    API-->>CLI: RemoteModuleVersion

Loading

Prompt To Fix All With AI

Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 3
tools/ravion-modules/src/generate-definitions.ts:184-198
**Pre-release versions sort higher than stable releases**

`compareSemver` falls back to `left.localeCompare(right)` when all three numeric parts are equal, but `parseInt("0-alpha", 10)` returns `0` — stripping the pre-release suffix before comparison. As a result, "1.0.0" and "1.0.0-alpha" produce identical numeric parts, and `"1.0.0".localeCompare("1.0.0-alpha")` is negative (shorter string sorts first), so `selectLatestVersion` would choose "1.0.0-alpha" over "1.0.0". The authoring schema's `SEMVER_PATTERN` explicitly accepts pre-release versions, so this incorrect ordering can produce wrong `definition.yml` files whenever a stable release co-exists with an earlier pre-release.

### Issue 2 of 3
tools/ravion-modules/src/publish.ts:101-102
**Missing auth token not surfaced until first API request**

`createDefaultRavionApiClient` accepts an absent `RAVION_API_TOKEN` silently and builds a client that omits the `Authorization` header. Every subsequent API call will fail with an obscure 401/403 from the server rather than a clear configuration error at startup. Checking at construction time mirrors the existing `baseUrl` guard.

```suggestion
  const token = options.token ?? process.env.RAVION_API_TOKEN;
  if (!token) {
    throw new PublishError("RAVION_API_TOKEN must be set to publish through the Ravion API.");
  }
  return new HttpRavionModuleApiClient(baseUrl, token);
```

### Issue 3 of 3
tools/ravion-modules/src/compiler.ts:199-220
**`$with.` tokens embedded in strings silently pass through unreplaced**

`WITH_TOKEN_PATTERN` is anchored (`^...$`), so it only replaces a value whose entire content is `$with.key`. A template string like `"arn:aws::$with.region:..."` would not match, the token would remain verbatim in the compiled output, and `rejectLeakedCompilerSyntax` would not catch it either (it only checks `LOCAL_TOKEN_PATTERN` and directive keys). Template authors may expect interpolation to work mid-string and get silently wrong data instead of an error.

_{Reviews (1): Last reviewed commit: "extract partials" | Re-trigger Greptile}

Greptile also left 3 inline comments on this PR.

[github-actions]

Ravion Module Publish Plan

Dry run only. No Ravion API mutations were made.

Module	Version	Action	Summary
rvn-aws-acm-certificate	0.0.2	Create Version	Create module version rvn-aws-acm-certificate@0.0.2.
rvn-aws-network	0.0.4	Create Version	Create module version rvn-aws-network@0.0.4.
rvn-aws-static	0.0.2	Create Version	Create module version rvn-aws-static@0.0.2.
rvn-ecs-cluster	0.0.3	Create Version	Create module version rvn-ecs-cluster@0.0.3.
rvn-ecs-web	0.1.1	Skip Version	Skip rvn-ecs-web@0.1.1; identical version already exists.

Diffs

rvn-aws-acm-certificate@0.0.2 Create Version

--- remote
+++ compiled
   - id: section_misc
     label: Misc
     type: section
-  - description: Additional tags for the ACM certificate. Ravion also adds Owner, ProjectGivenId, EnvironmentGivenId, ModuleGivenId, and ModuleId tags.
+  - description: A map of tags to assign to all resources. Default tags are `Owner`, `project_given_id`, `environment_given_id`, `module_given_id`, `module_unique_id`
     id: tags
     label: Tags
     required: false
     type: keyvalue
   - id: section_advanced
-    label: Terraform Settings
+    label: Terraform settings
     type: section
   - collapsible: true
-    description: Override the VPC, subnet, and security group for Pipeline Terraform runners. Must use the same AWS account as selected above.
-    id: execution_environment_id
-    label: Terraform Execution Environment
+    description: Override the environment's default version for this module
+    id: opentofu_version
+    label: OpenTofu version override
+    required: false
     type: string
-    values: $values:ravion/execution_environments
+    values: $values:opentofu/versions
+  - collapsible: true
+    description: Override Terraform state backend workspace name. Defaults to project + environment + module given ids.
+    id: ravion_state_backend_workspace
+    label: Ravion Terraform workspace name
+    type: string
   - default: {}
     description: Optional raw Terraform variable overrides for advanced module inputs or one-off overrides. Values here override the generated variables above.
     id: advanced_terraform_variables
@@
     required: false
     type: object
   - collapsible: true
-    description: Override Terraform state backend workspace name. Defaults to project + environment + module given ids.
-    id: ravion_state_backend_workspace
-    label: Ravion Terraform workspace name
+    description: Override the VPC, subnet, and security group for Pipeline Terraform runners. Must use the same AWS account as selected above.
+    id: execution_environment_id
+    label: Terraform Execution Environment
     type: string
+    values: $values:ravion/execution_environments
 readme: |
   ACM certificate requests an AWS Certificate Manager public certificate with DNS validation.
 
@@
             ProjectGivenId: <<project.given_id>>
           wait_for_validation: << module.input.wait_for_validation >>
         tool: opentofu
-        tool_version: <<defaults.opentofu_version >>
+        tool_version: << module.input.opentofu_version || defaults.opentofu_version >>
       variant: standard
     destroy:
       pipeline_id: << defaults.destroy_pipeline_id >>

rvn-aws-network@0.0.4 Create Version

--- remote
+++ compiled
     required: false
     type: object_map
   - id: section_advanced
-    label: Terraform Settings
+    label: Terraform settings
     type: section
   - collapsible: true
-    description: Override the VPC, subnet, and security group for Pipeline Terraform runners. Must have same AWS account as selected above.
-    id: execution_environment_id
-    label: Terraform Execution Environment
-    type: string
-    values: $values:ravion/execution_environments
-  - collapsible: true
     description: Override the environment's default version for this module
     id: opentofu_version
-    label: OpenTofu Version Override
+    label: OpenTofu version override
     required: false
     type: string
     values: $values:opentofu/versions
+  - collapsible: true
+    description: Override Terraform state backend workspace name. Defaults to project + environment + module given ids.
+    id: ravion_state_backend_workspace
+    label: Ravion Terraform workspace name
+    type: string
+  - default: {}
+    description: Optional raw Terraform variable overrides for advanced module inputs or one-off overrides. Values here override the generated variables above.
+    id: advanced_terraform_variables
+    label: Advanced Terraform variables
+    required: false
+    type: object
+  - collapsible: true
+    description: Override the VPC, subnet, and security group for Pipeline Terraform runners. Must have same AWS account as selected above.
+    id: execution_environment_id
+    label: Terraform Execution Environment
+    type: string
+    values: $values:ravion/execution_environments
 readme: |
   Production-ready AWS VPC with public and private subnets, NAT gateways, and compliance-ready flow logs.
 
@@
         repo: https://github.com/flightcontrolhq/modules
         stack_id: <<stack.id>>
         terraform_variables:
+          ...overrides: << module.input.advanced_terraform_variables >>
           enable_flow_logs: << module.input.enable_flow_logs >>
           enable_nat_gateway: << module.input.enable_nat_gateway >>
           name: << module.input.name >>
@@
       variant: standard
     destroy:
       pipeline_id: << defaults.destroy_pipeline_id >>
-  ravion_state_backend_workspace: << project.given_id + "-" + environment.given_id + "-" + module.given_id + "-" + stack.id  >>
+  ravion_state_backend_workspace: << module.input.ravion_state_backend_workspace || project.given_id + "-" + environment.given_id + "-" + module.given_id + "-" + stack.id>>
   type: opentofu

rvn-aws-static@0.0.2 Create Version

--- remote
+++ compiled
     label: Terraform settings
     type: section
   - collapsible: true
-    description: Override the execution environment for Terraform runners. Must use the same AWS account as selected above.
-    id: execution_environment_id
-    label: Terraform execution environment
-    required: false
-    type: string
-    values: $values:ravion/execution_environments
-  - collapsible: true
-    default: $values:first
+    description: Override the environment's default version for this module
     id: opentofu_version
-    label: OpenTofu version
+    label: OpenTofu version override
+    required: false
     type: string
     values: $values:opentofu/versions
   - collapsible: true
@@
     id: ravion_state_backend_workspace
     label: Ravion Terraform workspace name
     type: string
-  - collapsible: true
-    default: {}
+  - default: {}
+    description: Optional raw Terraform variable overrides for advanced module inputs or one-off overrides. Values here override the generated variables above.
     id: advanced_terraform_variables
     label: Advanced Terraform variables
+    required: false
     type: object
   - collapsible: true
-    description: Additional tags for all resources. Ravion also adds Owner, ProjectGivenId, EnvironmentGivenId, ModuleGivenId, and ModuleId tags.
+    description: Override the execution environment for Terraform runners. Must use the same AWS account as selected above.
+    id: execution_environment_id
+    label: Terraform execution environment
+    required: false
+    type: string
+    values: $values:ravion/execution_environments
+  - collapsible: true
+    description: A map of tags to assign to all resources. Default tags are `Owner`, `project_given_id`, `environment_given_id`, `module_given_id`, `module_unique_id`
     id: tags
     label: Tags
     required: false
@@
             Owner: Ravion
             ProjectGivenId: <<project.given_id>>
         tool: opentofu
-        tool_version: << module.input.opentofu_version >>
+        tool_version: << module.input.opentofu_version || defaults.opentofu_version >>
       variant: standard
     destroy:
       pipeline_id: << defaults.destroy_pipeline_id >>

rvn-ecs-cluster@0.0.3 Create Version

--- remote
+++ compiled
   - id: section_misc
     label: Misc
     type: section
-  - description: Additional tags for all resources. Ravion also adds Owner, ProjectGivenId, EnvironmentGivenId, ModuleGivenId, and ModuleId tags.
+  - description: A map of tags to assign to all resources. Default tags are `Owner`, `project_given_id`, `environment_given_id`, `module_given_id`, `module_unique_id`
     id: tags
     label: Tags
     required: false
     type: keyvalue
-  - default: {}
-    description: Optional raw Terraform variable overrides for advanced module inputs or one-off overrides. Values here override the generated variables above.
-    id: advanced_terraform_variables
-    label: Advanced Terraform variables
-    required: false
-    type: object
   - id: section_advanced
     label: Terraform settings
     type: section
   - collapsible: true
-    description: Override the environment's default OpenTofu version for this module.
+    description: Override the environment's default version for this module
     id: opentofu_version
     label: OpenTofu version override
     required: false
@@
     id: ravion_state_backend_workspace
     label: Ravion Terraform workspace name
     type: string
+  - default: {}
+    description: Optional raw Terraform variable overrides for advanced module inputs or one-off overrides. Values here override the generated variables above.
+    id: advanced_terraform_variables
+    label: Advanced Terraform variables
+    required: false
+    type: object
 readme: |-
   Production-ready AWS ECS cluster with Fargate, Fargate Spot, optional EC2 capacity, and shared load balancers.