otel logging connected to MCP servers

.gitignore

@@ -7,4 +7,7 @@ __pycache__/
 dist/
 .env
 .DS_Store
-**/.DS_Store
+**/.DS_Store
+.coverage
+.coverage.*
+htmlcov/

Dockerfile

@@ -1,6 +1,6 @@
 # Node Wire — Docker Image
 # ========================
-# This image packages the connector platform as a FastMCP server.
+# This image packages the connector platform as an MCP stdio server (manifest-driven).
 # ToolHive runs it as a container, injects secrets as env vars,
 # and proxies the stdio MCP transport to HTTP/SSE.
 #
@@ -11,7 +11,8 @@
 #   thv run --name node-wire-connectors --transport stdio \
 #     --secret ... node-wire:latest
 
-FROM python:3.12-slim
+# Digest-pinned base (update when bumping tag). See .github/workflows/docker-policy.yml.
+FROM python:3.12-slim@sha256:3d5ed973e45820f5ba5e46bd065bd88b3a504ff0724d85980dcd05eab361fcf4
 
 # Install system deps needed by some connector libs
 RUN apt-get update && apt-get install -y --no-install-recommends \
@@ -21,19 +22,42 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 WORKDIR /app
 
 # Copy source (build context = repo root)
-COPY pyproject.toml ./
 COPY src/ ./src/
 COPY config/ ./config/
-
-# Install platform + agents extras
-RUN pip install --no-cache-dir -e ".[agents]"
+COPY packages/runtime/dist/*.whl /wheels/
+COPY packages/connectors/http_generic/dist/*.whl /wheels/
+COPY packages/connectors/stripe/dist/*.whl /wheels/
+COPY packages/connectors/smtp/dist/*.whl /wheels/
+COPY packages/connectors/google_drive/dist/*.whl /wheels/
+COPY packages/connectors/fhir_cerner/dist/*.whl /wheels/
+COPY packages/connectors/fhir_epic/dist/*.whl /wheels/
+
+ENV PYTHONPATH=/app/src
+
+# Install runtime + connector packages using local wheel artifacts
+RUN pip install --no-cache-dir --find-links=/wheels \
+    node-wire-runtime \
+    node-wire-http-generic \
+    node-wire-stripe \
+    node-wire-smtp \
+    node-wire-google-drive \
+    node-wire-fhir-cerner \
+    node-wire-fhir-epic \
+    "mcp>=1.6.0" \
+    && rm -rf /wheels
+
+RUN groupadd --system --gid 1000 app \
+    && useradd --system --uid 1000 --gid app --home /app app \
+    && chown -R app:app /app
+
+USER app
 
 # Expose nothing — ToolHive manages the stdio proxy port internally
 # MCP_PORT / FASTMCP_PORT will be set by ToolHive if ever needed
 
 # Healthcheck: verify the package is importable
 HEALTHCHECK --interval=30s --timeout=5s --start-period=10s CMD \
-    python -c "from agents.mcp_entrypoint import _make_server; print('ok')" || exit 1
+    python -c "from agents.mcp_entrypoint import main; assert callable(main); print('ok')" || exit 1
 
-# Default entrypoint: run the FastMCP server on stdio
+# Default entrypoint: run the MCP server on stdio
 CMD ["python", "-m", "agents.mcp_entrypoint"]

README.md

@@ -10,15 +10,17 @@ For dependency management use any tool that understands `pyproject.toml` (e.g. `
 
 Each connector can run as its own independent MCP server (Docker image).
 
-| Image                   | Tool exposed               | Docker image                     |
-| ----------------------- | -------------------------- | -------------------------------- |
-| `nw-google-drive`       | `google_drive_upload_file` | `docker/google-drive/Dockerfile` |
-| `nw-smartonfhir-epic`   | `fhir_epic_read_patient`   | `docker/fhir-epic/Dockerfile`    |
-| `nw-smartonfhir-cerner` | `fhir_cerner_read_patient` | `docker/fhir-cerner/Dockerfile`  |
-| `nw-smtp`               | `smtp_send_email`          | `docker/smtp/Dockerfile`         |
+| Image                   | MCP tools (manifest) | Docker image                     |
+| ----------------------- | -------------------- | -------------------------------- |
+| `nw-google-drive`       | All `google_drive.<action>` (e.g. `google_drive.files.upload`) | `docker/google-drive/Dockerfile` |
+| `nw-smartonfhir-epic`   | All `fhir_epic.<action>` (e.g. `fhir_epic.read_patient`) | `docker/fhir-epic/Dockerfile`    |
+| `nw-smartonfhir-cerner` | All `fhir_cerner.<action>` (e.g. `fhir_cerner.read_patient`) | `docker/fhir-cerner/Dockerfile`  |
+| `nw-smtp`               | `smtp.send_email`    | `docker/smtp/Dockerfile`         |
 
 See [docs/mcp-servers.md](docs/mcp-servers.md) for build, env config, docker-compose, and ToolHive registration.
 
+**Packaging & Publishing (PyPI wheels, CI publish flow, secrets config):** [docs/packaging.md](docs/packaging.md).
+
 ---
 
 ## High-level architecture
@@ -37,7 +39,7 @@ The platform is split into three layers:
 
 **Purpose:** Provide shared execution and reliability so every connector behaves in a consistent way (validation, errors, retries, telemetry) without each connector reimplementing the same plumbing.
 
-**Location:** `src/runtime/` (base.py, models.py, errors.py, resilience.py, secrets.py, policy.py).
+**Location:** `src/node_wire_runtime/` (base_connector.py, models.py, errors.py, resilience.py, policy.py, observability.py, connector_registry.py, manifest.py).
 
 ### Main pieces
 
@@ -74,7 +76,7 @@ The platform is split into three layers:
 
 **Purpose:** System adapters that talk to external services. Each connector defines input/output models and implements `internal_execute` (and optionally registers its own exceptions with the ErrorMapper).
 
-**Location:** `src/connectors/`. Each connector lives in its own subpackage (e.g. `google_drive/`, `smtp/`, `stripe/`, `http_generic/`).
+**Location:** `src/node_wire_<name>/` (e.g. `src/node_wire_google_drive/`, `src/node_wire_smtp/`, `src/node_wire_stripe/`, `src/node_wire_http_generic/`).
 
 ### Common structure per connector
 
@@ -96,9 +98,11 @@ The platform is split into three layers:
 
 ### Connector-specific documentation
 
+**Connectors guide (`BaseConnector`, factory, manifest):** [docs/connectors.md](docs/connectors.md).
+
 **Details for each connector**—operations, request/response bodies, examples, and error handling—**are documented in that connector’s folder.**
 
-Examples: Google Drive has a full doc at `src/connectors/google_drive/README.md`; FHIR connectors are documented at `src/connectors/fhir_epic/README.md` and `src/connectors/fhir_cerner/README.md`. Other connectors may have a similar `.md` in their folder or document behavior in code and docstrings; always check the connector’s folder for up-to-date details.
+Examples: Google Drive has a full doc at `src/node_wire_google_drive/README.md`; FHIR connectors are documented at `src/node_wire_fhir_epic/README.md` and `src/node_wire_fhir_cerner/README.md`. Other connectors may have a similar `.md` in their folder or document behavior in code and docstrings; always check the connector’s folder for up-to-date details.
 
 ---
 
@@ -123,14 +127,14 @@ Examples: Google Drive has a full doc at `src/connectors/google_drive/README.md`
 ### gRPC / MCP
 
 - **gRPC:** Started when `MODE=GRPC`; server listens on port 50051.
-- **MCP:** Started when `MODE=MCP`; server exposes tools for discovery and invocation.
+- **MCP:** `MODE=MCP` starts a minimal MCP-style placeholder server (sufficient for local, manual inspection), but it is not the full stdio MCP server used for ToolHive and the agent layer.
 
 ### Entrypoint
 
 - Run with `python -m bindings_entrypoint` (or the `node-wire` script after install). The **MODE** environment variable selects:
   - **API** (default) – REST API on port 8000.
   - **GRPC** – gRPC server on port 50051.
-  - **MCP** – MCP server.
+  - **MCP** – minimal MCP-style placeholder server (see note above).
 
 ---
 
@@ -193,3 +197,11 @@ $env:GOOGLE_DRIVE_SA_JSON = Get-Content -Path $saPath -Raw
 ## Dependencies
 
 All dependencies are declared in `pyproject.toml` (Python >=3.11). They include: pydantic, FastAPI, uvicorn, tenacity, pybreaker, OpenTelemetry, grpcio, and connector-specific libraries (httpx, aiosmtplib, stripe, google-auth, google-api-python-client, etc.). See `pyproject.toml` for the full list and versions.
+
+---
+
+## Setup and development docs
+
+- Platform setup (REST/gRPC/agents MCP): [Setup.md](Setup.md)
+- Individual connector MCP servers (ToolHive): [docs/mcp-servers.md](docs/mcp-servers.md)
+- Creating a new connector: [docs/creating-a-connector.md](docs/creating-a-connector.md)

Setup.md

@@ -23,10 +23,11 @@ Node Wire is a Python framework that runs connector adapters (Google Drive, SMTP
 
 | Requirement | Version | Notes                                   |
 | ----------- | ------- | --------------------------------------- |
-| Python      | 3.12+   | `python --version` to check             |
+| Python      | 3.11+   | `python --version` to check             |
 | pip or uv   | Latest  | `pip install --upgrade pip`             |
 | Git         | Any     | To clone the repo                       |
 | Docker      | Latest  | Only needed for ToolHive MCP deployment |
+| Node.js     | Any LTS | Only needed for `npx @modelcontextprotocol/inspector` |
 
 
 ---
@@ -36,7 +37,7 @@ Node Wire is a Python framework that runs connector adapters (Google Drive, SMTP
 ```bash
 # 1. Clone the repository
 git clone <repo-url>
-cd connector-platform
+cd node-wire
 
 # 2. Install dependencies (recommended: uv)
 uv sync --extra agents
@@ -45,13 +46,17 @@ uv sync --extra agents
 uv run node-wire --help
 ```
 
+> **Install uv:** See the official installer docs at `https://docs.astral.sh/uv/`.
+>
 > **REST/gRPC only** (no AI agent features): `uv sync` without the extra is sufficient.
 >
 > **Alternative (pip):** If you’re not using `uv`, install editable deps with pip:
 >
 > - `pip install -e ".[agents]"` (includes MCP/LLM agent dependencies)
 > - `pip install -e .` (REST/gRPC only, no agent dependencies)
 
+> **Installing from PyPI wheels instead of source?** See [docs/packaging.md](docs/packaging.md) for the wheel build lifecycle, client install model, and pre-publish validation checklist.
+
 ---
 
 ## Configuration
@@ -67,14 +72,16 @@ cp sample.env .env
 
 You only need to fill in the sections for the connectors you plan to use. The platform starts successfully even if some credentials are missing — those connectors will simply return an error when called.
 
+> **Doc convention:** Environment variable names in the docs follow `sample.env`. Some legacy keys (like `stripe_api_key`) are intentionally lower-case because that is what the connector reads.
+
 ### Environment Variable Sections
 
 
 | Section          | Key Variables                                                                                                       | When Needed            |
 | ---------------- | ------------------------------------------------------------------------------------------------------------------- | ---------------------- |
 | **FHIR Epic**    | `EPIC_FHIR_BASE_URL`, `EPIC_TOKEN_URL`, `EPIC_CLIENT_ID`, `EPIC_KID`, `EPIC_PRIVATE_KEY`                            | Epic EHR integration   |
 | **FHIR Cerner**  | `CERNER_FHIR_BASE_URL`, `CERNER_TOKEN_URL`, `CERNER_CLIENT_ID`, `CERNER_KID`, `CERNER_PRIVATE_KEY`, `CERNER_SCOPES` | Cerner EHR integration |
-| **Google Drive** | `google_drive_sa_json`, `GOOGLE_DRIVE_FOLDER_ID`                                                                    | Google Drive connector |
+| **Google Drive** | `GOOGLE_DRIVE_SA_JSON`, `GOOGLE_DRIVE_FOLDER_ID`                                                                    | Google Drive connector |
 | **SMTP**         | `SMTP_HOST`, `SMTP_PORT`, `SMTP_USERNAME`, `SMTP_PASSWORD`                                                          | Sending emails         |
 | **LLM / Agent**  | `LLM_PROVIDER`, `GROQ_API_KEY` (or other provider key)                                                              | AI agent / ToolHive    |
 | **ToolHive**     | `TOOLHIVE_MCP_URL` (single) or `TOOLHIVE_MCP_URLS` (comma-separated, multi-server)                                  | ToolHive MCP proxy     |
@@ -95,27 +102,50 @@ The platform supports three modes. Set the `MODE` environment variable to switch
 | **gRPC**               | `MODE=GRPC uv run node-wire`      | `50051`      | gRPC clients                        |
 | **MCP (stdio)**        | `python -m agents.mcp_entrypoint` | stdio        | AI agents, ToolHive, Claude Desktop |
 
+> **Important:** `MODE=MCP` for `node-wire` / `python -m bindings_entrypoint` starts a minimal MCP-style placeholder server, not the full stdio MCP server used with ToolHive and the agent layer. For ToolHive/Inspector/agents, use `python -m agents.mcp_entrypoint` (or the per-connector MCP servers in `docs/mcp-servers.md`).
+
+### Configuration file (`config/connectors.yaml`)
+
+Connectors are loaded from `config/connectors.yaml`. Each connector has:
+
+- `enabled`: whether the connector is instantiated at startup
+- `exposed_via`: which protocols can access it (`rest`, `grpc`, `mcp`)
+
+If a connector is disabled (or not exposed for a protocol), requests to it will fail with “not configured / not available” even if your `.env` is correct.
+
+For details on adding a new connector to the runtime, see `docs/creating-a-connector.md`.
+
 
 ### REST API Quick Start
 
 ```bash
+# Local development: disable REST auth (do not use in production)
+export NW_REST_AUTH_DISABLED=true
+
 # Default port 8000
 uv run node-wire
 
 # If port 8000 is in use, override with PORT
 PORT=8001 uv run node-wire
 ```
 
+Equivalent entrypoint (without `uv`):
+
+```bash
+MODE=API python -m bindings_entrypoint
+```
+
 Once running:
 
-- **Health check:** `GET http://localhost:8000/health`
-- **Interactive docs (Swagger UI):** `http://localhost:8000/docs`
+- **Health check (no auth):** `GET http://localhost:8000/health`
+- **Interactive docs (Swagger UI):** `http://localhost:8000/docs` (requires API key when auth is enabled)
 - **Call a connector:** `POST http://localhost:8000/connectors/{connector_id}/{action}`
 
-Example — send an HTTP request via the generic connector:
+Example — send an HTTP request via the generic connector (with auth enabled):
 
 ```bash
 curl -X POST http://localhost:8000/connectors/http_generic/request \
+  -H "Authorization: Bearer $NW_REST_API_KEY" \
   -H "Content-Type: application/json" \
   -d '{"url": "https://httpbin.org/get", "method": "GET"}'
 ```
@@ -137,6 +167,9 @@ All responses use the same standard shape:
 
 ## Connectors Overview
 
+**Developer guide (`BaseConnector`, config, factory):** [docs/connectors.md](docs/connectors.md).
+
+
 
 | Connector        | What It Does                               | Credentials Needed                     | Setup Guide                                                                                   |
 | ---------------- | ------------------------------------------ | -------------------------------------- | --------------------------------------------------------------------------------------------- |
@@ -194,7 +227,7 @@ Supported configurations:
 Add to your `.env`:
 
 ```env
-stripe_api_key=sk_test_your_key_here
+STRIPE_API_KEY=sk_test_your_key_here
 ```
 
 Use a **test key** (`sk_test_...`) during development. Switch to a live key (`sk_live_...`) for production.
@@ -216,7 +249,7 @@ Quick summary of what you'll need:
 Add to your `.env`:
 
 ```env
-google_drive_sa_json=/absolute/path/to/service-account.json
+GOOGLE_DRIVE_SA_JSON=/absolute/path/to/service-account.json
 GOOGLE_DRIVE_FOLDER_ID=your-folder-id-from-drive-url
 ```
 
@@ -272,16 +305,25 @@ The platform exposes connector tools for AI agents via the MCP (Model Context Pr
 Each connector runs as its own independent MCP server. This is the preferred approach for modular, scalable deployments.
 
 
-| Image                   | Tool exposed               | Docker image                     |
-| ----------------------- | -------------------------- | -------------------------------- |
-| `nw-google-drive`       | `google_drive_upload_file` | `docker/google-drive/Dockerfile` |
-| `nw-smartonfhir-epic`   | `fhir_epic_read_patient`   | `docker/fhir-epic/Dockerfile`    |
-| `nw-smartonfhir-cerner` | `fhir_cerner_read_patient` | `docker/fhir-cerner/Dockerfile`  |
-| `nw-smtp`               | `smtp_send_email`          | `docker/smtp/Dockerfile`         |
+| Image                   | MCP tools (manifest) | Docker image                     |
+| ----------------------- | -------------------- | -------------------------------- |
+| `nw-google-drive`       | All `google_drive.<action>` (e.g. `google_drive.files.upload`) | `docker/google-drive/Dockerfile` |
+| `nw-smartonfhir-epic`   | All `fhir_epic.<action>` (e.g. `fhir_epic.read_patient`) | `docker/fhir-epic/Dockerfile`    |
+| `nw-smartonfhir-cerner` | All `fhir_cerner.<action>` (e.g. `fhir_cerner.read_patient`) | `docker/fhir-cerner/Dockerfile`  |
+| `nw-smtp`               | `smtp.send_email`    | `docker/smtp/Dockerfile`         |
 
 
 **Full guide (build, env config, ToolHive registration, multi-server agent usage):** [docs/mcp-servers.md](docs/mcp-servers.md)
 
+**FHIR tool arguments (Cerner / Epic)** — tool names are `fhir_cerner.<action>` and `fhir_epic.<action>`. Use field names from `tools/list` / the connector manifest. Typical payloads:
+
+| Action | When to use | Example arguments |
+| ------ | ----------- | ------------------- |
+| `read_patient` | You have a Patient id | `{"resource_id": "12724066"}` (Epic ids often start with `e`) |
+| `search_patients` | No id, or name-based search | `{"resource_ids": ["id1"]}` or `{"given_name": "...", "family_name": "..."}` or `{"search_params": {"identifier": "...", "family": "..."}}` (FHIR search param names) |
+
+The MCP server normalizes common LLM/legacy aliases (`patientId` / `patient_id` → `resource_id`; `patientId` inside `search_params` → `identifier`) before validation. Prefer canonical fields above when authoring prompts or clients.
+
 Quick start:
 
 ```bash
@@ -315,6 +357,12 @@ npx @modelcontextprotocol/inspector python -m agents.google_drive_mcp
 npx @modelcontextprotocol/inspector python -m agents.mcp_entrypoint
 ```
 
+### Troubleshooting quick hits
+
+- **Port 8000 in use**: set `PORT=8001` (or any free port) when starting the REST API.
+- **Connector “not configured”**: confirm it is `enabled: true` (and exposed for your protocol) in `config/connectors.yaml`.
+- **ToolHive + Google Drive auth failure**: inside ToolHive, `GOOGLE_DRIVE_SA_JSON` must be the JSON **contents** (not a file path). Locally, it can be an absolute file path (see `docs/mcp-servers.md`).
+
 ---
 
 ## Running Tests

config/connectors.yaml

@@ -1,3 +1,17 @@
+# Connector enable/list — see docs/connectors.md
+#
+# REST API auth (not stored here; set in environment):
+#   NW_REST_API_KEY — required for /connectors, /playground, /scenarios unless NW_REST_AUTH_DISABLED=true
+#   NW_REST_LOAD_DOTENV=false — recommended in production (no .env file)
+#
+# Plugin allowlist (optional):
+#   NW_ALLOWED_CONNECTORS=fhir_epic,http_generic,...
+#   NW_CONNECTOR_MODULE_PREFIX=node_wire_
+#
+# Secrets backend (optional):
+#   NW_SECRET_BACKEND=env | aws_env
+#   NW_AWS_SECRETS_MANAGER_SECRET_ID=...  (when aws_env)
+#
 connectors:
   http_generic:
     enabled: true