Skip to content

M2-32: Recording Hermes Agent before the build

Status: ready | Depends on: none | Milestone: M2

Goal

Before the Hermes adapter is built (M2-33), this ticket measures the six open points of 09 17.10 "To check before building" against a pinned public version of Hermes Agent with a fake model, and keeps the evidence as fixtures in plugins/hermes-adapter/tests. The six points are: how the pinned version calls format_tool_event, send_draft, set_status_text and edit_message; whether steer is available to a platform adapter; whether a public surface tells a plugin which credential type (API key or OAuth) the next turn uses, and whether OAuth wins over an API key; whether on_processing_complete reports an interrupted turn as cancelled; how a new session and a bind to an existing session are triggered through public ways, and how a plugin learns the Hermes session ID; and how the Hermes gateway recognises a plugin's own access policy (enforces_own_access_policy with policy "allowlist"). After this ticket each point has recorded observations, a verdict computed from them and a row in 09 17.10 with answer, fixture and consequence; where a claim of chapter 09 did not hold, the mapping in 09 section 17 (and in 10 10.1 where it describes Hermes) is adapted in the same change. The recording is not a one-off: a test in the component's default test run provisions the pinned commit read-only in a temporary directory, installs it into a throwaway environment, starts its gateway with a probe platform plugin that Hermes finds through its real plugin discovery and a local fake model, replays the scenarios, compares the normalized call patterns with the checked-in fixtures and proves the checkout and the installed files unchanged afterwards. Nothing in a run can reach a real model, a real credential, another Hermes installation or a productive runtime (PLAN 1 point 2). Why: progress hooks, steer and credential detection of Hermes are no promised plugin API (RK-12), and the proof of the actual billing mode may depend on a surface that is not public (RK-13, T-AI-08); an adapter built on guesses could carry organization traffic over a personal subscription or show a broken live box. The harness built here (pin, provisioning, isolation, fake model, control channel, integrity check) is the base on which M2-33 builds its Fake-Hermes scenarios.

Read first

  • docs/PLAN.md: section 1 "Principles" (points 2 to 4); section 3 "Way of working per ticket"; section 5 "Test strategy" (row "Test doubles": Fake-Hermes with M2-33); section 8 "M2 Agent platform, extensions and terminal": "Scope" (reference adapter for Hermes Agent), "Exit criteria" item 5 (the pinned Hermes version is unchanged before and after the test), "Test strategy" ("fake Hermes with a pinned version and a fake model (no costs)"), "Red team" ("faking the reported mode"), "Dependencies" (M2-32 can run at any time), "Risks" (first sentence); the rows M2-32 (this ticket), M2-33 (builds the adapter on this evidence), M2-29 (the SDK the adapter uses), M2-23 (overlap in the root Makefile, Design note 2), M2-34 and M2-35 (the same pattern for Fake-Claude, not here); section 19 P-09, P-10, P-11, P-13.
  • docs/spec/09-agents.md: 2.2 "Features" (followups: steer or queue), 4.8 "Heartbeat and reported state" (billing_evidence.key_type, degraded_reasons busy_mode_invalid and billing_evidence_unknown, capabilities), 7 "Follow-ups during a run", 8 "Stop" (point 3 "Hermes: interruption"), 9.1 "Lifecycle" (session.mode new, resume, bind), 9.4, 9.5 "Import binding and fail open" (point 2 "Hermes: like /resume"), 16.1 "Test doubles" (row Fake Hermes), the whole section 17 "Reference adapter: Hermes Agent" (17.1 to 17.10), 19 (R-AG-18, R-AG-19), 21 (hermes_adapter.platform_key, hermes_adapter.consumer).
  • docs/spec/10-backend-policy.md: 7 "Adapter check (I3)" (key_type = unknown counts as different from every target), 8 "Expected and reported", 10.1 "Hermes Agent".
  • docs/spec/11-extensions.md 11.3 "Runtimes and dependencies" (SDK dependencies compatible with the Hermes environment, checked in the Fake-Hermes test of M2-33, not here).
  • docs/spec/18-release.md 2 "Components" (row hermes-adapter, rule 1: VERSION and CHANGELOG.md per component).
  • docs/spec/19-security.md SEC-SC-1. docs/security/threat-model.md: T-AI-08 (residual "medium in the adapter"), T-SC-01. docs/RISKS.md: RK-12, RK-13.
  • docs/spec/23-quality.md: 1 "Principles", 2 "Gates", 3 "Component contract" (row "Python project") with "Documentation in code", 10 "Rulings" (R-QA-02), 11 (quality.coverage.*, repo.max_file_kb, deps.min_release_age).
  • docs/spec/02-architecture.md 6, the list "Places that carry the product name" (why the probe platform has no product name, Design note 7).
  • ADRs: 0022 (agents outside the server; "Confirmation": Fake-Hermes runs only against the public API), 0023 (signed invocation token, check in the adapter), 0024 (backend policy), 0026 (SDK), 0041 (Renovate with cooldown), 0044 (working name only at listed places), 0054 (trust marking in the context).
  • docs/DECISIONS.md: D-R-HERMES, D-L7, D-AG-17, D-AG-18, D-AG-19, D-W-79, D-W-100, section 10 "Rulings of the build milestones" (row format D-M2-32-<n>).
  • docs/quality/gates.md ("List", "Skipping and failing"), docs/quality/licenses.md ("License gate (make licenses)"), docs/quality/push-discipline.md, docs/quality/red-team-gate.md.
  • plugins/README.md, plugins/hermes-adapter/README.md (status "planned", "Fixed rules", "To check before the build").
  • docs/briefs/M2-23-generated-clients.md Design notes 9 (third bullet: the two known defects of the Python lines of the root Makefile), 10 (the key quality.coverage.python_min_percent) and 11 (shared tool versions); only these parts concern this ticket (Design note 2).
  • Repository files: root Makefile (variables PY_PROJECTS, PY_ON, targets fmt-check, lint, typecheck, test, coverage, licenses, osv), tools/pyproject.toml (tool versions and the ruff, mypy, pytest and coverage settings this project copies), tools/docs/tests/test_config_values.py, .gitleaks.toml (rule anthropic-credential-any), .pre-commit-config.yaml (hygiene hooks, check-added-large-files, text-rules), renovate.json (custom regex managers), sonar-project.properties (**/tests/** is test code, plugins/*/coverage.xml), tools/ci/license_check.py (SPDX header), tools/ci/components.py (plugins/ selects the gate component plugins).
  • Hermes Agent, read in the throwaway checkout of the pinned commit (never in a local installation): gateway/platforms/ADDING_A_PLATFORM.md, gateway/platforms/base.py, the plugin loader, the configuration reader, the credential resolution of the providers, the session store and the user guide pages on the CLI, busy_input_mode and access control.
  • Dependencies: the header names none. Milestone M0 is closed, so the tool chain, the root Makefile and the gates exist; this ticket touches no server code and needs no M1 or M2 ticket.

Fixed requirements that apply

Owner's fixed build rules (apply verbatim):

  1. No MVP. New from scratch and directly in the final state; stages are only the order.
  2. Strictly modular: clear module and service boundaries, dependencies only through defined interfaces, no cross access.
  3. Documentation in code: every public module, class, function and interface with a docstring/doc comment (purpose, parameters, return value, errors, example where useful); comments explain the why, not the what. API documentation is generated from code (OpenAPI from the routes, dartdoc, Python API docs in MkDocs).
  4. Documentation in the repo: README per module/service, ADR for every architecture decision, CHANGELOG per release, keep the documentation site current. A change without matching documentation is not done.
  5. Quality gates (SonarQube, linters, type checking, security scanners, test coverage) block the merge.

Rules for every brief:

  1. Test first: write the failing test, run it and keep the red output for the report, then make it green, then clean up.
  2. No en dash and no em dash in any file you write; the text-rules pre-commit hook rejects them. Use a hyphen or rephrase.
  3. White label: examples and test data use example.com and the people Alex, Sam and Robin; no company, host or personal names, no paths under a home directory.
  4. Limits, windows and thresholds only through named configuration keys listed in docs/operations/configuration.md; code comments and docs name the key, never the number.
  5. Specification first: a change to behaviour, API, event, right or configuration updates docs/spec/ or docs/api/ in the same change. The core contract (docs/spec/00-glossary.md, 03-data-model.md, 04-sync-protocol.md, 05-actions.md) is not edited by this ticket.

Product requirements implemented here:

  1. Adapter tests run against a pinned public version in a throwaway folder, never against a productive runtime, and nothing that exists changes (PLAN 1 point 2). Fake-Hermes as 09 16.1 describes it: a subprocess with the pinned Hermes version, a throwaway HERMES_HOME, the plugin found through the real plugin discovery, the model behind a local fake endpoint with fixed answers, the Hermes checkout unchanged before and after.
  2. No real model call and no costs: the model is a local fake endpoint; this ticket has no switch for real calls (PLAN 3 point 5, PLAN 8 "Test strategy").
  3. Each point of 09 17.10 is answered with evidence or the mapping is adapted; the pinned version lies in a throwaway folder and stays unchanged (plan row M2-32, PLAN 8 exit criterion 5).
  4. Only public Hermes plugin surfaces count as the adapter's way, and Hermes is never changed (09 17.1 and 17.2, plugins/hermes-adapter/README.md "Fixed rules"); documented override points, never parsing of text (D-L7, D-R-HERMES, R-AG-18).
  5. The actual billing mode is recognised from the key type only, never from a value; no credential value is read, stored, sent or logged by the harness, and no fixture holds one (09 17.8, 10 10.1, 09 4.8 billing_evidence, RK-13). An undeterminable type stays unknown and fails closed (10 7).
  6. Tests bind only 127.0.0.1 and port 0 (P-10); no network calls except the downloads of pinned versions (the Hermes commit, Python packages, a managed Python) (PLAN 3 point 5). Without network the recording test fails; it never skips (R-QA-02).
  7. Supply chain: the pinned commit is named by its full SHA, is older than deps.min_release_age, and its dependencies install from Hermes' own lock or from a hash-pinned list (SEC-SC-1, T-SC-01, ADR 0041).
  8. plugins/hermes-adapter meets the component contract of a Python project (23 3, "Documentation in code") and has VERSION and CHANGELOG.md (18 2 rule 1).

Scope

Paths are relative to the repository root. This ticket creates no new top-level make target, no migration, no operation, event, action or configuration key of the product, and changes nothing under docs/api/. The component Makefile gets the target record, called as make -C plugins/hermes-adapter record.

Create:

  • plugins/hermes-adapter/pyproject.toml, plugins/hermes-adapter/uv.lock: project hermes-adapter, version = "0.0.0", license = "Apache-2.0", classifier Private :: Do Not Upload, no runtime dependencies, [tool.uv] package = false (M2-33 adds the adapter package), dev group with ruff, mypy, pytest, pytest-cov, PyYAML and types-PyYAML in exactly the versions of tools/pyproject.toml; [tool.ruff], [tool.mypy], [tool.pytest.ini_options], [tool.coverage.*] per Design note 2.
  • plugins/hermes-adapter/VERSION (0.0.0), plugins/hermes-adapter/CHANGELOG.md (Keep a Changelog, English, like services/server/CHANGELOG.md).
  • plugins/hermes-adapter/Makefile: header comment like tools/design-tokens/Makefile, SHELL := bash, .SHELLFLAGS := -euo pipefail -c, target record (Design note 8).
  • plugins/hermes-adapter/tests/hermes_pin.toml (Design note 3); plugins/hermes-adapter/tests/hermes_requirements.txt only if the pinned commit has no uv.lock (Design note 4).
  • plugins/hermes-adapter/tests/conftest.py: the pytest option --record and the session fixtures of the recording.
  • plugins/hermes-adapter/tests/hermes_harness/: __init__.py, pin.py, integrity.py, provision.py, environment.py, sandbox.py, egress.py, fake_model.py, control.py, gateway.py, calls.py, observations.py, limits.py (Design notes 3 to 9).
  • plugins/hermes-adapter/tests/recording/probe_plugin/plugin.yaml and probe.py (Design note 7).
  • plugins/hermes-adapter/tests/recording/scenarios/<scenario>.json, one per scenario of Design note 9, and plugins/hermes-adapter/tests/recording/test_recording.py.
  • plugins/hermes-adapter/tests/unit/: test_pin.py, test_integrity.py, test_provision.py, test_environment.py, test_sandbox.py, test_egress.py, test_gateway.py, test_fake_model.py, test_control.py, test_calls.py, test_observations.py, test_fixtures.py.
  • plugins/hermes-adapter/tests/fixtures/recording/: <scenario>.json per scenario, observations.json, and README.md (English: what each file holds, which point it proves, how to record again).

Change:

  • docs/spec/09-agents.md (English, like the chapter): 17.10 becomes the table of answers (Design note 10); 17.1 (the pinned version and the evidence marker of steer), 16.1 row Fake Hermes (where the harness lives); and, only where a claim did not hold, the sections Design note 9 names for that point (17.2, 17.3, 17.4, 17.5, 17.7, 17.8, 17.9, 8 point 3, 9.5 point 2).
  • docs/spec/10-backend-policy.md 10.1, only where credential_type (Design note 9) shows another precedence or a further credential source.
  • docs/RISKS.md RK-12 and RK-13: column "Mitigation" (the recording runs in every test run against the pinned version; what it found); ratings unchanged.
  • docs/security/threat-model.md T-AI-08: the residual text, only if credential_type finds a public surface.
  • tools/ci/components.py and tools/ci/tests/test_components.py: chapter 09 selects the gate component plugins (Design note 11).
  • plugins/hermes-adapter/README.md (English: status "recording available (M2-32), adapter planned (M2-33)", layout of the tests, the pinned version by reference to tests/hermes_pin.toml, how to record again, the answers in one line each with a pointer to 09 17.10, "Fixed rules" adapted where a point changed them) and the row hermes-adapter/ of the table in plugins/README.md.
  • renovate.json: a regex custom manager and a package rule for the pin (Design note 3).
  • docs/DECISIONS.md section 10: rows D-M2-32-1 and following (Design note 12).
  • Root CHANGELOG.md under "Unreleased": the CI path filter for chapter 09 and, if made here, the Makefile and coverage changes below.
  • Only if the base branch does not contain M2-23 yet (Design note 2): root Makefile (typecheck, lint), docs/spec/23-quality.md 11 (key quality.coverage.python_min_percent), tools/docs/tests/test_config_values.py, docs/quality/gates.md (row coverage).

Out of scope

  • The adapter itself: its plugin.yaml, the adapter module, the logic modules, the installer, the compatibility layer that M2-33 builds on the surfaces found here, the supported version range of Hermes, the Fake-Hermes scenarios AG-08, AG-11, AG-13, AG-15 and AG-20, the leak test BR-13 and the compatibility test against the newest Hermes version (M2-33).
  • The Python SDK, its FakeServer and testing helpers (M2-29); the Claude-Code-Runner and Fake-Claude (M2-34, M2-35).
  • The context format of 09 17.6 beyond the request shapes recorded here (AG-20 in M2-33); the behaviour of cron delivery of 09 17.9 beyond the presence of cron_deliver_env_var and standalone_sender_fn.
  • Any change to docs/api/, to the core contract or to server code. The default value bothy of hermes_adapter.platform_key is a spec gap of M2-33 (it is no listed place in 02 6 "Places that carry the product name"); report it, do not change it.
  • From M2-23: everything except the two Makefile defects and the coverage key of Design note 2 (api-clients, bootstrap-ci, the SDK projects, the TypeScript key).
  • Publishing the component and its release chain (M7).

Boundaries

  • Work in a dedicated git worktree on the branch feat/M2-32-hermes-recording; one writer per worktree.
  • Conventional Commits with the component or area as scope (for example test(hermes-adapter): ..., docs(spec): ..., build(gates): ...) and a DCO sign-off on every commit (git commit -s); squash into meaningful commits before the merge (docs/quality/push-discipline.md).
  • Never push to main, never tag and never create remote repositories; the only remote steps are the branch push and the pull request of AGENTS.md. Never publish the component.
  • Do not modify files outside the scope above, except the CHANGELOG, README and docs/DECISIONS.md entries named in this brief.
  • Follow the host rules supplied by the launcher (ports, machine rules).
  • Tests bind only 127.0.0.1 and port 0 (P-10). The Hermes process tree opens no listening socket at all (Design note 5); the harness never connects to a port it did not open itself.
  • Stop only processes you started: the harness starts Hermes in a new process group and ends exactly that group, by its ID; never by name (no pkill, no killall), never a process the harness did not start.
  • Temporary directories only via mktemp -d or the pytest temporary directory factory, outside the repository, removed at the end (restore the write bits of the read-only checkout first).
  • Never read or print secrets. The Hermes subprocess and the provisioning steps get an environment built from an allowlist, never the caller's environment (Design note 5). Fake credentials are generated at run time, live only in the throwaway homes and in memory, and never appear in a tracked file, a fixture, a log line or a test failure message.
  • The machine that runs the tests may run other Hermes installations. Never find, read, start, stop, signal, configure or connect to any Hermes installation, Hermes home, gateway, service or process that the harness did not create in its throwaway root; never run a hermes executable from PATH; never run a Hermes command that installs or starts a system service, updates Hermes, logs in, pairs or opens a setup wizard.
  • No real model and no real credential: every provider and auxiliary model setting of the pinned version points at the fake endpoint, and Design note 5 makes every stored credential of the machine unreachable. A run that sees an unexpected credential or an attempt to reach a model provider fails at once.
  • Hermes is never changed: no patch, monkeypatch, sitecustomize, .pth file or import hook into Hermes code; the probe uses only what the pinned version offers to platform plugins, plus the internal call that the bullets credential_type and session_switch of Design note 9 allow when no public surface exists, recorded as internal.
  • No third-party prose in fixtures (Design note 8); no network calls except the downloads of pinned versions.
  • Never lower a threshold, disable a gate or delete a test. Do not extend quality/licenses/allowed.txt; if a dev tool of the project fails the license gate, stop with Verdict: BLOCKED <package and license>.
  • If the core contract lacks something this ticket needs, stop with Verdict: BLOCKED <reason> instead of building a deviation (P-11). If an answer can only be absorbed by a change to docs/api/ or to the core contract, stop with Verdict: BLOCKED <point and needed change> as well.
  • This brief was written before milestone M1 closed (D-W-100). Where the merged code of an M1 ticket names a package, type, function, file or make target differently from this brief, follow the code and list each difference in the report; where it behaves differently from the specification, stop with Verdict: BLOCKED <reason>.
  • The same holds for M2-23 if it is merged on the base branch: its names and its implementation of the Makefile lines and of the coverage key win over Design note 2; list each difference.

Design notes

  1. What this ticket builds and what it leaves to M2-33. M2-32 builds the recording harness, the probe plugin, the scenarios, the fixtures and the spec changes; M2-33 builds the adapter and its Fake-Hermes scenarios on the same harness (it imports hermes_harness, replaces the probe by the adapter and keeps the pin, the isolation and the integrity check). The harness API that M2-33 imports is: load_pin(tests/hermes_pin.toml), provision() into a temporary root outside the repository and release(), ThrowawayHomes, build_environment, the sandbox of sandbox.py on macOS, EgressRecorder, FakeModel with FakeCredentials, HermesGateway, and snapshot_checkout, installed_digest and assert_unchanged; M2-33 adds no second pin, no archive download and no Hermes cache inside the repository. M2-33 changes this project's pyproject.toml, uv.lock, VERSION and CHANGELOG.md instead of creating them, keeps tests/hermes_harness in the mypy and coverage scope, and keeps requires-python and ruff target-version no lower than Design note 2 sets them. Ruling for docs/DECISIONS.md: the live recording test is part of the component's default test run, so make test and every make check that selects the gate component plugins repeat it; without network it fails, it never skips. Reasons: 09 17.3 asks for a recording test that holds the call patterns, and RK-12 needs the check continuously, not once; a recording that only ran on the build day would be a stored value whose rule nobody checks again (R-QA-02). Ruling for docs/DECISIONS.md, public surface: a surface is public when the pinned version documents it for platform plugins (ADDING_A_PLATFORM.md, the user guide) or when it is a name without leading underscore on BasePlatformAdapter, on the plugin context passed to register(ctx) or on PlatformConfig; an underscore method that the pinned base class documents as an override point (_send_exec_approval_prompt, 09 17.1) is public for overriding only; everything else is internal.

  2. Component project and the overlap with M2-23. This ticket creates the first Python project under plugins/; the root Makefile discovers it through PY_PROJECTS, and fmt-check, lint, typecheck, test, coverage and licenses run on it (23 3, gates.md "Skipping and failing").

    • pyproject.toml: requires-python = ">=3.12" and ruff target-version = "py312" like tools/pyproject.toml (the harness runs in the project environment; the probe runs inside the pinned Hermes and is written for the oldest Python line the pinned version supports, Design note 7); ruff with the rule set of tools/pyproject.toml including D with convention google, per-file ignores only for tests/**/test_*.py and tests/**/conftest.py, so hermes_harness and the probe carry docstrings (rule 3); mypy strict = true over tests/hermes_harness, excluding tests/recording/probe_plugin/ (it imports Hermes, which the project environment does not contain); pytest with testpaths = ["tests"], pythonpath = ["tests"], addopts and filterwarnings like tools/pyproject.toml; coverage with branch = true, source = ["tests/hermes_harness"], omit the probe plugin, and fail_under equal to quality.coverage.python_min_percent with a comment naming the key. Ruling for docs/DECISIONS.md: the harness lives under tests/ because it is test support code that the component never ships; it is still measured, typed and documented like product code, and the probe is outside mypy and coverage because it only runs in the Hermes process, where the recording test proves it.
    • If the base branch does not contain M2-23 yet, two defects of the root Makefile and one missing key stop the gates; fix exactly these, as M2-23 Design notes 9 and 10 prescribe, and nothing else of M2-23: typecheck runs uv run --locked mypy inside each Python project (cd <project>), because mypy finds neither configuration nor target from the repository root; lint excludes <project>/.venv and <project>/generated from bandit besides <project>/tests; quality.coverage.python_min_percent ("Minimum coverage per Python project under sdk/ and plugins/", same standard value as quality.coverage.go_min_percent, held by fail_under in [tool.coverage.report] of each such project) is added to 23 11, SOURCES in tools/docs/tests/test_config_values.py checks every tracked Python project under sdk/ and plugins/ against it, and gates.md row coverage names it. If M2-23 is merged, use what it built; if its SOURCES entry checks only sdk/python, extend it to every tracked project. A conflict with a parallel M2-23 at the merge is resolved to one implementation (PLAN 3 point 2).
    • Verify that a run below the threshold fails before you rely on fail_under, and that bandit with all Python files excluded exits 0.

  3. The pin. tests/hermes_pin.toml (TOML, SPDX comment line) holds repository (exactly https://github.com/NousResearch/hermes-agent, the URL in 09 17.1), tag, commit (40 lower-case hex digits), python (the CPython line for the throwaway environment, the newest line the pinned requires-python allows), install (uv-lock or requirements), extras (list, empty unless the probe platform or the providers need one), and recorded_on (date). Ruling for docs/DECISIONS.md: pin the newest release tag of the public repository that is older than deps.min_release_age on the build day (the Renovate cooldown, ADR 0041), resolved to its commit with git ls-remote; if the repository has no release tags, pin the newest commit of its default branch older than that window and leave tag out. renovate.json gets a regex custom manager for this file (datasource github-tags, package NousResearch/hermes-agent, currentValue from tag, currentDigest from commit; for a tag-less pin git-refs) and a package rule with prBodyNotes: record again with make -C plugins/hermes-adapter record, review the fixture diff, update 09 17.10 before merging; the update fails the test run until then, on purpose (the pattern of the rule "release binaries"). pin.py exposes HermesPin (frozen dataclass of these fields) and load_pin(path) -> HermesPin, which raises PinError for a missing field, a short or non-hex commit, a branch name in tag or an unknown install.

  4. Provisioning in a throwaway folder (provision.py, integrity.py). provision(pin, root) -> HermesInstall works only inside root, a fresh temporary directory outside the repository:

    • Checkout: git init, git fetch --depth 1 <repository> <commit>, git checkout --detach FETCH_HEAD; HEAD must equal commit. Then the checkout becomes read-only (write bits removed recursively), so any write attempt fails loudly.
    • Install from an export, never from the checkout: git archive <commit> unpacked into root/build, because build backends write build/ and metadata into the source tree. With install = "uv-lock": uv sync --locked --no-dev --no-editable of the export with UV_PROJECT_ENVIRONMENT=root/venv; with install = "requirements": uv venv root/venv --python <python> and uv pip install --require-hashes -r plugins/hermes-adapter/tests/hermes_requirements.txt plus the export without dependencies. Ruling for docs/DECISIONS.md: use Hermes' own uv.lock when the pinned commit has one; otherwise generate plugins/hermes-adapter/tests/hermes_requirements.txt with uv pip compile --generate-hashes from the export, check it in next to the pin, and let record regenerate it when the pin changes (name it in the Renovate note).
    • Provisioning runs with the allowlist environment of Design note 5 without proxy variables (it needs the network), with UV_CACHE_DIR and UV_PYTHON_INSTALL_DIR set to the caller's uv cache and Python directories (asked from uv cache dir and uv python dir), so repeated runs do not download again, and with a throwaway HOME, so no git or uv credential helper of the caller is used.
    • Integrity: tree_digest(path) -> str is the SHA-256 over the sorted entries (relative path, file mode, SHA-256 of the content, symlink target); snapshot_checkout(path) -> CheckoutState records HEAD, git status --porcelain --ignored and the digest; installed_digest(venv) -> str digests the installed Hermes distribution files (from its RECORD); assert_unchanged(before, after) raises IntegrityError naming each changed, added and removed path. Snapshots are taken after provisioning and after the last scenario.
    • HermesInstall carries checkout, venv, executable (the Hermes CLI inside the venv, by absolute path) and both snapshots. release(install) restores the write bits so the temporary root can be removed.

  5. Isolation (environment.py, sandbox.py, egress.py, gateway.py). Ruling for docs/DECISIONS.md, the isolation set; each part closes one way to a real credential, a real model or another installation:

    • Throwaway homes: ThrowawayHomes creates under the temporary root a home, one Hermes home per configuration (hermes/<profile>), a working directory work, tmp and a shim directory shims; assert_inside(root, path) resolves symlinks and refuses any path outside the root.
    • Environment allowlist: build_environment(homes, profile, proxy_url, extra) -> dict[str, str] returns only PATH (shims first, then the venv's bin, then /usr/bin and /bin), HOME, HERMES_HOME, XDG_CONFIG_HOME, XDG_DATA_HOME, XDG_CACHE_HOME, XDG_STATE_HOME, TMPDIR, LANG, TZ (UTC), PYTHONDONTWRITEBYTECODE, PYTHON_KEYRING_BACKEND (the null backend of keyring), NO_COLOR, TERM (dumb), the proxy variables in upper and lower case pointing to the egress recorder, NO_PROXY with 127.0.0.1 and localhost only, and the scenario's own names (fake credentials, base URLs, as the pinned version reads them). Nothing of the caller's environment passes; in particular no API key, token, SSH_AUTH_SOCK, DBUS_SESSION_BUS_ADDRESS or cloud credential.
    • Credential stores: the shim directory holds executables named security and secret-tool that exit non-zero and record that they were called; the keyring backend is null; on Linux no D-Bus address exists in the environment, so the Secret Service is unreachable.
    • Network: the egress recorder (EgressRecorder, an HTTP proxy on 127.0.0.1 port 0) answers every request, CONNECT or absolute URI, with 403, records only host:port and never forwards. On macOS the Hermes process tree runs under sandbox-exec with a profile that denies outbound network except to loopback (sandbox.py), because the macOS keychain is reachable through the system API whatever the environment says; on Linux the environment above already makes every stored credential unreachable, so a leaked request could carry only fake credentials. A fake credential reaching a real provider costs nothing; that is the guarantee, and the report states it.
    • No listener: every platform of the pinned version except the probe and every server component it offers (API server, webhook receivers, dashboards) is disabled in the throwaway configuration; after the probe connected, HermesGateway.listening_sockets() lists the listening TCP and UDP sockets of the process tree and the run fails on any.
    • Process: HermesGateway starts the foreground gateway command of the pinned CLI by absolute path with start_new_session=True, waits for the probe's connect within a named deadline, and stop() ends the process group (SIGTERM, then SIGKILL after the stop deadline) and fails if a member survives. Toolsets of the probe platform: exactly one built-in tool without effect outside the throwaway homes and without network (for example a todo tool or a file read inside work); never a terminal, code execution, web, browser, MCP, delegation or messaging tool. If the pinned version needs model metadata at start, the fake endpoint serves it; the real metadata service is never allowed.

  6. Fake model (fake_model.py). FakeModel serves on 127.0.0.1 port 0 an OpenAI-compatible route (chat completions, streaming and non-streaming, and the model list) and an Anthropic route (messages, streaming events from message start to message stop, tool use blocks), plus any further route the pinned version needs for the configured providers, each replaying the scenario's model_script: per turn either tool calls or text in fixed chunks, with fixed usage numbers, optionally held on a named latch until the test calls release(latch). A request that the script does not cover (titles, compression, memory) gets a fixed short answer and is recorded as unscripted. Each request is recorded as its shape: route, model name, sequence of roles, the texts the scenario injected (verbatim), tool names and call IDs, and for every other text only sha256:<hex> and its length. Credentials: FakeCredentials generates per run a fake API key and a fake OAuth token, in the format the pinned version uses to classify credentials if it does (random content, never written to a file outside the throwaway homes); the endpoint compares the presented header with them and records only CredentialKind (api_key, oauth, none, unexpected); any other value is unexpected, is discarded at once and fails the run. Point every provider base URL and every auxiliary model of the pinned version at this endpoint (configuration file in the Hermes home and the variables the pinned version reads).

  7. Probe plugin and control channel (probe_plugin/, control.py). The probe is a platform plugin with plugin.yaml (kind: platform) and one module probe.py, the only file that imports Hermes; it uses the standard library only and registers through ctx.register_platform(...) in register(ctx). Ruling for docs/DECISIONS.md: the probe's platform name is recording_probe, because the product name may appear only at the places of 02 6 and the adapter's hermes_adapter.platform_key belongs to M2-33. The harness copies the probe into the place of the throwaway Hermes home where the pinned version discovers plugins (record the place as a fact). The probe subclasses BasePlatformAdapter, reads its settings only from PlatformConfig.extra (or the pinned equivalent; never os.getenv or os.environ, 09 17.2): the control URL and its flags (supports_draft_streaming, supports_status_text, enforces_own_access_policy and the policy value). It overrides every method that 09 17.3 maps and the pinned base class offers, with the pinned signatures; each override posts {"method", "args"} to the control channel, returns what the base contract expects (sequential message IDs for sends, None from format_tool_event) and holds no other logic. On connect() it starts a loop that takes commands from the control channel and executes them: inject (a MessageEvent built with build_source(chat_id = <channel id>, chat_type = "thread", thread_id = <thread id>, user_id = <user id>, user_name = <name>, message_id = <message id>) as 09 17.4 prescribes, passed to handle_message), interrupt and session (the ways of the bullets interrupt_outcome and session_switch of Design note 9), credential_type (the surface of the bullet credential_type) and describe (the presence and signature of each surface of note 9, from inspect). ControlServer (on 127.0.0.1 port 0) keeps the calls in arrival order, serves each command once in order, and offers wait_for(method, predicate) with a named deadline.

  8. Recording, normalization and fixtures (calls.py, observations.py, conftest.py, test_recording.py).

    • normalize(calls, redactions): IDs (chat, thread, message, session, tool call) become <id:n> in order of first use; the temporary root and the real home become <tmp> and <home> as path prefixes, also where such a path stands inside a longer string; the host name and the user name become <host> and <user> only where they are the whole string or a whole segment of an absolute path outside <tmp> and <home>, never inside a longer token and never below <tmp> or <home>, so the normalized output does not depend on the machine or the OS user that runs it; timestamps and durations are dropped; a string is kept verbatim only if it is a scenario text, an identifier (^[A-Za-z0-9_.:-]{1,64}$) or a placeholder path, else it becomes sha256:<hex> with its length. Ruling for docs/DECISIONS.md: fixtures hold no third-party prose (system prompts, status texts and messages of Hermes are hashed), because it would copy foreign text into this repository and could carry machine details.
    • derive_pattern(calls): the ordered list of calls with method, argument shape (names and types, enum-like values verbatim) and the final content of text arguments; runs of consecutive send_draft or set_status_text calls collapse into one entry with their last content. Ruling for docs/DECISIONS.md: the comparison ignores the repeat count of these two methods (it depends on chunk timing) and compares everything else exactly. compare_patterns(expected, actual) -> list[str] names the first difference and ends with the hint make -C plugins/hermes-adapter record.
    • Fixture <scenario>.json: scenario, pin (tag and commit), inputs_sha256, calls (normalized), model_requests (shapes and CredentialKind), pattern. observations.json: pin, inputs_sha256, recorded_on, plugin_discovery, surface (Design note 9 "Surface facts"), egress (sorted host:port with phase start or turn), credential_store_calls, and per point slug its facts and holds. All JSON with sorted keys, two-space indent and a final newline, each file below repo.max_file_kb; digests and commit IDs sit under names without key, token, secret or password, so gitleaks' generic rule does not read them as secrets, and no gitleaks allowlist entry is added.
    • inputs_sha256 is the digest over tests/hermes_pin.toml, tests/recording/probe_plugin/, tests/recording/scenarios/ and the modules of tests/hermes_harness/. Ruling for docs/DECISIONS.md: a change to any of them without recording again fails the offline tests, because the fixtures are evidence only while the inputs that wrote them hold.
    • Without --record the live test runs every scenario, derives patterns and observations and compares them with the fixtures. With --record (target record of the component Makefile: uv run --locked pytest tests/recording --record) it writes the fixtures, and only after every scenario passed and the integrity check held, through a temporary directory replaced at the end, so a failed run never leaves partial fixtures.
    • limits.py holds the named deadlines of the harness (provisioning, gateway start, a scenario step, stop grace). Ruling for docs/DECISIONS.md: these bound a test, not product behaviour, so they are named module constants and no configuration keys (the precedent of M1-73's probe); code and docs name the constant, never the number.

  9. The six points. Each point has a slug, scenarios, observed facts and a claim; observations.py computes holds from the facts, so the verdict is not written by hand. When a claim does not hold, the adaptation below applies; it changes only the named sections of 09 (and 10 10.1), never docs/api/ or the core contract. Ruling for docs/DECISIONS.md: one row per adaptation that applied, with its point.

    • progress_hooks (scenarios tool_and_text: probe flags as 09 17.3 maps them, the model calls the safe tool and then streams a text answer in several chunks; text_without_draft: draft streaming off, status text on, a streamed text answer). Facts per method: number of calls, position relative to the model requests, argument shape (the fields of the event given to format_tool_event; whether send_draft text is cumulative or a delta; when edit_message gets finalize=True), and whether any send carried a progress text. Claim: format_tool_event gets a structured event at the start and at the end of the tool call; returning None suppresses any progress message; send_draft is called during the streamed answer and its last text equals the answer; set_status_text is called at least once; without draft streaming the answer ends with exactly one edit_message with finalize=True. Adaptation in 17.3: every row states the observed pattern; a missing end signal closes the step at the next tool event, the next draft or answer, or on_processing_complete, whichever comes first; an event with text only gives that text verbatim as the step title, never parsed (D-L7); no send_draft calls make the buffered edit_message content the source of /last_text_excerpt; a method that is never called keeps its row with the note "not called in <tag>".
    • steer (scenarios busy_steer, busy_queue, busy_interrupt: display.platforms.recording_probe.busy_input_mode set to the mode, read back as the pinned version resolves it; Alex's message starts a turn whose model stream is held, Sam's message arrives in the same thread, then the latch is released). Facts: whether the per-platform key and each value are accepted, and whether Sam's text appears in a later model request of the same turn, in a new turn after the first ended, or ended the first turn early. Claim: steer is accepted for the plugin platform and Sam's text reaches the running turn without ending it. Adaptation: 17.5 sets queue (followups = queue), the adapter never reports steer, 17.1 records the finding, and the report tells M2-33 that its AG-13 scenario for Hermes covers queue only; if only a global display.busy_input_mode exists, 17.5 names the global key (one Hermes profile is one agent principal, 17.2).
    • credential_type (scenarios credential_api_key_only, credential_oauth_only, credential_both, credential_none, credential_login_file_only, credential_other_provider, credential_per_turn; each profile with its own Hermes home and gateway start; Anthropic as provider except credential_other_provider, which uses an OpenAI-compatible provider with an API key; credential_login_file_only puts a fake subscription login of another tool into the throwaway home at the place the pinned version reads, and records "not read" if it reads none; credential_per_turn changes the credential source between two turns without restart where the pinned version rereads one). Ground truth is the CredentialKind the fake endpoint saw for each turn; immediately before each turn the probe reports the answer of the candidate surface (the public one if the pinned version has it, else the internal function Hermes itself uses to pick the credential, named with module and signature). Facts per profile: ground truth, surface answer, surface name, public or internal, and the sources each profile used (process environment, the .env of the Hermes home, a login file). Claim: a public surface exists, its answer equals the ground truth in every profile, and OAuth wins in credential_both. Adaptations: with an internal function that matches the ground truth in every profile, 17.8 names it as the source of the compatibility layer with a contract test against the pinned version, and RK-13 and T-AI-08 keep "non-public surface" and "non-public interface"; with no candidate that matches in every profile, 17.8 makes the Anthropic rows unknown (fail closed) for the profiles it misclassifies, and the report names this as an owner decision with the recommendation to keep fail closed; another precedence corrects the sentence in 17.8 and the table in 10 10.1; a login file of another tool that Hermes uses adds to 10 10.1 that in organization the profile's home must not contain it (the installer checks); no per-turn choice changes "for every turn" in 17.8 to "at start", while the adapter still checks before every run.start. With a public surface that matches, the residual texts of T-AI-08 and RK-13 change to "public interface of <tag>" and "public surface of <tag>", ratings unchanged.
    • interrupt_outcome (scenario interrupt: Alex's message starts a turn whose model stream is held; the probe interrupts through the public way the pinned version offers a platform, in this order: a documented adapter or gateway method, else Hermes' own stop command passed through handle_message; then the latch is released). Facts: the way used, the value and type of the outcome given to on_processing_complete, whether the model stream was abandoned and whether a send followed. Claim: a public way interrupts the turn and the outcome identifies a cancellation. Adaptations: an outcome without that meaning changes the row on_processing_complete in 17.3: after an interruption the adapter itself started for run.cancel_requested, it reports cancelled with the reason of that request whatever Hermes reports; no public way changes 09 8 point 3 and 17.3 for Hermes: the adapter cannot stop a running turn, the server ends the run with cancel_timeout (09 8 point 4) and the adapter drops the turn's later output; the report names it as an owner decision with the recommendation to accept it for this version.
    • session_switch (scenarios session_new: two turns, then Hermes' new-session way, then a third turn; session_resume: a session in each of two threads, then in the second thread the resume way with the first thread's session ID; session_resume_missing: resume with an ID that does not exist). Facts: the way used (Hermes command through handle_message, the resume resolver, or a documented method), the session ID before and after each step and how the probe read it, whether earlier messages appear in the next model request, and what Hermes does with a missing ID. Claim: both switches work through public ways and the session ID is readable through a public way. Adaptations, in this order for reading the ID: a documented plugin or adapter API; Hermes' session store through a documented, machine-readable interface of the pinned version; otherwise the internal function Hermes uses, named in the facts, behind M2-33's compatibility layer with a contract test against the pinned version (as for the credential type), stated in 17.4 and covered by RK-12. Never parse the text output of a command. The ways for new and bind go into 17.3 (row "Hermes commands"), 9.5 point 2 and 17.4; if a missing ID silently starts a new session, 9.5 point 4 states that the adapter detects it by the changed ID and reports bind_failed: session_missing.
    • own_access_policy (scenarios access_policy_own: the probe declares its own access policy with policy "allowlist" as the pinned version expects it, Hermes' own allowlist of the platform is empty and no allow-all setting is set; access_policy_default: the probe declares nothing; in both Robin, on no Hermes list, sends a message). Facts: whether a model request follows, what Hermes sent back (hash and length), and how the pinned version reads the declaration (attribute or method and value). Claim: with the declaration the message reaches the turn, without it Hermes' default policy holds it back. Adaptations: if the declaration is not honoured but a documented per-platform "allow all users" setting exists, 17.7 point 3 says that in organization the installer sets it for the adapter platform and layer 2 is the gate, while personal keeps the allowlist with the credential_holder; if neither exists, 17.7 says that layer 3 holds the list of the people the server lets trigger (agent.triggerers.read), written by the installer with a Hermes restart on change, and the report names it as an owner decision with the recommendation to accept it until a later Hermes version offers a plugin policy.
    • Surface facts (part of every live run, not a point of their own): for each Hermes surface that 09 17.2, 17.3 and 17.9 name (ctx.register_platform, PlatformConfig.extra, env_enablement_fn, apply_yaml_config_fn, every method of the table in 17.3, MAX_MESSAGE_LENGTH, splits_long_messages, cron_deliver_env_var, standalone_sender_fn, thread_sessions_per_user) the probe reports presence and parameter names in the pinned version, and the harness reports where the probe was discovered. A surface that is missing gets the note "missing in <tag>" in its section and, where 17.2 or 17.9 relied on it, the way the pinned version offers instead.
    • Egress: every host:port the recorder saw goes into observations.json with its phase; an attempt to the API host of a configured model provider fails the run (a base URL the harness missed).

  10. Spec and documentation text (English in chapters, READMEs and CHANGELOGs). 09 17.10 keeps its heading (the plan row and the README cite it) and gets an introduction (measured in M2-32 against Hermes Agent <tag>, commit <first 12 hex digits>, recorded on <date>, with a fake model; evidence in plugins/hermes-adapter/tests/fixtures/recording/) and a table with the columns "Point", "Result", "Evidence", "Consequence": one row per slug of Design note 9 with the slug in backticks in the first cell, "confirmed" or "adapted" first in "Result" followed by the answer in one or two sentences, the fixture file names in "Evidence", and the changed sections (or "none") in "Consequence". 17.1: one bullet naming plugins/hermes-adapter/tests/hermes_pin.toml as the pinned version for tests, without repeating the tag (it stands only in 17.10, where a test holds it); the steer bullet with its evidence instead of "to be checked in the pinned version". 17.3 last paragraph and 17.8 "[derived] ... to be checked against the pinned version before use" give way to the evidence. 16.1 row Fake Hermes, column "Setup": "Harness plugins/hermes-adapter/tests/hermes_harness (M2-32)". After this ticket no "to be checked" remains in 09 17.1 to 17.9. Write no status code next to a problem code in any text (make docs checks such pairs). The tag and the commit appear in documentation only in 09 17.10, where test_section_17_10_matches_the_observations holds them; READMEs, RISKS.md and the CHANGELOG point to tests/hermes_pin.toml instead of repeating them, so no version number in the docs is unchecked.

  11. CI path filter. test_fixtures.py reads observations.json, the scenario fixtures, the pin and 09; it needs neither network nor Hermes. In CI a change under docs/ selects no gate component (NO_COMPONENT_PREFIXES in tools/ci/components.py), so a change of 09 alone would not run the checks that bind 17.10 to the evidence. Ruling for docs/DECISIONS.md: tools/ci/components.py adds the prefix docs/spec/09-agents.md to the gate component plugins, because chapter 09 is the contract of the adapters under plugins/; tools/ci/tests/test_components.py gets the case that this file selects plugins, while the existing case with docs/spec/07-permissions.md still selects none.

  12. Rulings for docs/DECISIONS.md, one row each with date, decision, reason and place: the live recording in the default test run and the definition of a public surface (note 1); the project layout with the harness under tests/, mypy and coverage scope, and, if done here, the overlap fixes of M2-23 (note 2); the pin rule and the Renovate rule (note 3); install from an export, read-only checkout, integrity of checkout and installed files, and the lock or requirements choice (note 4); the isolation set including the sandbox on macOS (note 5); the probe name recording_probe (note 7); fixtures without third-party prose, the pattern comparison and the inputs digest (note 8); the named deadlines (note 8); each adaptation that applied (note 9); the CI path filter for chapter 09 (note 11).

Tests (write them first)

Unit and offline tests, plugins/hermes-adapter/tests/unit/ (pytest, run by the root make test; no network, no Hermes):

  • test_pin.py: test_pin_file_is_complete_and_valid (the checked-in pin: repository equals the URL in 09 17.1, 40-hex commit, Python line, install mode, date); test_load_pin_rejects_invalid_pins (short SHA, upper-case hex, branch name as tag, unknown install mode, missing field: each PinError).
  • test_integrity.py: test_tree_digest_is_stable; test_tree_digest_detects_change[modified], [added], [removed], [mode], [symlink]; test_checkout_state_detects_moved_head_and_untracked_files (a throwaway git repository made with git init in the temporary directory); test_read_only_checkout_refuses_writes_and_release_restores_them; test_assert_unchanged_names_each_path.
  • test_provision.py: test_provision_from_a_local_repository (a throwaway git repository with a minimal pyproject.toml and a uv.lock without dependencies, given to provision as a file:// URL in a HermesPin built by the test: the checkout is at the pinned commit and read-only, the build happens in the export, the venv lies under the root, and a file written into the checkout afterwards makes assert_unchanged name it); test_provision_refuses_a_commit_that_does_not_match.
  • test_environment.py: test_environment_is_an_allowlist (the caller's environment holds ANTHROPIC_API_KEY, OPENAI_API_KEY, GH_TOKEN, SSH_AUTH_SOCK, DBUS_SESSION_BUS_ADDRESS, AWS_ACCESS_KEY_ID and HERMES_HOME with fake values; none reaches the result, and the result has exactly the allowed names); test_homes_live_inside_the_temp_root; test_assert_inside_refuses_paths_outside_the_root (also through a symlink); test_proxy_variables_point_to_the_egress_recorder (upper and lower case; NO_PROXY is loopback only); test_keyring_is_null_and_store_shims_come_first; test_provisioning_environment_has_no_proxy_and_no_credentials.
  • test_sandbox.py: one case per platform, chosen by sys.platform inside the test, no skip marker: on macOS test_sandbox_denies_outbound_except_loopback (under the profile a TCP connect to 192.0.2.1 port 443, a documentation address that routes nowhere, fails at once with a permission error, and a connect to a listener of the test on 127.0.0.1 port 0 succeeds); on Linux test_linux_environment_reaches_no_credential_store (no D-Bus address, HOME inside the root, the shims first on PATH).
  • test_egress.py: test_connect_and_absolute_uri_are_refused_and_recorded (CONNECT example.com:443 and GET http://example.com/x both get 403; the record is example.com:443 and example.com:80; no outbound socket is opened, asserted with a patched socket.create_connection).
  • test_gateway.py (with a stand-in command of the test instead of Hermes): test_stop_ends_the_whole_process_group (a parent that starts a child; after stop() no member of the group survives, and nothing outside the group is signalled); test_listening_sockets_reports_a_listener (a stand-in that listens on 127.0.0.1 port 0 is reported, one without listener gives an empty list); test_start_fails_after_the_named_deadline_without_connect.
  • test_fake_model.py: test_openai_stream_replays_the_script (chunks, tool call deltas, end marker); test_anthropic_stream_replays_the_script (event order, tool use block); test_hold_latch_blocks_until_released; test_credential_kind_is_classified_and_no_value_is_kept (fake API key in the API key header gives api_key, fake OAuth token as bearer gives oauth, none gives none, another value gives unexpected and marks the run failed; neither the stored request nor repr of the recorder contains a value); test_unscripted_request_gets_the_default_answer; test_request_shape_keeps_scenario_text_and_hashes_other_text; test_binds_loopback_port_zero.
  • test_control.py: test_calls_are_kept_in_arrival_order; test_commands_are_served_once_in_order; test_wait_for_fails_after_the_named_deadline.
  • test_calls.py: test_ids_become_placeholders_in_order_of_first_use; test_temp_root_home_host_and_user_are_redacted; test_user_and_host_inside_tokens_stay_verbatim (user name hermes: hermes_cli.config and <tmp>/hermes/probe stay unchanged); test_normalization_does_not_depend_on_user_or_host (the same calls normalized under two different user and host names give byte-equal output); test_third_party_text_is_hashed_and_scenario_text_kept; test_pattern_collapses_only_draft_and_status_runs; test_compare_names_the_first_difference_and_the_record_hint.
  • test_observations.py: test_claim_of_each_point (synthetic recordings per slug: one where the claim holds, one per way it fails; holds and the facts as expected).
  • test_fixtures.py (the checked-in evidence; red until the first recording):
    • test_every_point_has_an_observation: observations.json has exactly the six slugs of Design note 9, each with facts and holds.
    • test_fixtures_match_the_pin: every fixture names the tag and commit of tests/hermes_pin.toml.
    • test_fixtures_match_the_recording_inputs: inputs_sha256 of every fixture equals the digest computed now; a changed probe, scenario, harness module or pin without recording again fails here.
    • test_section_17_10_matches_the_observations: 09 17.10 has one row per slug; "Result" starts with "confirmed" exactly when holds is true, else with "adapted", and then "Consequence" names at least one section; every file named in "Evidence" exists; the introduction names the pinned tag and the first 12 digits of the commit.
    • test_no_open_check_remains_in_section_17: no "to be checked" in 09 17.1 to 17.9.
    • test_fixtures_hold_no_credential_and_no_third_party_text: the regex of the rule anthropic-credential-any read from .gitleaks.toml finds nothing in the fixtures (and does match a value made by FakeCredentials, so a leak would be caught); every string is a scenario text, an identifier, a placeholder or a sha256: digest.
    • test_fixtures_are_white_label_and_canonical: no e-mail address outside example.com, no absolute path, user names only Alex, Sam and Robin; sorted keys, two-space indent, final newline, each file below repo.max_file_kb.

Repository tools, tools/ci/tests/test_components.py: new case docs/spec/09-agents.md selects plugins; the existing case with docs/spec/07-permissions.md stays empty (red before the change of tools/ci/components.py).

Live recording, plugins/hermes-adapter/tests/recording/test_recording.py (needs network for the pinned downloads; provisions once per session; fails without network):

  • test_recording[<scenario>] for each scenario of Design note 9 (tool_and_text, text_without_draft, busy_steer, busy_queue, busy_interrupt, credential_api_key_only, credential_oauth_only, credential_both, credential_none, credential_login_file_only, credential_other_provider, credential_per_turn, interrupt, session_new, session_resume, session_resume_missing, access_policy_own, access_policy_default): the pattern equals the fixture's pattern.
  • test_observations_match_the_fixture: computed facts, holds, surface facts, plugin discovery and egress equal observations.json.
  • test_hermes_opened_no_listener, test_no_unexpected_credential_and_no_model_provider_contacted.
  • test_pinned_hermes_is_unchanged (defined last in the module): HEAD equals the pinned commit, git status --porcelain --ignored is empty, the tree digest and the installed digest equal the snapshots after provisioning, and the temporary root lies outside the repository. The session fixture's teardown removes the temporary root and fails the run if anything remains or if a process of the group survived.

Test IDs of the plan row (the row names no chapter test ID):

  • "every item from 09 is answered with evidence or the mapping is adjusted": test_every_point_has_an_observation, test_section_17_10_matches_the_observations, test_no_open_check_remains_in_section_17, test_observations_match_the_fixture and test_recording[<scenario>].
  • "the pinned version lies in a throwaway folder and stays unchanged": test_pinned_hermes_is_unchanged, the teardown check of the temporary root, test_fixtures_match_the_pin and test_read_only_checkout_refuses_writes_and_release_restores_them.

Red runs: keep the red output of the unit tests before their modules, of test_fixtures.py before the first recording and of test_recording before the fixtures. Run uv run --locked pytest tests/recording five times in a row green before the merge and report the five durations; a single different pattern is a flaky comparison to fix in calls.py, never a widened fixture.

Commands that must pass, from the repository root unless noted:

  • make -C plugins/hermes-adapter record (once, to write the fixtures), then cd plugins/hermes-adapter && uv run --locked pytest
  • cd plugins/hermes-adapter && uv run --locked mypy and uv run --locked ruff check . and uv run --locked ruff format --check .
  • cd tools && uv run --locked --quiet pytest ci/tests/test_components.py docs/tests/test_config_values.py (the second file matters when the coverage key changed here)
  • make fmt-check, make lint, make typecheck, make test, make coverage, make licenses, make osv, make gitleaks, make docs, make linkcheck
  • make check

Acceptance criteria

  • plugins/hermes-adapter/tests/hermes_pin.toml pins a public release of Hermes Agent by tag and full commit, older than deps.min_release_age, with a Renovate rule that asks for a new recording.
  • The live recording runs in the component's default test run: the pinned commit is checked out read-only in a temporary directory, installed from an export into a throwaway environment, started with the probe plugin through the real plugin discovery and a fake model, and after the run the checkout and the installed files are unchanged and the temporary root is removed.
  • No run can reach a real model or credential: environment allowlist, throwaway homes, null keyring and store shims, egress recorder, sandbox on macOS; the Hermes process opens no listener; tests show each part.
  • Each of the six points of 09 17.10 has observations and a computed verdict in observations.json, a fixture per scenario, and a row in 09 17.10 whose result matches the verdict; no "to be checked" remains in 09 17.1 to 17.9; every claim that did not hold is adapted in 09 (and 10 10.1) as Design note 9 prescribes, without a change to docs/api/ or the core contract.
  • Fixtures hold no credential value and no third-party prose, are white label and canonical; a change of pin, probe, scenarios or harness without recording again fails the offline tests, and a change of chapter 09 runs them in CI.
  • plugins/hermes-adapter meets the component contract (pyproject with uv lock, ruff with D, mypy strict, pytest with fail_under under quality.coverage.python_min_percent), has VERSION and CHANGELOG.md, and passes the license and OSV gates without a change to the allowlist.
  • RK-12, RK-13 and, where credential_type requires it, T-AI-08 are updated; both READMEs describe the recording; the rulings are in docs/DECISIONS.md.
  • make check is green.

Definition of done

  • Code and tests as specified; doc comment on every public symbol (purpose, parameters, return value, errors, example where useful); comments explain why.
  • READMEs of plugins/ (row hermes-adapter/), plugins/hermes-adapter and plugins/hermes-adapter/tests/fixtures/recording written or updated (layout, the pinned version by reference to the pin file, how to record again, what the harness guarantees and what it does not, what M2-33 reuses).
  • The changes to 09, 10 10.1, RISKS.md and the threat model of Design notes 9 and 10 made in the same change; rulings of note 12 recorded in docs/DECISIONS.md section 10 (one row each with reason).
  • plugins/hermes-adapter/CHANGELOG.md and root CHANGELOG.md entries under 'Unreleased'.
  • An ADR only if you take an architecture decision beyond this brief (MADR 4 as in docs/adr/0000-template-madr-4.md, headings and text in English, next free number, make adr-index).
  • make check green.
  • An independent code review over the diff, then the red team gate (docs/quality/red-team-gate.md, verdict JA or NEIN); on NEIN fix with a test per finding and have the delta checked again. Record both verdicts. Name the isolation tests and test_recording[credential_both] in the red team request (PLAN 8 "Red team": faking the reported mode).
  • Land the change as AGENTS.md ("How work lands on main") describes: a pull request from this branch, squash-merged only when review, red team and make check, ci-ok and the review threads are green; then git worktree remove.

Report

Write to {{REPORT_PATH}}:

  1. First line exactly Verdict: DONE or Verdict: BLOCKED <reason>.
  2. Commits (SHA and subject), changed files with line counts (the fixture directory as one line with its file count), test results with counts (the red run before the change and the green run after, and the five durations of the repeated recording), the summary of make check.
  3. Review and red team verdicts and the findings fixed.
  4. What is not proven, at least: on Linux the absence of a network sandbox (the guarantee rests on unreachable credentials), and every point whose evidence rests on an internal Hermes function.
  5. The pinned tag, commit and date; per point its verdict, the answer in one sentence and the sections changed; the egress hosts seen; owner decisions with recommendation (only those Design note 9 names); follow-ups that could not be done in this scope, each with its reason; spec gaps found, among them the default bothy of hermes_adapter.platform_key outside the list of 02 6 (for M2-33); and whether the root Makefile fixes and the coverage key were made here or came with M2-23.

Then write {{DONE_PATH}} with exit=0 sha=<sha> or exit=1 <reason>.