M2-48: Conformance and chaos suite of the terminal¶
Status: ready | Depends on: M1-72, M1-73, M2-24, M2-46, M2-47 | Milestone: M2
Goal¶
After this ticket the tests of 24 "Tests" run data driven against the real parts of the terminal, and they keep running in every make check. A catalogue services/server/test/terminal/catalogue.yaml names, for every ID TM-T01 to TM-T29 and for AZ-M54 to AZ-M61, ID-T28 and SEC-T05, where it runs and which ticket implemented it. The end-to-end suite under services/server/test/terminal starts the installation Compose file infra/compose/compose.yaml unchanged, with M1-73's smoke overlay and the new test overlay infra/compose/test/compose.terminal.yaml (for the two-node profile also M2-47's overlay and the test overlay infra/compose/test/compose.terminal-twonode.yaml): the server image built from the worktree with services/server/Dockerfile for the platform of the Docker host, PostgreSQL, the worker, the collector with its file export and a test-only TLS front for the instance origin; real host services bothy-host in Linux containers with systemd, installed with bothy-host install, paired with bothy-host pair and confirmed through confirmHost; and a Go test device that enrols with device keys and DPoP, confirms step-ups with its device key, opens and attaches sessions and speaks Noise through the host service's own internal/noise and internal/frame. TM-T15 runs on M2-47's two-node overlay behind its example edge; TM-T16 runs once per make check and again under load in make perf (services/server/test/chaos). The host-local platform suite of M2-42 and M2-45 (services/host/test/platform: the doctor matrix of exit criterion 8, the PTY and process tree tests, TM-T13 and the platform-only parts of TM-T22, TM-T24, TM-T28 and TM-T29) runs in a Linux container inside every make check and on disposable macOS, Windows 10 and Windows 11 machines that the launcher names; the report lists every platform run. A docs gate keeps chapter 24 "Tests", the catalogue, exit criterion 8 and threat model section 6.7 together, so every terminal threat names a test that exists. Why: M2-38 to M2-47 proved the terminal module by module with fakes; this suite proves the whole path at once (device, TLS edge, relay, host service, PTY, rights, revocation, limits, two nodes, restarts) and is the evidence that M2 exit criterion 8, the terminal points of the M2 red team and RK-23 ask for. It adds no product behaviour; a non-conformance it finds is fixed in the owning module (Design note 13).
Read first¶
docs/PLAN.md: section 1 "Principles"; section 3 "Way of working per ticket"; section 4 "Closing a milestone" (points 2, 4 and 5); section 5 "Test strategy" (rows "Test doubles", "End to end", "Load, endurance, chaos", "Security"); section 8 "M2 Agent platform, extensions and terminal": "Goal", "Scope" (terminal part), "Out of scope", "Exit criteria" 8 and 10, "Test strategy" (terminal sentences), "Red team" (terminal paragraph), "Dependencies", "Risks" (terminal sentences); block "terminal" (normative sources and order) with the rows M2-38 to M2-47 (what exists), M2-48 (this ticket, the contract) and the rows that depend on it: M7-23 (release chain of the host service), M7-27 (demo instance with a host service in a container, P-26) and V2-07 (terminal in a dev environment); the client rows M4-35, M4-36, M5-12 and M6-18 (their parts stay out of scope); the rows M1-30, M1-37, M1-69, M1-71, M1-72, M1-73 and M2-24 (dependencies or implied, see the next item); section 19: P-06, P-07, P-09, P-10, P-11, P-20, P-25, P-26.- Dependencies: the header names M1-72, M1-73, M2-24, M2-46 and M2-47 (D-W-107). Through them this ticket builds on M2-38 to M2-45 and on the M1 tickets M1-01 to M1-29, M1-31 to M1-36, M1-44 to M1-46, M1-48 to M1-57 and M1-60 to M1-68 (M2-43 depends on M1-67, D-W-105; all implied, so not in the header). D-W-102 struck no dependency from this ticket's own plan column (it always named M2-46 and M2-47); it struck from columns in the closure: M2-44 from M2-46, M2-42 from M2-45, M2-40 from M2-44, M2-39 from M2-43, M1-44 from M2-41, M1-35 from M2-40, M1-29 and M1-32 from M2-39. All of them stay in this read list. M1-73 brings M1-37, M1-69 and M1-71 into the closure; M1-30 left the header because M2-46 depends on it (D-W-108 point 9), so it is implied.
docs/spec/24-terminal.md(English, normative), all of it, in particular: 2.2 "Host servicebothy-host" (users, local channel, "One session user per person and host" points 1 to 6, the PTY helper, the table HD1 to HD11, "Check of the entries", the rule for names, "Message on a refused start"); 2.3 "Server host"; 3.1 to 3.4; 4.1 to 4.3; 5.1 to 5.5; 6.1 to 6.6; 7.2 to 7.4; 8.1 to 8.4; 9 "Multiple nodes"; 10.1 to 10.4; 11.1; 13 "Errors"; 14 "Tests" (every row and the sentence after the table); 15; 16 (R-TM-01 to R-TM-30); 17 "Configuration values of this chapter".docs/spec/07-permissions.md: 2.12 "Hosts and theterminalcapability", SR10, SR11, SR12, 11.1 "Test matrix" rows AZ-M54 to AZ-M61 (AZ-M60 as M2-39 corrects it: a host without holder).docs/spec/08-identity.md: 3.2 "Enrollment code", 4 "Initial installation", 6.2, 6.3 "Fresh step-up for the terminal", 7.3 "DPoP", row ID-T28.docs/spec/05-actions.md(core contract, read only): 4.14 rowshost.read,host.manage,host.pair,host.connect,terminal.open,terminal.attach,terminal.stream,terminal.read_own,terminal.close,terminal.terminate; section 6 rowshost.updated,terminal_session.changed,host.presence_changed.docs/spec/03-data-model.md(core contract, read only):hosts,host_session_users,terminal_sessions,grants(targethost),audit_events,server_signing_keys,sequence_counters.docs/spec/04-sync-protocol.md6.4 "Close codes".docs/spec/16-operations.md: 4.1 "Basic rules", 4.3 "Edge and identity", 4.4 "Terminal: forwarding and host service", 5.3 "Secrets in the Compose stack";docs/spec/19-security.md: SEC-TERM-1 to SEC-TERM-13, SEC-TLS-1, SEC-CORS-3, SEC-LOG-2, SEC-T05, SEC-T10, SEC-T18;docs/spec/02-architecture.md: 6 "Code layout" (thetest/line, the blockservices/host/, the componenttestsupport, "Places that carry the product name"), 11.2 "Log tail per api process", 15.1 "Start", 15.3 "Shutdown", 16 "Scaling path";docs/spec/23-quality.md1 and 3;docs/spec/20-privacy.md(retention classsecurityofterminal_sessions).docs/api/openapi.yaml: operationscreateHost,getHost,listHosts,updateHost,setHostState,issueHostPairingCode,pairHost,confirmHost,setHostLlmCredentials,openTerminalSession,attachTerminalSession,listTerminalSessions,getTerminalSession,closeTerminalSession,createUser,createEnrollmentCode,redeemEnrollmentCode,createStepUpChallenge,completeStepUpWithDeviceKey,logout,disableUser,revokeDevice,revokeSession,addOwner,revokeOwner,createGroup,addGroupMember,removeGroupMember,createGrant,revokeGrant,createAgent,createToken,installApp,getDiscovery; schemasHost(withdoctor,session_users,reported_session_users,connected),HostDoctorCheck,HostConfirm,HostUpdate,HostStateSet,HostLlmCredentialsSet,HostPairRequest,TerminalSession,TerminalSessionOpen,TerminalSessionGrant,TerminalCloseReason,TerminalCloseDetail,TerminalStreamHello,TerminalStreamIdentify,NoiseFrame,AppInstall,GrantCreate,Problem(x-problem-codes).docs/api/asyncapi.yaml: channelsterminalStream,hostControlandhostDatawith theirx-close-codes; messagesterminal.hello,terminal.identify,terminal.error,noise.frame,host.report,host.session.open,host.session.close,host.session.started,host.session.ended.- ADRs:
docs/adr/0068-terminal-through-a-host-service-with-noise-stream-and-invariant-i6.md("Confirmation"),0070-terminal-in-the-web-console.md,0067-jose-profile-with-the-standard-library.md,0063-performance-gate-for-the-event-log.md,0038-local-gates-first-one-command-identical-to-ci.md,0012-oci-images-compose-opentofu.md,0046-multi-user-dev-environment-only-with-isolation.md. docs/DECISIONS.md: the rulings T1 to T13 (D-T1 to D-T11 in section 7.1, T12 as D-W-57 and T13 as D-W-62 in section 9), D-T12 to D-T42 where 24 cites them, D-W-63 to D-W-91 and D-W-109 (through 24), D-N05, D-W-100, D-W-101, D-W-102, D-W-105; section 10 for the format of new rows and the rowsD-M2-38-<n>toD-M2-47-<n>of the merged terminal tickets. D-N05, D-W-57 and D-W-101 name this ticket.docs/security/threat-model.md: 5.16 "Terminal", 6.7 "Terminal" (T-TM-01 to T-TM-14 and the row T-TM-15 that M2-47 adds), 7 "What remains open", 8 "Link to tests", 9 "Maintenance rule".docs/RISKS.md: RK-22 to RK-30, RK-84, RK-88 to RK-90 (RK-23 names this ticket).docs/OPEN-QUESTIONS.md: OQ-03 and OQ-04 (their defaults apply; M2-45 and M2-46 built them) and OQ-24 (terminal only over a private network; default no).docs/observability/signals.md4.3a "Terminal" (bothy.terminal.relay.unpairedwithbothy.terminal.side,bothy.terminal.sessions.closed);docs/operations/configuration.md: the rowsterminal.*,host_service.*,auth.clock_leeway,authz.expiry_sweep_interval,ratelimit.rl.terminal.*,ratelimit.rl.host.*,ratelimit.rl.public_auth.*,server.public_url,server.listen_addr,server.trusted_proxies,cluster.mode, and section 9 "Compose variables" (BOTHY_PUBLIC_URL,BOTHY_TRUSTED_PROXIES,BOTHY_NGINX_VERSIONof M2-47);docs/operations/tailscale.md3 "Tailnet policy (required)".docs/quality/gates.md,docs/quality/toolchain.md,docs/adr/0039-only-github-hosted-runners-public-from-the-first-push.md(the GitHub-hosted runners of CI, D-W-92),docs/quality/red-team-gate.md,docs/quality/push-discipline.md.- Terminal briefs in
docs/briefs/(names as they use them):M2-38-noise-core.md(internal/noisewithInitiatorConfig,NewInitiator,WriteMessage1,ReadMessage2,EphemeralKeyHash,Transport;internal/frame; the hand-maintainedservices/host/.go-arch-lint.yml;TestVectorsTMT01,TestRecordingsTMT02Dart,TestRecordingsTMT02Negative);M2-39-terminal-hosts.md(host operations,host.updated, the corrected AZ-M60,TestAZM60_InstanceGrantGivesNoHost,TestUpdateHost_SessionUserTaken_TMT28b,TestUpdateHost_NameNotReported_TMT28e);M2-40-terminal-authz.md(decisions, reasons and their problem mapping,TestTMT08_SR10,TestTMT09_I6,TestTMT28a_NoSessionUser,TestIDT28_*,TestAZ_M54_*toTestAZ_M61_*,TestSubjectGate_TerminalActions);M2-41-host-pairing.md(pairHostanswers401 unauthenticatedto every failed redemption,409 fingerprint_mismatch,TestTMT19_FingerprintMaxAttempts,TestTMT20_PairingCodeRejections,TestTMT20_WrongChecksumNoDatabase);M2-42-host-install-doctor.md(commands, flags and exit codes with78for a refused start, the configuration file/etc/bothy-host/config.yaml, thedoctorcheck nameshd1_usertohd11_shared_locationsand codes, Fake-claudeinservices/host/testdata/fakeclaude,TestHostPairAgainstServer, the platform suiteservices/host/test/platformwithmake -C services/host platform-test, its result file and itsTestPlatform_*tests);M2-43-host-control.md(internal/control,internal/control/controltest, the control test clientinternal/terminal/adapters/hosttest,TestDuplicate_TMT23,TestLiveSessions_HostRestarted_TMT16);M2-44-terminal-sessions-relay.md(sessions, session token, relay, the close codes of its Design notes 7 to 9, the event-log tail as the one end path, the Fake-Hostservices/server/test/terminal/fakehost,TestOpen_TMT07_StepUp,TestAttach_TMT12,TestOpen_TMT29g_WorkdirIndexOutsideOwnList,TestTerminalOps_NonHumanTokens);M2-45-host-pty-profiles.md(the check order of the token,4101and4104on the data connection, scrollback replay, process trees,TestTMT03Replay,TestTMT04TokenFailures,TestTMT05NoStart,TestPlatform_TMT21_Sessions,TestPlatform_TMT28c,TestPlatform_TMT28d,TestPlatform_TMT29*,TestPlatform_HelperChannelBreak);M2-46-terminal-revocation-limits.md(revocation, limits, audit, metrics,services/server/test/terminal/*_test.gowithTestTMT06_NoContentAnywhere,TestTMT10_RevocationKinds,TestTMT11_*, the Fake-Host with Noise and a mode that never reports idle);M2-47-terminal-multi-node.md(infra/compose/compose.multi-node.yaml,infra/compose/edge/nginx.conf,BOTHY_NGINX_VERSION,services/server/test/terminal/multinode,TestTMT15_TwoNodesWithHashRouting,TestTMT15_WithoutHashRouting). - Other briefs:
docs/briefs/M1-71-contract-tests.md(shared harnessservices/server/test/harness:ServerwithPublicURL,SecretsDir,OwnerandAdminDB,ClientwithDo, DPoP and nonce retry,StepUp,NewUser,NewAgent; "extend it only additively"),M1-73-compose-smoke.md(infra/compose/smoke.sh, overlayinfra/compose/test/compose.smoke.yamlwithSMOKE_SERVER_IMAGEandpull_policy: never,infra/compose/test/smoke.env, collector overlayinfra/compose/test/otel/file.yaml,SMOKE_*variables, the bootstrap path withbothy setup bootstrap-owner, Docker Engine and Compose minimum versions),M1-72-performance-gate.md(services/server/test/chaos, the one build tag of perf and chaos code, theperfrecipe, durable databases for chaos tests),M1-69-canary-test.md(services/server/test/canary:Registry,Class,Sinks,Scan,AllowRules),M1-23-server-image.md(services/server/Dockerfile, build argumentsVERSIONandREVISION, theimagetarget),M1-32-signing-keys.md(key storesecrets.signing_key_dirwith one PEM file<purpose>-<kid>.pemper key; the image prepares that directory for the named volume),M1-37-identity-bootstrap.md(enrollment code purposesfirst_loginandrecovery),M1-30-authz-expiry.md,M1-45-identity-revocation.md,M2-24-apps-install.md,M2-31-agent-conformance-suite.md(the sibling data-driven suite: scenario format, red-run table). - Code on the base branch:
services/server/test/harness,services/server/test/canary,services/server/test/chaos,services/server/test/terminal(withfakehost,multinode,hostpair_test.goand M2-46's tests),services/server/internal/terminal(withadapters/wsrelayandadapters/hosttest),services/server/api/operations.yaml,services/server/Dockerfile;services/host(cmd/bothy-host,internal/noise,internal/frame,internal/controlandcontroltest,internal/session,internal/pty,internal/profiles,internal/doctor,testdata/fakeclaude,test/platform,Makefile,.go-arch-lint.yml,README.md);infra/compose(compose.yaml,.env.example,smoke.sh,test/,compose.multi-node.yaml,edge/); rootMakefile(targetstest,docs,perf,trivy);tools/ci/components.py;.github/workflows/ci.yml(jobgates).
Fixed requirements that apply¶
Owner's fixed build rules (apply verbatim):
- No MVP. New from scratch and directly in the final state; stages are only the order.
- Strictly modular: clear module and service boundaries, dependencies only through defined interfaces, no cross access.
- Documentation in code: every public module, class, function and interface with a docstring/doc comment (purpose, parameters, return value, errors, example where useful); comments explain the why, not the what. API documentation is generated from code (OpenAPI from the routes, dartdoc, Python API docs in MkDocs).
- Documentation in the repo: README per module/service, ADR for every architecture decision, CHANGELOG per release, keep the documentation site current. A change without matching documentation is not done.
- Quality gates (SonarQube, linters, type checking, security scanners, test coverage) block the merge.
Rules for every brief:
- Test first: write the failing test, run it and keep the red output for the report, then make it green, then clean up.
- No en dash and no em dash in any file you write; the
text-rulespre-commit hook rejects them. Use a hyphen or rephrase. - White label: examples and test data use
example.comand the people Alex, Sam and Robin; no company, host or personal names, no paths under a home directory. Host names and session user names in tests are neutral (r1,r2,r3,r5,term-alex,term-sam,term-robin,term-owner), the instance origin ishttps://chat.example.com:8443, and the product name appears only at the places of 02 "Code layout" ("Places that carry the product name"); suite names start withtermsuite. - Limits, windows and thresholds only through named configuration keys listed in
docs/operations/configuration.md; code comments and docs name the key, never the number. The short windows of the test stack live only inservices/server/test/terminal/profiles.yaml, keyed by those names (Design note 4); bounds that only limit how long a test polls are test constants, not product limits. This ticket adds no configuration key. - Specification first: a change to behaviour, API, event, right or configuration updates
docs/spec/ordocs/api/in the same change. The core contract (docs/spec/00-glossary.md,03-data-model.md,04-sync-protocol.md,05-actions.md) is not edited by this ticket.
Product requirements implemented here:
- Every ID of the PLAN row runs green in
make check: TM-T01 to TM-T13, TM-T15 to TM-T17, TM-T19 to TM-T24, TM-T28 and TM-T29, together with AZ-M54 to AZ-M61 and ID-T28 without the console part (M2 exit criterion 8); TM-T16 runs in addition inmake perfas the chaos test (PLAN row, PLAN 5 row "Load, endurance, chaos", P-07; Design note 10). - The tests run data driven against the real server, real host services in Linux containers and a Go test device; every expectation is written literally from 24 14 and the chapters it names, never computed from server or host code (PLAN row, PLAN 8 "Test strategy").
- The Go test device runs the handshake and the frames with the Noise package of the host service (PLAN 8 "Test strategy"; Design note 6).
- The PTY tests and the
doctormatrix of exit criterion 8 run also on macOS, Windows 10 and Windows 11 on own machines, never on a hosted runner; the report lists every platform run (PLAN row, exit criterion 8; Design note 11). - The stack is the installation Compose file with test overlays and the server image built from the worktree with its Dockerfile; the installation file stays unchanged (P-06; Design note 3).
- Every scenario was red before its implementation, proven in the reports of the implementing tickets; this report names them (PLAN row; Design note 15).
- Threat model 6.7 names a test for every threat, and a gate keeps it so (PLAN row, exit criterion 8, threat model 8; Design note 12).
- A missing Docker runtime or
docker buildx, a Docker Engine or Compose below the minimum versions of M1-73, a Docker host without cgroup version 2 or withkernel.yama.ptrace_scopebelow1(M2-42's HD10 codeptrace_scope_low) fails the suite with a message naming the fix; nothing is skipped (P-09, R-QA-02). - Tests bind only
127.0.0.1and port 0 on the machine that runs them; inside the Compose project nothing is published except the TLS front on127.0.0.1with a random host port (P-10, 16 4.1). - No terminal content, session token, access token, device key, host key, pairing code or enrollment code appears in any log line, test failure message, catalogue, scenario file or report (SEC-TERM-1, R-TM-10, SEC-LOG-2).
Terminal security requirements (D-W-101):
- Rights per machine: the additional right
terminalon the targethost, SR10, SR11 with I6 and SR12 (07 "Hosts and theterminalcapability"). M2-39 and M2-40 built them as decisions ofauthorize, M2-44 applies them at opening and attaching, M2-45 repeatsownand the binding ofosuon the machine. This ticket adds no code to them and proves them end to end over HTTP, the device stream and a real host service: TM-T08 (SR10 including the server host,scenarios/tm-t08.yaml), TM-T09 (I6), TM-T28 (a), (b), (c) and (e) (SR12 and the binding ofosuin the host service), TM-T17 (agent and app tokens), AZ-M54 to AZ-M61 in one scenario file (scenarios/az-m54-61.yaml), andTestSessionRunsAsSessionUser(every shell runs as the session user of its person). - Fresh step-up: device key in the apps, only passkey in the console (08 6.3, ID-T28). This ticket proves the device-key path end to end: TM-T07 (no step-up, too old, consumed), TM-T12 (another device, the same device inside and after
terminal.quick_reattach_window) and ID-T28 without the console part (scenarios/id-t28.yaml: older thanterminal.step_up_max_age, used for an earlier opening, two parallel openings with one confirmation give exactly one session). The step-up with password and TOTP of ID-T28 stays proven by M2-40'sTestIDT28_PasswordTotpAndOidcStepUp(catalogue run of kindunit); the console parts (passkey only,reasonpasskey_required) belong to M6-18, the step-up keys of the apps to M4-36 and M5-12. - Network: there is no public inbound path to the terminal; the host service opens only outbound connections (R-TM-01); an installation with the profile
tailscalereaches the server only over the tailnet with a policy only for port 443 and without Funnel (D-OP-03,docs/operations/tailscale.md); the product does not require Tailscale; whether the terminal must be reachable only over a private network is owner question OQ-24 (default no). This ticket enforces it in the test stack and proves it: the host containers sit on their own internal network that reaches only the TLS front and publish no port, the only published port is the TLS front's on127.0.0.1(TestComposeTerminalOverlay), and a running host service and its helpers hold no TCP or UDP socket on a non-loopback address (TestHostServiceListensOnNoNetworkPort; M2-42'sTestPlatform_NoNetworkListenerper platform); the whole suite runs with the profiledirectand without any Tailscale component. M2-43 and M2-44 own the connections; an answer "yes" to OQ-24 changes them, and this suite follows. - Audit without content: M2-46 owns audit and telemetry without content, M2-44 the relay that records only length, time and session ID. This ticket proves it end to end with TM-T06 (canary in the relay view, the container logs, a database dump, the collector export and the journal of the host service) and with the audit checks of TM-T07 to TM-T10, TM-T17, TM-T23 and TM-T28, which assert action, decision and the codes in
details, never a value. - Idle timeout and maximum duration (24 "Limits"): M2-45 measures idle on the host, M2-46 adds the server side and
terminal.sweep. This ticket proves both with TM-T11 in the stack profilelimitsagainst a real host service, including the manipulated host service that never reports idle (M2-44's Fake-Host connected through the TLS front, Design note 9). - Service user and session users without administrator rights (T12, HD rules): M2-42 built
install,doctorand the platform suite, M2-45 the PTY in the helper of the session user. This ticket runs that suite (TestPlatform_TMT13,TestPlatform_TMT22,TestPlatform_TMT24,TestPlatform_TMT26,TestPlatform_TMT29_Parts,TestPlatform_PTYRunsAsSessionUserand the rest) in a Linux container in everymake checkand on macOS, Windows 10 and Windows 11, which together give thedoctormatrix of exit criterion 8 (HD1 to HD3, HD9, HD11 at a location for all, and HD11 afterinstallwith confirmation on Windows 10 and 11); every end-to-end scenario starts from a host that passeddoctorafterbothy-host install(TestBaselineDoctorPasses), and no shell of a session runs asrootor as the service user (TestSessionRunsAsSessionUser). - Threats of threat model section 6.7 that concern this ticket, each with its test: T-TM-01 (TM-T06 end to end; TM-T01 and TM-T02 by M2-38), T-TM-02 (TM-T03 to TM-T05), T-TM-03 (TM-T07, TM-T08, AZ-M54, AZ-M55, AZ-M58, AZ-M60), T-TM-04 (TM-T13 in the platform suite, TM-T24), T-TM-05 (TM-T22, TM-T23; TM-T18 by M4-35), T-TM-06 (TM-T19, TM-T20), T-TM-08 (TM-T09, TM-T13, TM-T28, TM-T29, AZ-M56, AZ-M57, AZ-M61; TM-T27 in the platform suite), T-TM-09 (TM-T17, AZ-M59), T-TM-11 (TM-T10, TM-T11, TM-T16, TM-T28), T-TM-12 (TM-T22, TM-T28; TM-T26 in the platform suite), T-TM-13 (TM-T24, TM-T29), T-TM-14 (TM-T21, TM-T24, TM-T29 (i) on Windows; TM-T26 in the platform suite), T-TM-15 (TM-T15 end to end). Not touched here: T-TM-07 (M6-18) and T-TM-10 (TM-T14, M4-35). The docs gate of Design note 12 checks this mapping against 6.7.
Scope¶
Paths are relative to the repository root. Names of M2-24 and M2-38 to M2-47 below are the ones their briefs give; where the merged code names them differently, follow the code (Boundaries).
Create:
services/server/test/terminal/catalogue.yaml,profiles.yaml,scenarios/<id>.yaml(one file per scenario of Design note 9:tm-t03totm-t12,tm-t15,tm-t16,tm-t17,tm-t19totm-t24,tm-t28,tm-t29,az-m54-61,id-t28), andREADME.md(English) if the folder has none, else extend it.services/server/test/terminal/suite/(packagesuite, componenttestsupport):doc.go,catalogue.go,scenario.go,profiles.go,conductor.go,world.go,expect.go,checks.go, with unit tests (Design notes 4, 7 and 8).services/server/test/terminal/stack/(packagestack): Compose project lifecycle, preflight, the server image, test CA, host binaries, host containers, rendering of the TLS front configuration, with unit tests (Design notes 3 and 5).services/server/test/terminal/device/(packagedevice): the Go test device and its sidecar driver, with unit tests (Design note 6).services/server/test/terminal/e2e/conformance/,e2e/hostpolicy/,e2e/limits/,e2e/twonode/,e2e/restart/,e2e/hostplatform/: one test package each withmain_test.go(TestMainwith one stack) andscenarios_test.go(Design notes 2 and 10).services/server/test/chaos/terminal_test.gowith the build tag of M1-72:TestChaosTMT16(Design note 10).services/host/test/noisedevice/(main.go,protocol.go, tests): the Noise sidecar of the test device (Design note 6).infra/compose/test/compose.terminal.yaml(test only),infra/compose/test/compose.terminal-twonode.yaml(test only, profiletwonode),infra/compose/test/host/Dockerfile(test only),infra/compose/test/terminal/nginx.conf.tmplandrouting-single.conf,routing-hash.conf,routing-split.conf(the TLS front, Design note 3).tools/docs/terminal_catalogue.pywithtools/docs/tests/test_terminal_catalogue.py(Design note 12);tools/ci/tests/test_host_platform_bundle.py(Design note 11).
Change:
services/host/Makefile: targetplatform-bundle, andplatform-testbuilt on it (Design note 11).services/host/test/platform(M2-42, M2-45): the marker guard and, only if absent, the process tree tests of Design note 11.services/host/.go-arch-lint.yml:test/noisedevicein the component oftest/**(Design note 6).services/server/test/harness: only additively,Externaland the stream credentials of Design note 6 if missing.services/server/test/canary: only additively, a way to add an external text source toSinks(for example(*Sinks).AddSource(name string, r io.Reader) error) if the mergedSinkscannot take one (Design note 9, TM-T06).services/server/test/terminal/fakehostandservices/server/internal/terminal/adapters/hosttest: only additively, pairing throughpairHostand dialing an external URL with a given TLS configuration and dialer if missing (Design note 9, TM-T11).- Root
Makefile: one line in the recipe ofdocsthat runstools/docs/terminal_catalogue.py(Design note 12). tools/ci/components.pyandtools/ci/tests/test_components.py(Design note 14).- Code of M2-38 to M2-47 only where the suite finds a non-conformance with the specification (Design note 13).
Docs (English, like the files; Design note 16):
docs/spec/24-terminal.md14 (paragraph after the table),docs/spec/02-architecture.md6, ADR 0068 "Confirmation" (thenmake adr-index),docs/security/threat-model.md8,docs/quality/gates.md,docs/quality/toolchain.md,infra/compose/test/README.md,infra/compose/README.md.services/server/README.md,services/host/README.md,services/server/CHANGELOG.md,services/host/CHANGELOG.mdunder "Unreleased",docs/DECISIONS.mdsection 10 rowsD-M2-48-<n>(Design note 17).
This ticket creates no migration, no API operation, no event, no action, no configuration key and no new top-level make target; platform-bundle lives in services/host/Makefile.
Out of scope¶
- The client views: TM-T14 and TM-T18 (M4-35), the console parts of TM-T07, TM-T12 and ID-T28 and the threat T-TM-07 (M6-18), the step-up keys of the apps (M4-36, M5-12). The catalogue records them with their ticket under
deferred(TM-T14, TM-T18) ordeferred_parts(the console parts). - TM-T25 (M2-45), TM-T26 and TM-T27 (M2-42) are not in exit criterion 8: the catalogue points to their tests, and the platform runs of this ticket execute them because they belong to the platform suite.
- New behaviour of the server or the host service; fixes only as Design note 13 allows.
- Signatures, packages and the release chain of
bothy-host(M7-23); the demo instance (M7-27, which may reuse the host image of Design note 5); terminal in dev environments (V2-07); the multi-node operation beyond the terminal relay (M7-17); performance targets of other tickets. - M2-46's integration tests in
services/server/test/terminal/*_test.goand M2-47'sservices/server/test/terminal/multinode: they stay as they are (Design note 2). - Answers to OQ-03, OQ-04 and OQ-24 (their defaults apply).
- Changes to the core contract.
Boundaries¶
- Work in a dedicated git worktree on the branch
feat/M2-48-terminal-conformance; one writer per worktree. - Conventional Commits with the component as scope (for example
test(terminal): ...,test(host): ...,test(chaos): ...,build(host): ...,docs(spec): ...,ci: ...) and a DCO sign-off on every commit (git commit -s); squash into meaningful commits before the merge (docs/quality/push-discipline.md). - Never push to
main, never tag and never create remote repositories; the only remote steps are the branch push and the pull request ofAGENTS.md. - Do not modify files outside the scope above, except the CHANGELOG, README and
docs/DECISIONS.mdentries named in this brief. - Follow the host rules supplied by the launcher (ports, machine rules); they also name the disposable macOS, Windows 10 and Windows 11 machines with administrator rights for the platform runs.
- Tests bind only
127.0.0.1and port 0; the one published port of a stack is the TLS front's on127.0.0.1with a random host port; PostgreSQL runs only inside the throwaway Compose project or as apgtestcontainer on a random port (P-09, P-10). Without a Docker-compatible runtime the suite fails; it never skips. - Stop only processes you started, by PID. Temporary directories only via
mktemp -d(Got.TempDir()), removed at the end. Containers, networks and volumes only of the suite's own Compose projects (name prefixtermsuite-), removed at the end of each test binary withcompose down --volumes --remove-orphans, after which the test asserts that nothing of the project remains; the server image and the host image of Design notes 3 and 5 stay as build cache under their content-addressed local tags (termsuite-server:<hex>,termsuite-host:<hex>), andinfra/compose/test/README.mdsays how to remove them. - Never read or print secrets. Keys, tokens, pairing and enrollment codes created inside a throwaway stack are test fixtures: they stay in memory or in files of mode
0600in the temporary directory, are never printed and are deleted with it; the one read of the stack's own signing key store for TM-T04 (Design note 9) is such a fixture. No network calls except package downloads for pinned tools and Go modules, images pinned by digest (among them the stages ofservices/server/Dockerfile) or by M2-47's version variable, and the pinned Debian snapshot of Design note 5. - No real coding agent, no model call, no LLM account: only M2-42's Fake-
claudeand the marker programs of Design note 5. Nothing in this ticket costs money. privilegedonly for the host test containers ofinfra/compose/test/compose.terminal.yaml, never in an installation file. The suite never changes a kernel setting of the Docker host.- Platform runs only on machines the launcher's host rules name as disposable test machines for the host service; the build session creates the marker file of Design note 11 there and nowhere else;
bothy-host install, users, groups, services and the ACL change of TM-T29 (i) never touch any other machine. - Never lower a threshold, disable a gate, delete a test or raise a timeout (
go test,PERF_TIMEOUT,timeout-minutesin.github/workflows/ci.yml); rate limits and windows change only in the test profiles. - If the core contract lacks something this ticket needs, stop with
Verdict: BLOCKED <reason>instead of building a deviation (P-11). - This brief was written before milestone M1 closed (D-W-100). Where the merged code of an M1 ticket names a package, type, function, file or make target differently from this brief, follow the code and list each difference in the report; where it behaves differently from the specification, stop with
Verdict: BLOCKED <reason>. - The same holds for M2-24 and M2-38 to M2-47: their merged names win over the names here; list each difference. Where their behaviour differs from the specification, Design note 13 applies.
- This ticket also needs M1-30, M1-72, M1-73 and M2-24 (with M1-37, M1-69 and M1-71 in their closure); M1-72, M1-73 and M2-24 are in the header (D-W-107) and M1-30 is implied through M2-46 (D-W-108 point 9). If one of them is not merged on the base branch, stop with
Verdict: BLOCKED <ticket> not merged. - Every container keeps its real clock; asynchronous effects are awaited by polling, never by sleeping for a window (Design note 8).
Design notes¶
-
Start check. Run
make checkon the fresh worktree; if it is red, stop withVerdict: BLOCKED base branch redand the failing gate. Then confirm on the base branch: the operations of "Read first" underenabledinservices/server/api/operations.yaml(among themdisableUser,revokeDevice,revokeSessionof M1-45,createAgentof M1-51,installAppof M2-24,redeemEnrollmentCodeof M1-37, and every terminal operation of M2-39 to M2-44); the jobauthz.expiry_sweep(M1-30) andterminal.sweep(M2-46);services/server/test/harness(M1-71) andservices/server/test/canary(M1-69);services/server/Dockerfile(M1-23, with the key store directory of M1-32);infra/compose/smoke.sh,infra/compose/test/compose.smoke.yaml,infra/compose/test/smoke.envandinfra/compose/test/otel/file.yaml(M1-73);services/server/test/chaosand aperfrecipe that runs it (M1-72);services/server/test/terminal/fakehost(M2-44, with Noise after M2-46);infra/compose/compose.multi-node.yamlandinfra/compose/edge/nginx.conf(M2-47);services/host/test/platformandmake -C services/host platform-test(M2-42, M2-45). A missing piece of M2-38 to M2-47 ends withVerdict: BLOCKED dependency <ticket> missing; for M1-30, M1-72, M1-73 and M2-24 the Boundaries apply. -
Architecture: four layers, one catalogue. Ruling for docs/DECISIONS.md.
- End to end (
make check): the packages underservices/server/test/terminal/e2e/run the scenario files against one Compose stack per test package (Design note 3) with real host services in Linux containers (Design note 5) and the Go test device (Design note 6). The split is:conformance(TM-T03 to TM-T10, TM-T12, TM-T17, TM-T19, TM-T20, TM-T22, TM-T23, TM-T28, AZ-M54 to AZ-M61, ID-T28, and the tests of requirements 21, 23 and 26),hostpolicy(TM-T21, TM-T24, TM-T29),limits(TM-T11, profilelimits),twonode(TM-T15, profiletwonode),restart(TM-T16, own stack, because it restarts the server),hostplatform(the platform suite in a Linux container, Design note 11). - Chaos (
make perf):TestChaosTMT16on the same stack code (Design note 10). - Platform (disposable macOS, Windows 10 and Windows 11 machines): the platform suite of M2-42 and M2-45, host-local against M2-43's
controltest(Design note 11). - Earlier tests: TM-T01 and TM-T02 stay the vector and cross tests of M2-38 in
services/hostandpackages/bothy_noise; the server-level tests of M2-39 to M2-47 stay where they are (M2-46's phrase that this ticket "turns them into its data-driven suite" means that this suite repeats them data driven against the real host service, because no test is deleted). The catalogue names all of them as runs of kindunit, and every end-to-end handshake is an additional interop proof of the same code.
- End to end (
-
Stack. Ruling for docs/DECISIONS.md (reasons: the host service must reach the server at an address that is not its own, or it reports itself as the server host after 24 2.3; a server inside the test process would have to bind outside loopback for the containers, which P-10 forbids; the suite runs the image an operator runs, because the image carries more than the binary: M1-32 prepares the key store directory
secrets.signing_key_dirin it so that the named volumesigning-keysinherits owner and mode, and a bare base image with a mounted binary would leave that volume unwritable for the server user; the TLS front stands where the operator's edge stands in the profiledirect).- Files, in this order:
infra/compose/compose.yamlunchanged, M1-73'sinfra/compose/test/compose.smoke.yaml, for the profiletwonodeM2-47'sinfra/compose/compose.multi-node.yaml, theninfra/compose/test/compose.terminal.yaml, and for the profiletwonodelastinfra/compose/test/compose.terminal-twonode.yaml, which overrides only the servicesapi-2andedgethat exist only with M2-47's overlay (so the profilesdefaultandlimitsrender neither); every call passes--project-directory infra/composeand--env-file infra/compose/test/smoke.env, assmoke.shdoes. Project nametermsuite-<package>-<8 random hex>. Before the first Compose call, unset every exportedBOTHY_,COMPOSE_,OTEL_,SMOKE_andTERMSUITE_variable of the calling shell, assmoke.shdoes; then set, in the environment of each Compose call (it wins over the env file),COMPOSE_PROFILES=direct(so thesmokeservices of M1-73 stay inactive),BOTHY_PUBLIC_URL=https://chat.example.com:8443, theSMOKE_*variables the smoke overlay interpolates (SMOKE_SERVER_IMAGEthe image below,SMOKE_SECRETS_INPUTan empty directory of mode0700,SMOKE_RUN_ID,SMOKE_OUT,SMOKE_TMPandSMOKE_CANARYin the temporary directory, assmoke.shsets them) and the suite variables, which start withTERMSUITE_and are documented ininfra/compose/test/README.md. - Server image: in
TestMainof each e2e package,docker buildx build --load --platform linux/<arch> --file services/server/Dockerfile --build-arg VERSION=<services/server/VERSION> --build-arg REVISION=<git rev-parse HEAD> --iidfile <tmp>/server.iid .from the repository root, the arguments of M1-23'simagetarget for one platform, with<arch>fromdocker version --format '{{.Server.Arch}}'; then tag the resulttermsuite-server:<first 12 hex of the image ID>and use that tag asSMOKE_SERVER_IMAGE. An unchanged tree is a cache hit of BuildKit and takes seconds; packages that build the same tree at the same time share BuildKit's identical steps and arrive at the same tag, and the content-addressed tag keeps parallel worktrees apart, so no lock is needed. Report the wall time of the first (uncached) and of a cached build. The smoke overlay already pointssecrets-init,migrate,apiandworkeratSMOKE_SERVER_IMAGEwithpull_policy: never; the twonode overlay does the same for M2-47'sapi-2, which extendsapiof the installation file. The commands, roles, users, volumes and hardening stay those of the installation. - Server configuration: an
env_filewith the server keys of the profile (Design note 4) onapiandworker(terminal overlay) and onapi-2(twonode overlay).api(andapi-2in the twonode overlay) get inenvironmentBOTHY_SERVER__LISTEN_ADDR: ":8080"(the smoke overlay's127.0.0.1:8080would keep the TLS front out) andBOTHY_SERVER__TRUSTED_PROXIESfromTERMSUITE_TRUSTED_PROXIES, andports: !reset []; thereapi-2getsenvironment: !overridewith the same mapping as the renderedapiplus M2-47'sBOTHY_CLUSTER__MODE: multi, because YAML anchors do not cross files andextendswould bring back the installation's keys that the smoke overlay drops, and M2-47'sedgegetsports: !reset []. - TLS front (
tls-front, test only): the imagenginxinc/nginx-unprivilegedatBOTHY_NGINX_VERSIONof M2-47,userthe uid and gid of the test process (it reads the key file of mode0600),read_only,cap_drop: [ALL],no-new-privileges,tmpfson/tmp, the folderTERMSUITE_FRONT_DIRread-only with the renderednginx.conf, the certificate, the key androuting.conf; networksdefaultandtermhosts, ontermhostswith the aliaschat.example.com;portsexactly127.0.0.1::8443.nginx.conf.tmplhaslisten 8443 ssl,pidand temp paths under/tmp,access_log off,error_log /dev/stderr crit,client_max_body_size 0,proxy_http_version 1.1, the upgrade headers,Host $http_host,X-Forwarded-For $proxy_add_x_forwarded_for,X-Forwarded-Proto https,proxy_buffering off,proxy_request_buffering off, and the timeouts of M2-47's edge computed from the profile'sterminal.idle_timeoutplusterminal.idle_grace(stream paths) andterminal.host_heartbeat_intervaltimesterminal.host_heartbeat_miss_limit(all else).routing-single.confsends everything toapi:8080,routing-hash.confto M2-47'sedge:8080,routing-split.confthe device stream paths toapi:8080, the host data paths toapi-2:8080and the rest toedge:8080; the stack switches modes by replacingrouting.confin the mounted folder and runningnginx -s reloadin the container. - Networks:
termhosts(internal: true, test only) carries onlytls-frontand the host containers, so a host reaches nothing but the TLS front; the server services stay on the installation's networks.TERMSUITE_TRUSTED_PROXIESis the subnet of<project>_default, read aftercompose up --no-start(which creates the project networks) withdocker network inspect(fieldSubnet, the documented installation step of M2-47's overlay andinfra/compose/README.md), thencompose up --detach --waitof the services of the profile with the variable set (Compose recreates the containers whose configuration changed). - TLS and name: one test CA per stack (ECDSA P-256 with
crypto/x509, valid one day, key only in the temporary directory) and a server certificate forchat.example.com. The device dialschat.example.com:8443through a dialer that maps it to the published port (docker compose port tls-front 8443) and trusts the test CA, so DPoPhtu, theOriginrule andserver.public_urlstay those of the instance. - Bootstrap:
docker compose exec -T api /usr/local/bin/bothy setup bootstrap-owner --handle <h> --display-name <n>(M1-37, assmoke.shruns it), stdout to a file of mode0600in the temporary directory, never printed; the enrollment code enrols the stack's bootstrap owner throughharness.External(Design note 6). The bootstrap owner only creates the people of each case (Design note 7). - Preflight, each a failure with the fix in the message (requirement 18):
dockeronPATH;docker buildx version; Engine and Compose versions;docker inforeports cgroup version 2;/proc/sys/kernel/yama/ptrace_scoperead in a throwaway container of the host image is at least1. - Removal in
TestMainafter the scenarios; on failure firstcompose ps --alland the last lines of the logs ofapi,workerand every host container, filtered to codes, into the test output. TestComposeTerminalOverlayrendersdocker compose config --format jsonfor each profile and asserts: the first file isinfra/compose/compose.yamland no overlay of this ticket redefines an installation service beyond the keys named in this note; the only published port is127.0.0.1::8443oftls-front;privilegedonly ontermhost;termhostonly ontermhosts; for the profiletwonode,apiandapi-2have equal image, command, volumes and environment (withBOTHY_CLUSTER__MODEmultiin both); for the profilesdefaultandlimitsneitherapi-2noredgeexists; no Tailscale service. A second check in every e2e package compares the SHA-256 of every file underinfra/composebeforecompose upand aftercompose down, so the stack never writes the installation files.
- Files, in this order:
-
Profiles.
services/server/test/terminal/profiles.yamlholds per profile (default,limits,twonode) the server keys and the host-service keys with their test values and one comment each on why the value is needed. Choose values the merged configuration schema accepts; never change the schema for the suite. The loader checks the relations the scenarios rely on and fails naming both keys:terminal.session_token_ttlplusauth.clock_leewaybelowterminal.open_timeout(TM-T04 must seetoken_expiredbefore the opening times out) and at least a test constant that leaves TM-T03 time to disconnect, attach and replay before the first token expires;terminal.quick_reattach_windowbelowterminal.detach_grace;terminal.host_heartbeat_intervaltimesterminal.host_heartbeat_miss_limitbelowterminal.detach_grace(TM-T16: the server notices the lost host before the grace ends); inlimitsterminal.idle_timeoutplusterminal.idle_gracebelowterminal.max_session_duration; indefaultevery window long enough that no scenario reaches it by accident;ratelimit.rl.terminal.*,ratelimit.rl.host.*,ratelimit.rl.public_auth.*,ratelimit.rl.admin.*,ratelimit.rl.read.*,ratelimit.rl.write.*,terminal.max_sessions_per_personandterminal.max_sessions_per_hosthigh enough for the parallel cases of a package;authz.expiry_sweep_interval,terminal.sweep_interval,terminal.pairing_code_ttl,terminal.step_up_max_age,terminal.host_duplicate_probe_timeout,host_service.credential_check_intervalandhost_service.entry_scan_timeoutshort enough for TM-T10, TM-T11, TM-T20, TM-T07, TM-T23 and TM-T29.defaultalso setstelemetry.logs.leveldebugandtelemetry.traces.sampler_ratio1.0, so TM-T06 scans the most verbose output.twonoderepeatscluster.modemulti, which M2-47's overlay already sets forapi,api-2andworker; the loader checks that both agree.auth.session_max_ageis never shortened, because it would also end the session of the stack's bootstrap owner (Design note 9, TM-T10). Server keys reach the server asBOTHY_*variables through theenv_fileof Design note 3; host-service keys go into/etc/bothy-host/config.yamlof each host (M2-42). -
Host containers. Ruling for docs/DECISIONS.md.
infra/compose/test/host/Dockerfile(SPDX header, comment "test only"):FROM debian:<stable>-slim@sha256:<digest>; APT sources pinned to one timestamp ofsnapshot.debian.org(recorded in a comment;Acquire::Check-Valid-Until=false, because theReleasefiles of a snapshot expire), so the package set is reproducible;apt-get install --no-install-recommendsofsystemd,systemd-sysv,dbus,sudo,cron,procps,acl,iproute2,ca-certificates,passwd,util-linux;STOPSIGNAL SIGRTMIN+3;CMD ["/lib/systemd/systemd"]; the marker file/etc/termsuite-disposable(Design note 11); nobothy-hostinside. The image runs as root because systemd is process 1 and the host service creates real users and services (24 2.2);make trivystays green, with only the root-user finding of this file marked by Trivy's inline ignore comment and the reason next to it. Adoctorfinding caused by the image (for example a listable home of a system user) is fixed in the Dockerfile, never indoctor. The tag istermsuite-host:<first 12 hex of the SHA-256 of the Dockerfile>; the stack builds it with BuildKit when the tag is missing.- Service
termhostincompose.terminal.yamlwith a profile, soupnever starts it: the image above,pull_policy: never,privileged: true(systemd as process 1 needs a writable cgroup tree, and M2-45 creates a cgroup per session underDelegate=yes),tmpfson/run,/run/lockand/tmp, networktermhostsonly, noports. Each host of a case is one containercompose run --detach --name <project>-<case>-<host> termhost, removed at the end of the case. - Bring-up per host, following the operator steps of 16 4.4 and M2-42's README, all as root through
docker exec: copybothy-host(built once per test binary inservices/hostwithGOTOOLCHAIN=local,CGO_ENABLED=0,GOOS=linuxand theGOARCHof the Docker host; ownerroot, mode0755) and the Fake-claudebuilt fromservices/host/testdata/fakeclaudeinto folders owned byroot; add the test CA to the system trust store (update-ca-certificates), sopairand the control client trust the TLS front (M2-43's client uses the system roots);bothy-host install --session-user <first>andbothy-host session-user add <name>for the others; write the host-service keys of the profile and of the case into/etc/bothy-host/config.yaml, thenbothy-host restart;bothy-host pair --server https://chat.example.com:8443 --code <code>with the code thatcreateHostreturned to the case's Owner (or a new one fromissueHostPairingCodefor a re-pairing), passed throughdocker execarguments and never written to the container's disk or the test output; read the fingerprint from its output;confirmHostby the case's Owner with the firstterminal.fingerprint_confirm_charscharacters,llm_credentials,credential_holder_id,declared_server_host,session_usersandreasonfrom the scenario; pollgetHostuntilstateisactive,connectedis true and every assigned name is inreported_session_users.bothy-host doctormust exit0at this point (TestBaselineDoctorPasses). - Process starts that must never happen (24 14 "trace of the process starts": hooks,
apiKeyHelper, anotherclaude, a foreignnode) are proven with marker programs: each such program is a script that appends its own name to a file in/var/tmp/termsuite-markers(mode1777); the case asserts that the folder holds no file. Reason: any execution, through any path and as any user, leaves the mark. - Processes of a session are counted with
psin the container: the processes of the session user minus its PTY helper, and the background processes started with a unique marker argument (setsid sleep <marker> &,nohup sleep <marker> &).
-
Go test device. Ruling for docs/DECISIONS.md: Go forbids importing
services/host/internal/noisefrom the server module, and a requirement of the server module on the host module would pull the host module into the server's build; the device therefore keeps HTTP, DPoP and WebSocket inservices/server/test/terminal/deviceand runs Noise and frames in the sidecarservices/host/test/noisedevice, which importsinternal/noiseandinternal/frameof the host service unchanged. TM-T01 and TM-T02 keep proving that implementation against independent vectors and the Dart package.- Identity: the DPoP client of M1-71's harness (
Client,Do,StepUp,NewUser,NewAgent). If the harness only serves servers it started, addharness.External(tb testing.TB, opts ExternalOptions) *ServerwithExternalOptions{PublicURL string; HTTPClient *http.Client; OwnerEnrollmentCode string}, which enrols the owner with that code and fresh device keys, leavesSecretsDirempty and makesAdminDBfail the test when called (the suite reads the database only through Design note 8), and, if missing, a method ofClientthat returns its access token and a DPoP proof for a given method, URL and nonce (foridentify). People come fromcreateUserwithlocal_account: true(afirst_logincode, M1-37) andredeemEnrollmentCode; a second device of a person fromcreateEnrollmentCode(arecoverycode for a person with a device) andredeemEnrollmentCode(08 3.2); step-ups fromcreateStepUpChallengeandcompleteStepUpWithDeviceKey. - Sessions:
openTerminalSessionwithclient_ephemeral_key_hashfrom the sidecar; the streamstream_pathwithterminal.helloandterminal.identify(DPoP with the nonce ofhello, noOrigin); every text frame validated withapi/specschema.ValidateChannelFrame; message 1 with the session token, message 2,start, then frames. The device keeps every binary message it sent and received in memory (the relay view of TM-T06) and offers a cut of its TCP connection without a close frame (TM-T16). - Sidecar protocol, JSON lines on stdin and stdout, one process per device session, built once per test binary with
go buildinservices/host(GOTOOLCHAIN=local,CGO_ENABLED=0), as M2-42'sTestHostPairAgainstServerbuildsbothy-host:start {host_static, session_id, host_id}returns{ephemeral_public, cek};message1 {payload}returns{message};message2 {message}returns{payload};seal {frame}returns{message};open {message}returns{frame}; errors{ok: false, error: {kind}}with the names of the sentinel errors ofinternal/noiseandinternal/frame. Bytes are base64url without padding. stderr carries no bytes of keys, tokens or frames. services/host/.go-arch-lint.yml:test/noisedevicejoins the component M2-42 uses fortest/**; if there is none, add a component fortest/**that may use every component and vendor of the module and that no production component may use (02 6, the rule of the server).make archpasses.
- Identity: the DPoP client of M1-71's harness (
-
Catalogue and scenario files. Ruling for docs/DECISIONS.md.
catalogue.yaml:format: 1and one entry per ID of 24 14 (TM-T01 to TM-T29) and per ID AZ-M54 to AZ-M61, ID-T28 and SEC-T05 that threat model 6.7 names. Fields:id;titleandexpected(the English cells of the row, literally);runs(each{kind, ref, os}withkindone ofe2e(scenario file and package),platform(test function ofservices/host/test/platform;osfromlinux,macos,windows10,windows11),perf(Go test name) andunit(module path and test function of the implementing ticket));implemented_by(ticket IDs and merged test names that recorded the red run, Design note 15);deferred({ticket, reason}, only for IDs outside exit criterion 8);deferred_parts(a list of{part, ticket}for the parts of an ID that another ticket proves: the console parts of TM-T07, TM-T12 and ID-T28 with M6-18);threats(T-TM IDs of 6.7).- Scenario files
scenarios/<id>.yaml, decoded strictly with known fields only, with the YAML library the server module already uses (M1-07):id,title,sources,profile,world(people and owners, groups, agents, apps, hosts with holder,llm_credentials,credential_holder_id,declared_server_host, session users, host-service keys, profiles and working directories, grants),cases(named lists of steps). Each case gets its own instance of the world: fresh people with unique handles (display names Alex, Sam, Robin and Owner, the case's Owner made an owner withaddOwnerby the bootstrap owner) and its own host containers, so no case can disturb another; cases run witht.Parallel(), bounded by a test constant on the number of host containers starting at once. Lettered parts of a 24 14 row that build on each other are steps of one case. - Step kinds, exactly one per step:
as: <person>withcall: <operationId>andwith(method and path come fromdocs/api/openapi.yaml);device: <name>withdo:one ofenroll,step_up,open,attach,connect,handshake,start,send,expect_output,replay_message1,cut,close,wait_close;host: <name>withdo:one ofexec(as root, the operator),exec_as(a user),restart_service,reboot(docker restartof the container: the machine restarts with its file system),pause,unpause,net_down,net_up,clone_state;server:one ofrestart,front_mode,sql(read only; a write only withbypass: true, Design note 9);wait: <configuration key>(the profile value, then polling);check:one ofsession,host,audit,processes,markers,canary,metric,readyz;parallel:.expecton every call and stream step:status,code,reason,errors({code, pointer}),close({code, reason}), op-specific fields;keepbinds names for later steps. - Expectations are literal data from 24 14 and the chapters it names (07, 08, 13 "Errors", 10.3); the suite never calls server or host code to compute one.
-
Conductor.
- Checks read state through the public API (
getTerminalSessionforstate,close_reason,close_detail,exit_code;getHostforstate,connected,doctorwithcheck,ok,codeandsession_user,session_usersandreported_session_users) and through read-only SQL withpsqlin thepostgrescontainer (audit_eventswithaction,decisionand the keys ofdetails;terminal_sessions;hostswithreported_doctoranddisabled_reason;host_session_users). Every read names its table and column as 03 does. - Asynchronous effects are awaited by polling the condition every 250 milliseconds until it holds or
t.Deadline()minus a margin passes; a step waits for a window of the specification only throughwait: <key>. - A failed expectation names scenario, case, step index, expected and observed codes, states and counts, never a body, a token or terminal output.
- Each e2e package must finish inside the default timeout of
go test, because theMakefileline stays as it is; if one does not, split it by area into further packages and report the wall time of each.
- Checks read state through the public API (
-
Notes per scenario (everything else literally from 24 14).
- TM-T03: the relay forwards every binary message unchanged (M2-44), so a message the device sends reaches the host exactly as if the relay had inserted it. Open
shell,start, run a shell, disconnect (detached), attach from the same device withinterminal.quick_reattach_window, then send the recorded message 1 of the first opening on the new stream before itsexpplusauth.clock_leeway(the step records the instant and fails the case as invalid, not as passed, when it comes later): the stream closes with4101and close reasontoken_replayed, the session staysdetached(M2-44 Design note 7), and exactly one shell of the session user exists before and after. Ruling for docs/DECISIONS.md. - TM-T04: wrong
audis the token of a session onr1in the handshake of a session on a second hostr1bof the same person; wrongsidis the token of session S1 in the handshake of S2 on the same host; wrongcekis a handshake with another ephemeral key than the hashed one; a foreignkidis a token with the right claims signed by a fresh Ed25519 key of the test under akidthat is not in the JWKS;typagent-invocation+jwtis a token with the right claims signed with the activeterminalkey of this throwaway stack: itskidfrom theactiverow of purposeterminalinserver_signing_keys(read-only SQL), its PEM fileterminal-<kid>.pemcopied fromsecrets.signing_key_dirofapiwithdocker compose cpinto a file of mode0600in the temporary directory (M1-32's format; the key stands for a faulty server, which is what the case is about); expired is the own token sent afterterminal.session_token_ttlplusauth.clock_leeway. Each: no shell process, the stream closes with4104andtoken_invalid, expired with4101andtoken_expired, and the session endsopen_failedwith that detail. Ruling for docs/DECISIONS.md. - TM-T05: handshake complete, no
start: no shell process; the session never reachesactiveand endsopen_failedwithopen_timeout, the stream closes with4100(M2-44TestRelay_OpenTimeoutNoStart, M2-45TestTMT05NoStart). - TM-T06: random canary parts from
test/canary'sRegistry(classcontent); the device typesprintf '%s%s\n' <part1> <part2>and asserts that the output holds<part1><part2>(positive control); the session token is registered as classsecret. Afterwards no part, no concatenation and no token appears in the device's recorded binary messages (the relay view), incompose logsof every service, inpg_dump --data-onlyof the database, in the files of the collector overlayfile, or injournalctlof thebothy-hostunits of the host container; matching usesScanoftest/canary(case, base64 and hex variants) overSinksthat hold these sources (Scope: the additive source method if missing), withAllowRulesthat allow the terminal canaries and the session token in no sink. A unit test proves that the wiring finds a canary written into one log line on purpose. - TM-T07, TM-T08, TM-T09, TM-T12, TM-T17, AZ-M54 to AZ-M61, ID-T28: the world of M2-40 with neutral names: the case's Owner; Alex, Sam and Robin, Robin a second owner; group Leser with member Sam; host
r1(assigned to Alex,organization, session usersterm-alexandterm-sam),r2(personal, holder Robin,term-robinandterm-owner),r3(assigned to Sam,declared_server_hosttrue,term-samandterm-owner),r5(no holder,organization, for AZ-M60); grantsterminalto Alex and Sam onr1and to Sam onr3; agent Lotse; app Tracker (a minimal test manifest withschema_version,slugtracker,nameTracker,versionand one read scope, without addresses, so nowebhook.allowed_hostsis needed). Only the rule under test denies. Expected answers: AZ-M54 and Alex onr1201; AZ-M55 and Sam onr1403 terminal_deniedwithreasonnot_host_assignee; Sam onr3403 terminal_deniedwithreasonserver_host_owner_only; AZ-M56 and a second owner onr2403 terminal_deniedwithreasoni6_holder_only; AZ-M57 and the grants of TM-T09409 i6_violation; AZ-M58 and TM-T07401 step_up_required; AZ-M59 and TM-T17403 forbiddenwithreasonsubject_kind_not_allowedon each of the five session operations, and4004on the device stream, each with a deny entry (agent token fromcreateAgentandcreateToken, app token frominstallAppandcreateToken); AZ-M60404 not_found, identical to a random host ID; AZ-M61403 terminal_deniedwithreasonno_session_user. TM-T12: another device without step-up401 step_up_required, the same device in the same login session withinterminal.quick_reattach_window200, after it401 step_up_required. - TM-T10: one case per kind of 24 8.3, each with the smallest world in which the revocation under test is the only change that ends the session (for the group rows a host assigned to the member, with the
terminalgrant on the group, as in M2-46's world), each session runningsetsid sleep <marker> &andnohup sleep <marker> &:revokeGrant(access_revoked); a grant with an expiry passingauthz.expiry_sweep(access_revoked);removeGroupMemberand an expiring membership (access_revoked);revokeOwnerof an owner working on a host not assigned to it (access_revoked, SR10);disableUser(user_disabled);revokeDevice(device_revoked);revokeSession(session_revoked);logout(logout);setHostStatetodisabledandretired(host_disabled,host_retired);issueHostPairingCodeon the active host (host_rekeyed);updateHostwithrevoke_session_users(access_revoked);updateHostwith anotherassigned_user_id(access_revokedfor the former assignee);setHostLlmCredentialstopersonalwith another holder (i6_violation). Expected: the stream closes with4100and the reason, no process of the session and no marker process is left, the row has thatclose_reason, and oneterminal.terminateentry carries it. Three kinds stay catalogue runs of kindunit(M2-46'sTestTMT10_RevocationKindsand the tests named here), each for a reason the real stack cannot change:bothy-host session-user removerestarts the host service, so its sessions endhost_restartedbefore the nextreportdrops the name (R-TM-27); that row of this suite expectshost_restarted, and theaccess_revokedof areportwithout the name stays proven by M2-43'sTestReport_DroppedNameEndsSessions_SR12; areportthat newly names the machine the server host cannot be produced through the TLS front (24 2.3), so it stays M2-43'sTestReport_ServerHostEndsNonOwnerSessions_SR10; the end of a login session byexpires_atwould need a shortauth.session_max_age, which would also end the bootstrap owner's session in the middle of the package, so it stays M2-46's sweep row (TestTMT11_SweepCatchesDeadNode), whilerevokeSessionandlogoutprove the same end path here. Ruling for docs/DECISIONS.md. - TM-T11 (
limits): idle (the real host ends withidle_timeoutafterterminal.idle_timeout), maximum duration across an attach from a second device (max_duration), detach (detach_expiredafterterminal.detach_grace), and the manipulated host service: M2-44's Fake-Host (with Noise after M2-46, in its mode that never reports idle) paired throughpairHostand connected through the TLS front with the test CA and the dialer of Design note 3; the server ends that session afterterminal.idle_timeoutplusterminal.idle_gracewithidle_timeout. Add the external constructor additively, with a test, if it is missing. - TM-T19 and TM-T20: pairing through
bothy-host pairin a container. TM-T19:confirmHostwith a wrong prefix gives409 fingerprint_mismatchuntilterminal.fingerprint_max_attempts, thengetHostshowspairingwithout keys, and the next control connection of the host service is closed with4110(in the journal of the unit, codes only, or in the audit entry ofhost.connect, whichever the merged code writes; record which). TM-T20: a code used twice, an expired code and a code with a wrong checksum each makepairexit1after401 unauthenticated(M2-41); "without database access" stays proven by M2-41'sTestTMT20_WrongChecksumNoDatabase(catalogue run of kindunit), because an end-to-end test cannot observe it. - TM-T21, TM-T24, TM-T28 (e), TM-T29 (
hostpolicy, except TM-T28 inconformance): "instance operator" steps run as root in the container, steps of a person in her shell session; "the console shows" asserts the server data the console reads (Host.doctorentries withsession_user, and the assignment of that name inHost.session_users). "the service manager does not restart the host service on its own" issystemctl show -p NRestarts -p ExecMainStatus -p ActiveState bothy-host.servicewithNRestarts=0,ExecMainStatus=78and the unit not active, read again after severalterminal.host_heartbeat_interval; "then the host restarts" in (g) is the host stepreboot. A subscription login is the credential file of 24 7.4 written by the Fake-claudeloginmode in the person's shell, withorganizationandhost_service.claude_binset to the Fake-claude; HD6 finds it withinhost_service.credential_check_interval. The long runner of TM-T24 is a crontab entry that opens the home once a trigger file exists, created aftersession-user remove. TM-T29 (e) fills the home with enough entries to exceed the profile'shost_service.entry_scan_timeout. The parts that need Windows or a case-insensitive file system ((i) and parts of (j)) run only in the platform runs, and TM-T28 (d) only on macOS (M2-45'sTestPlatform_TMT28d). - TM-T22: the shell of a session reads
host_service.state_dirand the key file and gets "Permission denied" in its output; then the operator grants the session user read access withsetfacland runsbothy-host restart: the start is refused with exit status78, andHost.doctorhas an HD8 entry withokfalse and one of M2-42's HD8 codes (state_access_for_session_userorhelper_reads_state, whichever the merged code reports for an ACL entry; record which). - TM-T23:
clone_statecopieshost_service.state_dirofr1as a tar stream into a second container with the same session users; both connected: both closed with4112,r1disabledwithdisabled_reasonduplicate_connection, audithost.connectwith codeduplicate_connection; second run with the older container paused: the new connection is accepted withinterminal.host_duplicate_probe_timeoutand serves a session, and afterunpausethe older instance shows close code4009in its journal (codes only) or, if M2-43 logs no close code, the audit entry ofhost.connectnames it; record which. - TM-T28 (c): the names are changed "bypassing the server" with
sqlsteps markedbypass: true, the only writes of the suite outside the API: one transaction renameshost_session_users.os_userwithin the constraints of 03 (andhosts.reported_session_usersfor the unknown name; for the bound name the row of its first person is renamed away first, because the name is unique per host over all rows) and raisessequence_counters.permission_versionby one, as every server-side change of these rows does (03hosts); expected4104withsession_user_unknownand withsession_user_mismatch, no shell. Ruling for docs/DECISIONS.md.
- TM-T03: the relay forwards every binary message unchanged (M2-44), so a message the device sends reaches the host exactly as if the relay had inserted it. Open
-
TM-T15 and TM-T16. Ruling for docs/DECISIONS.md.
- TM-T15 (
twonode): M2-47's overlay withapi,api-2, the example edgeedgeandcluster.mode = multi, one host container, the TLS front in modehash. With hash routing: sessions are opened one after the other, each runs (start, output) and is closed withcloseTerminalSession; M2-44's relay writes one log record per pairing at its end with the session ID, socompose logsofapiandapi-2show on which node each session was paired; the case opens sessions until both nodes have one (bounded by a test constant) and asserts that no4102occurred. Then the front switches tosplit: the device stream lands onapiand the host data connection onapi-2; afterterminal.open_timeoutboth close with4102,bothy.terminal.relay.unpairedrises by one withbothy.terminal.sideclientand by one withhostin the collector export, the session endsopen_failedwithopen_timeout, and/usr/local/bin/bothy healthcheck --url http://127.0.0.1:8081/readyzexits0in both api containers before, during and after. - TM-T16: the PLAN row lists TM-T16 among the IDs green in
make checkand again formake perf; it runs in both.scenarios/tm-t16.yaml(packagerestart, profiledefault): ashellsession withsetsid sleep <marker> &and output in the scrollback; then, each shorter thanterminal.detach_grace,docker compose restart api(the device stream closes, M2-44's start step leaves the sessiondetached, the host service reconnects withlive_sessions), the device's TCP connection cut without a close frame, and the host container disconnected fromtermhostsand reconnected (the device gets4102, the session isdetached); after each, an attach with a fresh step-up replays the scrollback betweenreplay_beginandreplay_end, and the shell and the background process live. Thenbothy-host restartin the container: the session endshost_restarted(M2-43'slive_sessionsrule), no process of the session and no marker process is left, andattachTerminalSessiongives409 invalid_state_transition(M2-44 Design note 6) without a token.TestChaosTMT16inservices/server/test/chaos/terminal_test.go(build tag of M1-72, no-race, as M1-72's recipe runs it) runs the same file on its own stack with several parallel sessions (a test constant) that print continuously; every session survives the first three faults and endshost_restartedat the fourth. Its database is the stack's PostgreSQL with the image defaults (fsyncandsynchronous_commiton), which is what M1-72 requires of every chaos database (ADR 0063 point 2), so it needs nopgtestoption.
- TM-T15 (
-
Platform runs. Ruling for docs/DECISIONS.md.
- The platform suite is M2-42's
services/host/test/platform(build tagplatform) with the tests M2-45 added; it drives the realbothy-hostas root or Administrator against M2-43'sinternal/control/controltest. Its tests together cover thedoctormatrix of exit criterion 8 (TestPlatform_TMT13, the HD9 part ofTestPlatform_TMT24, the location-for-all part (f) and the Windows part (i) ofTestPlatform_TMT29_Parts), TM-T22, TM-T24, TM-T26, TM-T27, TM-T28 (c) and (d) and the PTY tests. If neither M2-45's package tests (TestKillTreeEndsBackground,TestSessionCgroup,TestJobObjectKillOnClose) nor its platform tests (TestPlatform_HelperChannelBreak) cover, on each platform, the end of the whole process tree (24 6.5) andbothy-host restartending PTY and tree (24 6.2), addTestPlatform_TreeKill(a session withsetsidandnohupbackground processes, on Windows a detached child process, ends and nothing remains) andTestPlatform_RestartKill, using the session helpers of M2-45's platform tests. - The suite's
TestMaingains a guard, written in an untagged file of the package (guard.gowithguard_test.go, somake checktests it) and called by the taggedTestMain: without the marker file/etc/termsuite-disposable(Windows%ProgramData%\termsuite-disposable) it printsplatform: not a disposable test machine (marker file missing)and exits2. The host image of Design note 5 has the marker; on the machines the host rules name, the build session creates it before the run. make -C services/host platform-bundle OUT=<dir> [GOOS=<os> GOARCH=<arch>]: a usage error withoutOUT(exit2); builds, without running anything, what a run needs for that platform:bothy-hostasrelease-buildbuilds it, the Fake-claude(and itsnodebuild), the test binarygo test -c -tags platform ./test/platform, the test binaries of M2-45's packagesinternal/pty,internal/session,internal/profilesandinternal/confirm, andbundle.json(commit, Go version, file list with SHA-256). Every binary is prepared exactly as M2-42'splatform-testprepared it before this change; on macOS that includes what HD10 requires of the helper binary (the hardened runtime withoutget-task-allow, codehelper_unprotected, M2-42 Design note 8), which only the tools of macOS can apply, so a macOS bundle is built on the macOS machine, while Windows bundles may be cross-built.platform-testbuilds the bundle of the build host and runs it, so both paths execute the same files.- Each run writes the result file M2-42 defines (platform, OS version, Go version, commit, test names, pass or fail, durations). In
make check,e2e/hostplatformbuilds the Linux bundle and runs it as root in a fresh host container; it fails when any test fails and prints the counts. On macOS, Windows 10 and Windows 11 the build session copies the bundle to each disposable machine, runs it there as administrator, fetches the result file and lists it in the report. A missing run on one of these systems ends the ticket withVerdict: BLOCKED platform runs missing: <list>: the branch stays unmerged with all commits, and a later run of this brief only adds the runs.
- The platform suite is M2-42's
-
Docs gate (
tools/docs/terminal_catalogue.py, run bymake docs; tests intools/docs/tests/test_terminal_catalogue.py). Ruling for docs/DECISIONS.md.- The IDs of the table in 24 14 equal the TM-T entries of the catalogue;
titleandexpectedequal the cells. - The PLAN row M2-48 and exit criterion 8 of PLAN section 8 are parsed (lists and ranges "X to Y" of TM-T, AZ-M and ID-T IDs); every such ID has at least one run of kind
e2e,unitorplatformwithoslinux(these run inmake check) and nodeferred; TM-T16 has also aperfrun; the IDs of thedoctormatrix that exit criterion 8 names for every platform (TM-T13 for HD1 to HD3, TM-T24 for HD9, TM-T29 for HD11 at a location for all and part (i)) haveplatformruns formacos,windows10andwindows11;deferred_partsname only tickets of the plan, and the console parts of TM-T07, TM-T12 and ID-T28 are present there with M6-18. - Every T-TM row of threat model 6.7 names at least one test; every test ID named there (TM-T, AZ-M, ID-T, SEC-T) has a catalogue entry whose
threatscontain that threat; every Go test name named there exists asfunc <Name>(in a Go module of the repository; everyPLAN Mx-yyreference names an existing ticket row. - Every
refexists: scenario files on disk, Go test functions asfunc <Name>(in the named module (forplatforminservices/host/test/platform), Dart tests by their name inpackages/bothy_noise/test; everyimplemented_byanddeferredticket exists in the plan.
- The IDs of the table in 24 14 equal the TM-T entries of the catalogue;
-
Fixes. A non-conformance the suite finds is fixed in the owning module of M2-38 to M2-47, with a regression test in that module that was red first, spec first outside the core contract; each fix is listed in the report. A fix that needs a change of 00, 03, 04 or 05 ends the ticket with
Verdict: BLOCKED <reason>. -
CI path filter and runner. The suite reads
infra/compose/andinfra/otel/, which select no component today. Ruling for docs/DECISIONS.md:GATE_COMPONENTS["server"]intools/ci/components.pygains the prefixesinfra/compose/andinfra/otel/;test_components.pygets one case for each. Changes underservices/host/already selectserver.docs/quality/gates.md(the server prerequisites in "Skipping and failing") states that the Docker runtime of the GitHub-hosted CI runners (ADR 0039, D-W-92) must start privileged containers on cgroup version 2 withkernel.yama.ptrace_scopeof at least1. Measure the wall time ofmake gateswith the suite; if thetimeout-minutesof the jobgatesin.github/workflows/ci.ymlno longer holds it with margin, report the measured times as an owner decision with a recommendation; never raise it yourself. -
Red runs. Write each scenario file before the conductor supports its steps, run it and keep the red output. Prove that each kind of expectation bites (status, code, reason, errors, close code and reason, every
checkkind) with unit tests that feed a wrong expected value. The report has one row per ID of the PLAN row and exit criterion 8: the implementing ticket and the merged test that recorded its red run (from the catalogue'simplemented_by; expected names include M2-38TestVectorsTMT01,TestRecordingsTMT02Dart; M2-39TestAZM60_InstanceGrantGivesNoHost,TestUpdateHost_SessionUserTaken_TMT28b,TestUpdateHost_NameNotReported_TMT28e; M2-40TestTMT08_SR10,TestTMT09_I6,TestTMT28a_NoSessionUser,TestIDT28_ConcurrentOpeningsOneConfirmation,TestAZ_M57_GroupGrantOnPersonalHost,TestAZ_M59_AgentToken; M2-41TestTMT19_FingerprintMaxAttempts,TestTMT20_PairingCodeRejections; M2-42TestPlatform_TMT13,TestPlatform_TMT22,TestPlatform_TMT24,TestPlatform_TMT29_Parts; M2-43TestDuplicate_TMT23,TestLiveSessions_HostRestarted_TMT16; M2-44TestOpen_TMT07_StepUp,TestAttach_TMT12,TestOpen_TMT29g_WorkdirIndexOutsideOwnList,TestTerminalOps_NonHumanTokens; M2-45TestTMT03Replay,TestTMT04TokenFailures,TestTMT05NoStart,TestPlatform_TMT21_Sessions; M2-46TestTMT06_NoContentAnywhere,TestTMT10_RevocationKinds,TestTMT11_IdleWithoutHostReport; M2-47TestTMT15_TwoNodesWithHashRouting,TestTMT15_WithoutHashRouting), citing the red output where the launcher made the earlier reports available, otherwise naming the gap under "What is not proven". -
Specification and docs text (English, like the files). 24 14 gains a paragraph after the table: the tests run data driven in
services/server/test/terminalagainst the real server and host services in Linux containers, the host-local parts also on macOS, Windows 10 and Windows 11 test machines (services/host/test/platform), TM-T16 also inmake perf, the catalogue as the list of places. 02 6: thetest/line namesterminal/(suite, device, scenarios) andchaos/, the blockservices/host/namestest/(noisedevice,platform). ADR 0068 "Confirmation" gains one bullet naming the suite, the platform runs and TM-T16 inmake perf. Threat model 8: one paragraph on the catalogue and its gate.docs/quality/gates.md: rowtest(Docker withdocker buildx, privileged containers, cgroup version 2 andkernel.yama.ptrace_scopeof at least1for the terminal suite), rowdocs(the catalogue check), and the section onperf(TM-T16).docs/quality/toolchain.md: the Debian snapshot and the images of the suite.infra/compose/test/README.md: the overlay, the server image built from the worktree, the TLS front, the host image, theTERMSUITE_*variables, the removal of the cached images (termsuite-server:*,termsuite-host:*).infra/compose/README.md: the files table names the test overlays as test only. -
Rulings for
docs/DECISIONS.md, section 10, one rowD-M2-48-<n>each with date, decision, reason and place: architecture and the reading of M2-46's phrase (note 2); stack with the server image built from the worktree, the TLS front, the networks and trusted proxies (note 3); host containers,privileged, Debian snapshot, the Trivy exception, marker programs (note 5); device and Noise sidecar (note 6); catalogue, scenario files and one world per case (note 7); relay faults at the device end, token forgery with the stack's key, the Fake-Host for TM-T11, the rows of TM-T10 that endhost_restartedor stay unit runs, the writes "bypassing the server" in TM-T28 (c) (note 9); TM-T15 through the TLS front and TM-T16 in both gates (note 10); platform runs, the marker file,platform-bundleand the verdict without runs (note 11); docs gate (note 12); CI path filter and runner (note 14).
Tests (write them first)¶
Unit, Go, without containers (services/server/test/terminal/...):
suite:TestCatalogue_Load(strict decoding; an unknown field, a duplicate ID, a run with an unknownkindoros, an ID of the PLAN row withdeferred, and adeferred_partsentry withoutticketare rejected, from a table of broken documents);TestScenarioFiles_Load(every file underscenarios/loads;idequals the file name; every step has exactly one kind; everycallnames an operation ofdocs/api/openapi.yaml; everywaitkey is a row ofdocs/operations/configuration.mdand set in the file's profile; every call and stream step has anexpect; every case ends with acheck; a writesqlstep withoutbypass: trueis rejected, andbypassis allowed only intm-t28.yaml);TestProfiles_Relations(each relation of Design note 4 fails with both key names when broken; every key exists in the configuration reference);TestExpectationsBite(for each expectation andcheckkind a wrong expected value makes the step fail with scenario, case and step index and without body or token);TestScenarioFiles_NoSecretsAndWhiteLabel(no compact JWS, no token prefix of ADR 0061, no PEM block, onlyexample.comaddresses, only the people of the world and the neutral host and session user names);TestCanaryWiringFindsPlantedValue(Design note 9, TM-T06).stack:TestStackPreflightMessages(a fakedockerexecutable first onPATH: missingdocker, missingbuildx, old Engine, old Compose, cgroup version 1 andptrace_scope0each fail with the fix in the message);TestServerImageBuildArgs(the build command of Design note 3: Dockerfile, platform from the Docker host,VERSIONfromservices/server/VERSION,REVISIONfromHEAD, and the tagtermsuite-server:with the first 12 hex characters of the image ID, against a fakedocker);TestFrontConfigRendering(the three routing modes; the timeouts equal the profile's keys as Design note 3 computes them);TestComposeTerminalOverlay(Design note 3; needsdocker compose, not a running stack).device:TestSidecarProtocolRoundTrip(against a fake sidecar process: every op and every error kind);TestDialerMapsInstanceOrigin(chat.example.com:8443reaches the mapped port and verifies against the test CA only).
Unit, Go (services/host):
test/noisedevice:TestProtocolHandshakeAgainstResponder(the sidecar as initiator against the responder ofinternal/noise: the payloads of message 1 and 2 and every frame of M2-38's Go recording round-trip);TestProtocolErrors(garbled JSON, unknown op, wrong order: error kinds, no bytes on stderr);TestCekEqualsEphemeralKeyHash.test/platform:TestMainRefusesWithoutMarker(untagged helper test of the guard: no marker gives exit2and the message).
Python (tools): docs/tests/test_terminal_catalogue.py with the four checks of Design note 12, each with a negative fixture (a missing 24 row, an ID of the PLAN row without run, a threat without test, a ref to a missing function) and one run against the repository that passes; ci/tests/test_host_platform_bundle.py (test_platform_bundle_without_out_is_usage_error; test_platform_bundle_writes_bundle_for_linux: bundle.json lists every file with its SHA-256, and the test binary has the format ELF); ci/tests/test_components.py cases for infra/compose/ and infra/otel/.
End to end in make check (one package per stack; Docker as in requirement 18; TestScenarios of each package has one subtest <scenario>/<case> per case; the expected values are those of Design note 9 and 24 14, written literally into the scenario files):
e2e/conformance(profiledefault):tm-t03/replay_message1: the recorded message 1 of the first opening on the stream of a quick reattach gives4101withtoken_replayed; the session staysdetached; exactly one shell before and after.tm-t04/wrong_aud,wrong_sid,wrong_cek,foreign_kid,invocation_typ: no shell process, the stream closes with4104andtoken_invalid, the session endsopen_failedwithtoken_invalid;tm-t04/expired:4101withtoken_expired,open_failedwithtoken_expired.tm-t05/no_start: messages 1 and 2 complete, nostart: no shell process, the stream closes with4100, the session endsopen_failedwithopen_timeout.tm-t06/canary_session: the positive control sees the canaries in the output; no finding in relay view, container logs, database dump, collector export and host journal.tm-t07/no_step_up,too_old,consumed:401 step_up_required, each with a deny audit entry ofterminal.open.tm-t08/foreign_host,own_host,server_host:403 terminal_deniedwithreasonnot_host_assignee;201;403 terminal_deniedwithreasonserver_host_owner_only.tm-t09/grant_second_person,grant_group,switch_to_personal:409 i6_violation, nothing changed;tm-t09/second_owner_opens:403 terminal_deniedwithreasoni6_holder_only.tm-t10/<kind>, one case per row of Design note 9: the stream closes with4100and the reason, the row has thatclose_reason, no process of the session and no marker process is left, oneterminal.terminateentry;tm-t10/session_user_remove:host_restarted.tm-t12/other_device,same_device_within_window,same_device_after_window:401 step_up_required,200,401 step_up_required.tm-t17/agent_token,app_token:403 forbiddenwithreasonsubject_kind_not_allowedonopenTerminalSession,attachTerminalSession,listTerminalSessions,getTerminalSessionandcloseTerminalSession,4004on the device stream, one deny audit entry each.tm-t19/fingerprint_max_attempts:409 fingerprint_mismatchfor each wrong prefix up toterminal.fingerprint_max_attempts, thengetHostshowspairingwithout keys and the next control connection closes with4110.tm-t20/code_twice,code_expired,wrong_checksum:pairexits1after401 unauthenticated; state and keys of the host stay as they were before the rejected attempt.tm-t22/read_state,helper_with_read_access: "Permission denied" in the session output; the start is refused with exit status78and an HD8 entry withokfalse.tm-t23/older_answers: both connections closed with4112, the hostdisabledwithdisabled_reasonduplicate_connection, audithost.connectwithduplicate_connection;tm-t23/older_silent: the new connection accepted and serving a session, the older one closed with4009.tm-t28/a_no_session_user:403 terminal_deniedwithreasonno_session_userand audit;b_name_taken,b_second_assignment:409 conflictwithreasonsession_user_taken, nothing changed;c_unknown_name,c_bound_name: no shell,4104withsession_user_unknownandsession_user_mismatch;e_not_reported:400 validation_failedwitherrors[].codesession_user_not_on_hostonconfirmHostandupdateHost, nothing stored;e_name_removed: after the nextreport403 terminal_deniedwithreasonno_session_user, no4104.az-m54-61/az_m54toaz_m61, one case per row with the answers of Design note 9 and one audit entry per decision.id-t28/too_old,used_for_earlier_opening:401 step_up_required;id-t28/concurrent_one_confirmation: of two parallel openings with one confirmation exactly one gives201and one session, the other401 step_up_required.TestBaselineDoctorPasses(every freshly installed host passesbothy-host doctorwith exit0, andHost.doctorhas no failed entry);TestHostServiceListensOnNoNetworkPort(ss -ltnupandss -lunpin a host container with a running session: no socket of the service user or a session user on a non-loopback address; no published port of any host container, fromdocker inspect; from a host container onlychat.example.com:8443answers, whileapi:8080andpostgres:5432do not resolve);TestSessionRunsAsSessionUser(ps -o user=of every process of a session is its session user, neverrootor the service user).
e2e/hostpolicy(profiledefault):tm-t21/atotm-t21/g: the seven steps of the row of 24 14 in one case each, building on the state before, with the expected values of that row (among themhost_policy_violationandsubscription_login_detectedin (a), two HD6 entries in (b),403 terminal_deniedwithreasonno_session_userin (d), the refusedsession-user addin (e),host_restartedin (f), and in (g) the refused start withreport, the name inHost.doctor, the restart policy of Design note 9 and the reopening afterbothy-host restart).tm-t24/foreign_home_listable(start refused with HD9, unnamed report entry),without_right(passes),open_home_of_a(HD11 with the name of A, only A's sessions end),after_restart,after_remove,long_runner_reopens(HD9 with the name of A,report, the service ends),after_delete_and_restart(HD9 passes, B opens).tm-t29/a,b,c,d,e,f,g,h,j_posix,k,l: the parts of the row of 24 14 that need neither Windows nor a case-insensitive file system, each with the expected values of that row (HD11 with the name of B and no session for B while others open; HD7 lists;403 terminal_deniedwithreasonprofile_unknownand4104withprofile_unknownin (g); the marker folder empty in every case).
e2e/limits(profilelimits):tm-t11/idle_real_host(idle_timeout),max_duration_across_attach(max_duration),detach_grace(detach_expired),manipulated_host(the Fake-Host never reports idle; the server ends withidle_timeoutafterterminal.idle_timeoutplusterminal.idle_grace); each stream closes with4100, and no process of the session is left on a real host.e2e/twonode(profiletwonode):tm-t15/hash_routing(sessions paired on both nodes, no4102),tm-t15/without_hash_routing(4102on both sides,bothy.terminal.relay.unpairedplus one withclientand plus one withhost,open_failedwithopen_timeout,/readyzready on both nodes).e2e/restart(profiledefault):tm-t16/faults_then_host_restart(the PTY and the background process survive the server restart, the cut of the device and the loss of the host network, each attach replays the scrollback; afterbothy-host restartthe session endshost_restarted, no process is left, and the attach gives409 invalid_state_transition).e2e/hostplatform:TestPlatformSuiteLinux(the Linux bundle in a fresh host container; every test passes, among themTestPlatform_TMT13(start refused with a code for each case of the TM-T13 row),TestPlatform_TMT22,TestPlatform_TMT24,TestPlatform_TMT29_Partsand M2-45's PTY and process tree tests; the counts in the test output).
Catalogue runs of kind unit that make check already runs and this ticket points to (Design note 2): TM-T01 (TestVectorsTMT01 in services/host/internal/noise, TM-T01 cacophony and TM-T01 noise-c in packages/bothy_noise/test/vectors_test.dart); TM-T02 (TestRecordingsTMT02Dart, TestRecordingsTMT02Negative, the TM-T02 tests of recordings_test.dart); the password and TOTP part of ID-T28 (TestIDT28_PasswordTotpAndOidcStepUp, because local passwords and TOTP come from M1-39, outside this ticket's stack setup); "without database access" of TM-T20 (TestTMT20_WrongChecksumNoDatabase); the three rows of TM-T10 of Design note 9; and the server-level tests of M2-39 to M2-47 for every ID they name.
Chaos in make perf: TestChaosTMT16 (the parallel sessions of tm-t16.yaml survive server restart, device cut and host network loss with their scrollback, and all end host_restarted after bothy-host restart with no process left).
Red-run evidence (PLAN row): every scenario file was run red before the conductor supported its steps (output kept for the report), TestExpectationsBite proves that each expectation kind fails on a wrong value, and the table of Design note 15 names the red run of the implementing ticket per ID. Threat model (PLAN row): test_terminal_catalogue.py fails on a T-TM row without a test and passes on the repository.
Platform runs (Design note 11): the bundle on disposable macOS, Windows 10 and Windows 11 machines; every test passes.
Existing tests that must stay green: the tests of M2-38 to M2-47 in services/server and services/host (among them M2-42's TestHostPairAgainstServer, M2-46's tests in services/server/test/terminal and M2-47's services/server/test/terminal/multinode and services/server/test/edge), the packages/bothy_noise tests (TM-T01, TM-T02), tools/ci/tests/test_components.py, and the smoke and canary tests of M1-73 and M1-69.
Red team focus of the milestone (PLAN 8 "Red team", terminal paragraph): name in the red team request TM-T03 to TM-T06 (replay and token swap at the relay, content), TM-T07 and TM-T12 (step-up), TM-T08, TM-T09, TM-T17 and TM-T28 (rights, SR10, I6, SR12, agent and app tokens), TM-T10 (each kind of revocation), TM-T13, TM-T22, TM-T24 and TM-T29 (rights escalation, shared locations, doctor), TM-T15 (misrouting), TM-T16 (restarts and network loss), TM-T21 (standstill), and the platform runs.
Commands that must pass, from the repository root:
cd services/server && go test -race -count=1 ./test/terminal/...
cd services/server && go vet -tags perf ./test/chaos/...
cd services/host && GOTOOLCHAIN=local go vet -tags platform ./... && GOTOOLCHAIN=local go test -race -count=1 ./...
make -C services/host platform-bundle OUT=<temporary directory>
cd tools && uv run --locked --quiet pytest docs/tests/test_terminal_catalogue.py ci/tests/test_host_platform_bundle.py ci/tests/test_components.py
make arch
make codegen-check
make lint
make trivy
make docs
make linkcheck
make adr-index
make perf
make check
make perf must show TestChaosTMT16 passing; the performance targets of other tickets are not this ticket's business, but report its full result.
Acceptance criteria¶
-
catalogue.yamllists TM-T01 to TM-T29, AZ-M54 to AZ-M61, ID-T28 and SEC-T05 with their runs, implementing tickets and threats; the docs gate holds it equal to 24 14, the PLAN row, exit criterion 8 and threat model 6.7. - TM-T01 to TM-T13, TM-T15 to TM-T17, TM-T19 to TM-T24, TM-T28 and TM-T29 are green in
make check, end to end against the real server, real host services in Linux containers and the Go test device, or in the platform suite in a Linux container where Design note 11 places them; AZ-M54 to AZ-M61 and ID-T28 without the console part are green end to end. - TM-T16 is green in
make perf(TestChaosTMT16) and inmake check; after server restart, device network loss and host network loss the PTY survives and the attach replays the scrollback; afterbothy-host restartthe session endshost_restarted, no process is left, and the attach gives409 invalid_state_transition. - TM-T03 to TM-T05: a replayed message 1 gives
4101withtoken_replayed; wrongaud,sid,cek, a foreignkidand thetypof an invocation token give4104withtoken_invalid; an expired token gives4101withtoken_expired; no PTY starts in any of these cases or withoutstart. - TM-T06: the canary of input and output and the session token appear in none of relay view, container logs, database dump, collector export and host journal, while the device saw the output.
- TM-T07, TM-T08, TM-T09, TM-T12, TM-T17, TM-T20, TM-T28 and AZ-M54 to AZ-M61:
401 step_up_required,403 terminal_deniedwithreasonnot_host_assignee,server_host_owner_only,i6_holder_onlyorno_session_user,409 i6_violation,409 conflictwithreasonsession_user_taken,400 validation_failedwitherrors[].codesession_user_not_on_host,403 forbiddenwithreasonsubject_kind_not_allowed,404 not_foundand401 unauthenticatedfor a failed pairing exactly where 24 14, 07 11.1 and the terminal briefs say so, each with its audit entry. - TM-T10 and TM-T11: every kind of revocation of 24 8.3 and every limit ends the session with the right
close_reason, closes the stream, ends PTY and background processes and writes the audit entry; the manipulated host service cannot keep a session open. - TM-T19, TM-T21 to TM-T24 and TM-T29 (Linux parts) behave in real host containers exactly as their rows of 24 14 say: the fingerprint fallback with
4110; the standstill through HD6 and its end by the operator steps, with a refused start that reports and is not restarted; "Permission denied" and HD8;4112,disabledwithduplicate_connectionand4009; HD9 and HD11 with the names the rule for names allows; no marker program ever runs. - TM-T15: with hash routing sessions run on both nodes; without it each lone side closes with
4102,bothy.terminal.relay.unpairedrises withclientandhost, and/readyzstays ready. - The
doctormatrix of exit criterion 8 refuses the start on Linux, macOS and Windows for each violation of HD1 to HD3, HD9 and HD11 at a location for all; HD11 passes on Windows 10 and Windows 11 afterbothy-host installwith confirmation (TM-T29 (i)); the PTY tests ran on all three systems; the report lists every platform run. - The stack is the installation Compose file unchanged with test overlays and the server image built from the worktree; no host container publishes a port or listens on a non-loopback address, the host network reaches only the TLS front, and the suite needs no Tailscale component.
- Every scenario was red before its implementation: the report has one row per ID with the implementing ticket and its red run.
- Every T-TM threat of 6.7 names a test that exists; ADR 0068 "Confirmation", 24 14, 02 6 and the quality docs name the suite.
- No terminal content, token, key, pairing or enrollment code in test output, logs of the suite or the report.
-
make checkis green.
Definition of done¶
- Code and tests as specified; doc comment on every public symbol (purpose, parameters, return value, errors, example where useful); comments explain why.
services/server/test/terminal/README.md,services/server/README.md,services/host/README.md,infra/compose/test/README.mdandinfra/compose/README.mdupdated (layout of the suite, profiles, the TLS front, how to run one scenario, the platform bundle, the marker file, prerequisites, removal of the cached images).- Rulings of Design note 17 recorded in
docs/DECISIONS.mdsection 10 (one row each with reason); the changes to 24, 02, ADR 0068, the threat model and the quality docs made in the same change;make adr-indexrun. services/server/CHANGELOG.mdandservices/host/CHANGELOG.mdentries under 'Unreleased'.- An ADR only if you take an architecture decision beyond this brief (MADR 4 as in
docs/adr/0000-template-madr-4.md, headings and text in English, next free number,make adr-index). make checkgreen;make perfrun withTestChaosTMT16green.- An independent code review over the diff, then the red team gate (
docs/quality/red-team-gate.md, verdictJAorNEIN) with attention to the terminal items named under Tests; onNEINfix with a test per finding and have the delta checked again. Record both verdicts. - Land the change as
AGENTS.md("How work lands onmain") describes: a pull request from this branch, squash-merged only when review, red team, the platform runs andmake check,ci-okand the review threads are green; thengit worktree remove.
Report¶
Write to {{REPORT_PATH}}:
- First line exactly
Verdict: DONEorVerdict: BLOCKED <reason>. - Commits (SHA and subject), changed files with line counts, test results with counts (the red run before the change and the green run after), the summary of
make checkand ofmake perf, the wall time of each e2e package, of the server image build (uncached and cached) and ofmake gates. - Review and red team verdicts and the findings fixed.
- The list of platform runs: one row per run with operating system, version, architecture, machine kind (container or disposable machine), commit, date, the number of passed and failed tests, and the SHA-256 of the result file.
- The table of red runs of Design note 15, and every non-conformance fixed under Design note 13 with its regression test.
- What is not proven (at least: the platforms or versions that ran nowhere, the console parts of TM-T07, TM-T12 and ID-T28 (M6-18), the client parts TM-T14 and TM-T18 (M4-35), "without database access" of TM-T20 beyond M2-41's test, the three rows of TM-T10 that stay unit runs (Design note 9), and which of the journal or the audit showed the close codes of TM-T19 and TM-T23 and which HD8 code TM-T22 saw).
- Follow-ups that could not be done in this scope, each with its reason; every difference between the names in this brief and the merged code (D-W-100); spec gaps found; and, if the CI timeout of Design note 14 no longer holds, the measured times with a recommendation.
Then write {{DONE_PATH}} with exit=0 sha=<sha> or exit=1 <reason>.