Skip to content

M7-14 Portability test in CI with the full stack

No Date Ruling Reason
D-M7-14-1 2026-10-25 Order of one storage run (Design note 1, 16 11.1 "As built" item 1): interface, sources, bundle values, pull and signature check of every version image, install.sh, comparison of the running images, overlay local, the smoke scenario of M1-73 with the code install.sh printed, the portability scenario with a second code of bothy setup issue-enrollment-code (the first device is revoked at the end of the M1-73 scenario), backup pg_full, logical, objects (driver fs only: with s3 the object backup is a mirror into an off-site bucket, and the installation of the test has none, so bothy-backupd backup objects answers s3_mirror_not_configured) and restore-test, the export of Loki, Prometheus and Tempo with the canary scan, for fs the tailnet check, then the digests. fs runs before s3. The smoke steps of 11.1 step 3 that M1-73 already proves (owner, space, channel, thread, message, WebSocket reception) run as its unchanged scenario; the new steps are the subcommand bothy-smoke portability. One probe for both, black box through the public API (M1-73); no second implementation of the enrollment and the gateway.
D-M7-14-2 2026-10-25 Trial path (Design note 2): make portability-trial builds the server and product trial of M7-10 and M7-29 for the platform of the build host, with the images built by the host's default builder and pushed with docker push (a fresh docker-container builder pulls every base image again; on the build host its disk ran full), then runs make portability with PORTABILITY_INSTALL_SOURCE=release-download, PORTABILITY_RELEASE_DIR of the trial and PORTABILITY_TRIAL_PUBLIC_KEY (the throwaway key of the trial). Signing, verification, tags and the product build are the code of the trial of M7-10; the release chain itself stays proven by make release-trial. The trial-only variables (PORTABILITY_TRIAL_PUBLIC_KEY, PORTABILITY_TRIAL_STORAGES, PORTABILITY_TRIAL_FAIL_AFTER_INSTALL, PORTABILITY_TRIAL_KEEP) are refused when GITHUB_ACTIONS is set; --hold of tools/portability/portability_trial.py serves the trial release for runs by hand. The negative cases: a forced failure after the installation must print docker compose ps --all and the logs before the cleanup; the tag of bothy-server moved by the registry API to the platform manifest of the signed index (the same image, its digest unsigned; imagetools create would only wrap it into the same index again) must fail at the verification, before install.sh. Docker treats the registry on 127.0.0.1 as plain HTTP without an insecure-registries entry; cosign in its container reaches it through the host network. D-W-127: the same test code in trial and release; the negative cases prove that a passing run means something.
D-M7-14-3 2026-10-25 Tailnet case without an account (Design note 3): next to the running fs installation the tailscale sidecar and api-tailscale start without a login (profile tailscale added); in /proc/net/tcp and tcp6 of their shared network namespace the public listener 8080 must listen on loopback only, and the only listener beyond loopback may be the internal one (8081, 16 4.2: health and the incoming webhooks of the stack, reached by Prometheus and Alertmanager over the Compose network; the embedded DNS of Docker on 127.0.0.11 counts as loopback). On a tailnet only the port tailscale serve exposes then reaches the API; that the tailnet admits port 443 only is the tailnet policy of R-OP-03, a setting of the account, and a real tailnet is not part of the test. R-OP-03; the check needs neither an account nor a network outside the host, and it found that the internal listener listens on all addresses by design.
D-M7-14-4 2026-10-25 Isolation of an installation: install.sh runs in a new session (no controlling terminal, so the enrollment code goes to standard output into a file of mode 0600) with COMPOSE_PROJECT_NAME=bothy-portability-<random> and 0 for BOTHY_HTTP_PUBLISH_PORT, BOTHY_OTLP_PUBLISH_PORT, BOTHY_PROMETHEUS_PUBLISH_PORT, BOTHY_ALERTMANAGER_PUBLISH_PORT and BOTHY_GRAFANA_PUBLISH_PORT in the shell environment, so Docker picks free ports where it binds (a port picked free on the host was taken inside the Docker VM of the build host in one trial run); the test reads the port of api with docker compose port; the overlay local is switched on the same way (profile observability, BOTHY_OTEL_EXPORTER=local, BOTHY_TRACE_SAMPLE_PERCENT=100). Compose prefers the shell environment over .env, so the files install.sh wrote stay unchanged, and these are no options of install.sh. Alertmanager and Grafana stay off: Alertmanager refuses to start without its webhook URLs (M7-20) and neither holds exported data. No fixed host port and no fixed project name (deterministic tests); the options file stays the only source of the options (16 11.1 rule 2).
D-M7-14-5 2026-10-25 The test adapter is an adapter on the Python SDK whose runtime is bothy_sdk.testing.doubles.RecordingHandler, extended to send pong in execute and to write ready after its first poll. It uses REST polling (pollAgentEvents) through an httpx transport that sends every request to the published port, while the DPoP proofs name the public URL (the probe does the same with its base and public URL). The probe creates the agent (adapter_kind custom), its token bound to a P-256 key it generates and writes as PKCS#8 PEM, both with mode 0600 in the adapter folder, and waits for ready before it mentions the agent, because a new consumer starts at head_seq. The runner and the adapter need uv on the runner (astral-sh/setup-uv pinned by SHA in portability.yml). Rule 11 of the brief: no model; the gateway would need a WebSocket redirect the SDK has no seam for, polling is a supported transport of the SDK.
D-M7-14-6 2026-10-25 Names that differ from the brief (D-W-100): the runner is tools/portability/portability_run.py and the trial tools/portability/portability_trial.py (the Python tooling of the repository runs in tools/, a module named portability would be shadowed by its folder); tests/portability/ keeps the options file, the adapter (linted with the tools' ruff configuration) and the README. The status check of the test on the release is the matrix job full stack on a fresh runner of portability.yml; the digests are printed at its end. The static check that the installation's compose.yaml hands a server container only BOTHY_* keys with the status active in docs/operations/configuration.md is a unit test of the runner (requirement 17); the trial proves the start. Merged names win; the requirements stay the same.
D-M7-14-7 2026-10-25 The first trial run with --storage s3 found that every installation with the S3 driver failed: migrate aborted on the cross-check file_s3, because the installation's compose.yaml gave the server the http endpoint of the in-stack SeaweedFS without the egress.allowed_hosts entry with allow_private that 12 "S3 driver" requires. The installation now sets BOTHY_EGRESS__ALLOWED_HOSTS from the new Compose variable BOTHY_EGRESS_ALLOWED_HOSTS (default seaweedfs:8333;allow_private, also in .env.example and the smoke overlay, which mirrors the installation's server environment). An operator who points BOTHY_S3_ENDPOINT elsewhere changes it with it. The documented install command uses --storage s3 (16 3.1); the portability test exists to catch exactly this, and a fix in one place keeps the install path working.
D-M7-14-8 2026-10-25 The next trial run with --storage s3 found the second gap M3-02 named as a plan gap: secrets-init reported s3_access_key and s3_secret_key as pending, so the server of every s3 installation stopped at its start (secret not found). In the catalog of internal/platform/secrets/stack both stay supplied entries but carry a generator (s3_access_key key, 32 bytes as hex, s3_secret_key token, 32 bytes as base64url; neither holds /, which would break a SigV4 credential scope): absent from _store and the source, init generates them, and SeaweedFS of the profile s3 takes them as its admin identity; for an external S3 service the owner supplies both, init adopts them, and a changed source replaces them like any supplied entry (16 5.3 rule 3; a generated origin would have turned that rotation into a conflict, found by the delta red team). They are generated with every profile; unused without the profile s3. The brief of M3-02 assigned the end-to-end profile s3 to the installation and this test; one generator in the one catalog keeps every installation path working.
D-M7-14-9 2026-10-25 The digest the test verifies is the one docker pull reports for the tag (its line Digest:), not an entry of RepoDigests: a host that pulled a signed image of the same repository before keeps that digest in RepoDigests, and the unsigned case of the trial passed the verification with it once. The verification must name the image the tag points at now; the trial's unsigned case holds it.