Skip to content

M2-27: Incoming webhooks native and alertmanager

Status: ready | Depends on: M2-05, M2-24 | Milestone: M2

Goal

After this ticket an external system writes into exactly one channel of the instance through an incoming webhook, without a session or an API token of its own. A person with manage on the channel creates the webhook for an installed app that has write there (step-up, audit) and receives its URL with a bothy_whi_ token exactly once; the server keeps only the HMAC. The sender posts either with Authorization: Bearer <token> or with the token as the last path segment, in the format native (Markdown with optional thread_id, title and client_msg_id) or with the unchanged webhook payload of Prometheus Alertmanager, which the server turns into one message per notification. The checks run in the order of 11 8.4 (rl.webhook_in, token, body size, format, authorize of the app), and every refusal of the token or of the app's rights looks like an unknown webhook (404). The message lands in the thread chosen by the rules of 11 8.3, carries origin = app with the app as author, resolves no mentions and never triggers an agent (AZ-M53, SR4). The path token appears in access logs, traces and Problem.instance only as the route pattern. A cron job that sends native with one client_msg_id per run gets exactly one message per run however often it retries (11 10.4), and the instance's own Alertmanager reaches the same routes on the internal listener (16 4.2), which M7-20 then configures. This builds the two built-in inbound formats of R-EX-08 and the cron and alarm paths of requirement 19 (11 section 10, PLAN block "reference integrations").

Read first

  • docs/PLAN.md: section 3 "Way of working per ticket"; section 5 "Test strategy"; section 8 "M2 Agent platform, extensions and terminal": "Goal" (P-18: apps and webhooks use the same public API), "Scope" (11, webhooks of 06 and docs/api/webhooks.md), "Exit criteria" 3 (AZ-M53), 9 (the cron path of M2-27 is green) and 10, "Test strategy" (fakes with fixtures from the vendor's public documentation, no real third-party service), "Red team" ("exceeding the scopes of an app", apps and webhook targets as trust boundaries); block "agents and extensions" rows M2-24 (apps, installations, scopes), M2-25, M2-26 (outgoing webhooks), M2-27 (this ticket) and M2-28; block "reference integrations" (introduction: cron messages over the incoming webhook, monitoring alarms over the format alertmanager, no integration calls a real service in tests) and rows M2-49, M2-50; rows M1-69, M1-70, M6-08, M7-08 and M7-20; section 18.1 row 19m.
  • Dependencies (D-W-103): the header names M2-05 (the acceptance subscriber runs.OnMessagePosted, needed to prove "no run" in AZ-M53) and M2-24 (tables apps and app_installations, app principals, installation, uninstall, the authorize facts of app principals). Implied through them and therefore not in the header: M1-19 (ratelimit.Limiter, rl.webhook_in keyed by webhook_id, failed-credential hook of StageAuthenticate), M2-04, M2-02, M1-56 (createMessage, createThread, MessagePosted), M1-55 (threads), M1-54, M1-53 (channels, ChannelArchived), M1-44 (token format with kind whi, HMAC with auth.token_pepper_ref), M1-35, M1-34, M1-32, M1-31, M1-29, M1-28, M1-25, M1-24, M1-20, M1-18, M1-17, M1-16, M1-13, M1-12, M1-11, M1-10 and the tickets before them; M1-62 (moveThread), M1-68 (bothy.message.created) and M1-69 (canary scenario) are in the closure as well (through M2-02). Used only if merged: M1-70 (leak and timing tests), M2-26.
  • docs/spec/11-extensions.md: 1 "Principles" (point 4: the server never loads a URL from user data); 3.1 "Format" (row incoming_webhooks of the manifest); 4.1 "Installation"; 4.3 "Suspension and uninstallation" (row "Grants, incoming webhooks": suspend keeps them, uninstall revokes them); 5 "Scopes and grants"; 6 "Isolation and limits" (row "Rate limits": rl.webhook_in per webhook; row "Never allowed"); 8 "Incoming webhooks" with 8.1 "Creation and call", 8.2 "Formats" (with the Alertmanager evidence), 8.3 "Target thread", 8.4 "Rules" (the flowchart is the order of checks); 10 "Ready-made integrations as examples" with 10.3 "Monitoring alerts" and 10.4 "Cron reports"; 14 "Decisions of this chapter" (R-EX-08, R-EX-09, R-EX-11); 16 "Configuration values of this chapter".
  • docs/spec/03-data-model.md (core contract, read only): incoming_webhooks (all columns, keys, index, retention config, the last sentence); api_tokens (the paragraph on the format with kind whi in incoming_webhooks.token_hash); apps; app_installations (state, granted_scopes); messages (origin, client_msg_id, client_payload_sha256, the unique pair (author_principal_id, client_msg_id), the check trigger_state IS NULL OR origin = 'human'); threads (kind, parent_message_id, the unique index on parent_message_id); channels (kind, free_response, archived_at, the paragraph "Mention rule"); 6.3 "Lock order".
  • docs/spec/04-sync-protocol.md: 5.1 "Messages: client_msg_id".
  • docs/spec/05-actions.md: rules 1.1, 1.2, 1.3 and 1.9; 2.2 "Target of a resource"; 2.3 "Effective level" point 5; 2.4 (column Audit denials, aggregation R-AZ-04); 2.5 "Decision"; 2.6 "Scopes for apps" (row webhook_in.post: no scope, token of the incoming webhook); 3 "Rate-limit classes" (rl.webhook_in, rl.public_auth with failed credentials); 4.11 "Apps and webhooks" rows webhook_in.read, webhook_in.create, webhook_in.revoke, webhook_in.post; 5 "Agent invocation agent.invoke" point 1.
  • docs/spec/06-api.md: 1 "Overview" (table of surfaces); 2 "Basic rules" (rows "No effect", "No null", "Unknown values", "Size"); 3 "Authentication per principal kind" (row "Incoming webhook"); 4 "Authorization in the API"; 5 "Errors" (point 1 on instance with {token}, points 3 and 7); 7 "Idempotency" (rows "Incoming webhook" and "POST with a one-time secret in the response (enrollment code, API token, later webhook secrets)", point 3 with the generated exemption table); 9 "Rate limits"; 12 "Webhooks"; 16 "Endpoints" (generated).
  • docs/spec/07-permissions.md: 2.10 "Apps"; 4.1 "Decisions and reasons" (app_not_installed, no_grant, scope_missing, level_insufficient, channel_archived); 4.2 "Flow"; 4.5 "Archived spaces and channels"; "SR4 Only humans trigger"; 10 "Examples" (the world); 11.1 "Test matrix" (the world paragraph with G-6 for the app Tracker, row AZ-M53); 11.2 rows AZ-P1 and AZ-P4.
  • docs/spec/08-identity.md: 12 "API tokens for agents, apps and tools" (form with kind whi, storage as HMAC, the bullet "Incoming webhooks").
  • docs/spec/02-architecture.md: 3.2 "Roles of a process" (row of the internal listener: incoming webhooks from the stack in roles api and worker); 4.3 "Ports, events and sources per module" (row apps: port IncomingWebhooks, subscription chat.ChannelArchived); 4.4 "Allowed dependencies" (row apps); 8.1 to 8.3; 9.1 "Chain in the api node"; 9.3 "Proof of the check".
  • docs/spec/16-operations.md: 4.2 "Two listeners"; 4.3 "Edge and identity".
  • docs/spec/19-security.md: SEC-IN-1, SEC-IN-2, SEC-IN-4 (6.1), SEC-OUT-1, SEC-OUT-2 (6.2), SEC-LOG-1 to SEC-LOG-3 (9), SEC-T10 (14).
  • docs/api/openapi.yaml: operations listIncomingWebhooks, createIncomingWebhook, revokeIncomingWebhook, postIncomingWebhook (with security: [], x-idempotency-exempt: client_id); schemas IncomingWebhook, IncomingWebhookCreate, IncomingWebhookCreated, IncomingWebhookList, IncomingWebhookPost, IncomingWebhookResult, Problem (x-problem-codes, the description of instance), FieldError, JsonObject, MessageOrigin; parameters WebhookIdPath, TokenPath; responses NotFound, PayloadTooLarge, TooManyRequests, UnprocessableEntity; header Idempotent-Replay; security scheme bearerAuth; for the test world installApp, uninstallApp, suspendApp, resumeApp, putChannelBehavior, setSpaceOperator, createMessage, createThread, moveThread, archiveChannel, listChannelThreads.
  • docs/api/webhooks.md: 8 "Incoming webhooks".
  • ADRs 0007 (migration numbers, migrations.sha256), 0013, 0015, 0016, 0017, 0022, 0053, 0061.
  • docs/DECISIONS.md: D-LS-06, D-EX-02, D-API-01, D-W-100, D-W-103; section 10 for the format of new rows.
  • docs/security/threat-model.md: TB6, asset A5, table 5.11 "Apps and webhooks", T-TK-03 (theft of a webhook URL; tests AZ-M53 and SEC-T10), T-DS-01. docs/RISKS.md: RK-20 (token in the URL in proxy logs). No item of docs/OPEN-QUESTIONS.md concerns this ticket.
  • docs/observability/alerting.md 2 "Alert channel in the instance"; infra/observability/alertmanager/alertmanager.yaml (receiver bothy-ops with url_file and send_resolved); docs/observability/signals.md rows bothy.ratelimit.rejected and bothy.message.created (attribute bothy.message.origin).
  • docs/operations/configuration.md: webhook_in.max_body_bytes, webhook_in.formats_enabled, ratelimit.rl.webhook_in.per_minute, ratelimit.rl.webhook_in.burst, ratelimit.rl.public_auth.per_minute, ratelimit.rl.public_auth.burst, message.max_content_bytes, thread.title_max_chars, identity.last_active_granularity, server.public_url, server.internal_listen_addr, server.trusted_proxies, http.max_json_body_bytes, sync.idempotency_key_ttl, database.tx_retry_max, audit.deny_aggregate_window, auth.token_pepper_ref, authz.not_found_timing_tolerance.
  • M1 briefs: M1-10 (access log with the route pattern, TestLogsNeverContainSecretsHeadersOrRawPaths), M1-11 (httpserver.Internal, "later tickets may add stack-internal webhook routes"), M1-12 (pgtest, migrations/README.md), M1-13 (txn.Subscribe, txn.Publish, testdata/txn_registrations.golden), M1-17 (api/operations.yaml, routes.Operation), M1-18 (chain stages, (*Chain).Use, problem writer with {token} in instance, body limit per route, StageValidate and its errors[].code vocabulary, httpservertest), M1-19 (ratelimit.Limiter, key webhook_id for rl.webhook_in, the failed-credential hook of StageAuthenticate), M1-20 (idempotency.MarkOneTimeSecret), M1-21 (design note 1: the internal listener evaluates no forwarded headers), M1-24 (auditing.Recorder), M1-26 (kernel/textcheck), M1-28 (actor fields InstallationID and GrantedScopes, requirement atoms All, AppKind, Level, deferred entries and their golden list, authorize.Filter, the visibility row for incoming_webhook), M1-31 (response validator in services/server/test/apitest), M1-44 (identity/domain/apitoken, the authenticator rejects kind whi, one-time secret, last_used_at granularity), M1-53 (ChannelArchived{ChannelID, SpaceID, ArchivedAt}), M1-55 (chat.Threads.Info, D1 parent rule per channel kind), M1-56 (D1 flow, D2 body rules, D3 request hash, D4 origin, D5 mentions, D7 events, D8 createThread), M1-62 (moving a thread changes its channel).
  • M2 briefs: docs/briefs/M2-24-apps-install.md (port and use cases of apps), M2-02-channel-behavior-config.md (putChannelBehavior with free_response), M2-05-runs-state-machine.md (OnMessagePosted, step 1: return unless Origin = human), M2-19-agent-messages.md (the same pattern for origin = agent).
  • Code on the base branch: internal/apps (port, domain, app, adapters of M2-24), internal/messaging, internal/chat, internal/identity (Tokens, domain/apitoken, HMAC hasher), internal/authz, internal/kernel/{actor,authorize,auditing,errs,ids,idempotency,textcheck,txn,clock}, internal/platform/{httpserver,ratelimit,config}, api/routes, internal/wire.

Fixed requirements that apply

Owner's fixed build rules (apply verbatim):

  1. No MVP. New from scratch and directly in the final state; stages are only the order.
  2. Strictly modular: clear module and service boundaries, dependencies only through defined interfaces, no cross access.
  3. Documentation in code: every public module, class, function and interface with a docstring/doc comment (purpose, parameters, return value, errors, example where useful); comments explain the why, not the what. API documentation is generated from code (OpenAPI from the routes, dartdoc, Python API docs in MkDocs).
  4. Documentation in the repo: README per module/service, ADR for every architecture decision, CHANGELOG per release, keep the documentation site current. A change without matching documentation is not done.
  5. Quality gates (SonarQube, linters, type checking, security scanners, test coverage) block the merge.

Rules for every brief:

  1. Test first: write the failing test, run it and keep the red output for the report, then make it green, then clean up.
  2. No en dash and no em dash in any file you write; the text-rules pre-commit hook rejects them. Use a hyphen or rephrase.
  3. White label: examples and test data use example.com and the people Alex, Sam and Robin; no company, host or personal names, no paths under a home directory.
  4. Limits, windows and thresholds only through named configuration keys listed in docs/operations/configuration.md; code comments and docs name the key, never the number.
  5. Specification first: a change to behaviour, API, event, right or configuration updates docs/spec/ or docs/api/ in the same change. The core contract (docs/spec/00-glossary.md, 03-data-model.md, 04-sync-protocol.md, 05-actions.md) is not edited by this ticket.

Product requirements implemented here:

  1. createIncomingWebhook (webhook_in.create: manage on the channel, step-up, Audit yes, rl.admin, Idempotency-Key) creates a webhook for one channel that always writes as its app (incoming_webhooks.app_id) with write on the channel, optionally with a fixed thread. The answer carries the URL with the token exactly once; only token_hash (HMAC-SHA-256 with the server pepper) is stored; a replay of the same Idempotency-Key answers 409 conflict with reason: one_time_secret (11 8.1, 05 4.11, 03 incoming_webhooks, 08 12, ADR 0061, 06 7).
  2. A call is a POST with Authorization: Bearer <token> or with the token as the last path segment. An unknown webhook, a wrong token and a revoked webhook give the same 404 (11 8.1, R-EX-09, 06 1 and 3).
  3. Order of checks (11 8.4): rl.webhook_in (429 rate_limited), token (404 not_found), body up to webhook_in.max_body_bytes (413 payload_too_large), format switched on and valid (400 validation_failed), authorize of the app (404 not_found), then the message and 201 with message_id.
  4. Formats native and alertmanager only, switched by webhook_in.formats_enabled (11 8.2, R-EX-08). native with client_msg_id is idempotent: a repeat answers 200 with the original and Idempotent-Replay: true, other content 422 idempotency_key_reuse (06 7, 04 5.1). alertmanager gives one message per notification with state, count, alertname, a summary per alert from labels and annotations, and generatorURL and externalURL as text (11 8.2).
  5. Target thread: the fixed thread of the webhook, else thread_id of the body when the thread belongs to the channel, else a new thread with title (required in forums), else the main thread of the text channel (11 8.3).
  6. The message has origin = app and the app as author, resolves no mentions and never triggers an agent; content up to message.max_content_bytes; no attachments on this path (11 8.4, 03 incoming_webhooks, 07 2.10 and SR4, 05 5 point 1, R-EX-09; AZ-M53).
  7. Rate-limit class rl.webhook_in per webhook. The path token appears in access logs, traces and Problem.instance only as the route pattern /api/v1/hooks/{webhook_id}/{token} (05 3, 06 5 point 1, 19 SEC-LOG-3, 08 12).
  8. listIncomingWebhooks (webhook_in.read, manage, never a token) and revokeIncomingWebhook (webhook_in.revoke, manage, step-up, Audit yes; a second revoke has no effect, 06 2 "No effect") (05 4.11).
  9. Uninstalling the app revokes its incoming webhooks; suspending keeps them (11 4.3), and a post is refused while the installation is not active (07 4.1 app_not_installed).
  10. Senders inside the stack (Alertmanager) reach the same routes on the internal listener, in every role (16 4.2, 02 3.2).
  11. No JSON null in answers; every answer validates against openapi.yaml (ADR 0053).

Scope

All paths under services/server/ unless they start with docs/ or infra/. Names of M1 and M2 code below are the expected ones; where the merged code names them differently, follow the code (Boundaries).

Create:

  • migrations/<next free number>_apps_incoming_webhooks.sql: table incoming_webhooks exactly as 03: all columns with types, nullability and defaults; primary key id; foreign keys app_id -> apps(principal_id), channel_id -> channels(id), thread_id -> threads(id), created_by -> principals(id), each ON DELETE RESTRICT; unique token_hash; index (channel_id) WHERE revoked_at IS NULL; privileges for the application role as migrations/README.md (M1-12) prescribes; Down part for development. Then make migrations-lock.
  • internal/apps/domain/incoming/ (or the domain package layout M2-24 chose for apps): format.go (Classify, Design note 4), native.go (body rules), alertmanager.go (ParseAlertmanager, RenderAlertmanager(n Notification, maxBytes int) string, Design note 7), target.go (ChooseTarget, a pure function over the facts of Design note 5), hashctx.go (Design note 6c); testdata/alertmanager/*.json with the expected *.md.
  • internal/apps/app/incoming.go: use cases ListIncoming, CreateIncoming, RevokeIncoming, AuthenticateIncoming, PostIncoming; the subscriber OnChannelArchived(ctx context.Context, ev chat.ChannelArchived) error; revokeIncomingForApp for the uninstall use case.
  • internal/apps/adapters/postgres/queries/incoming_webhooks.sql and the regenerated sqlc code: ListActiveIncomingByChannel, InsertIncoming, GetIncomingForAuth (webhook row joined with the app's installation that is not uninstalled), LockIncomingForPost (FOR SHARE), RevokeIncoming, RevokeIncomingByApp, RevokeIncomingByChannel, TouchIncomingLastUsed (conditional on identity.last_active_granularity).
  • internal/apps/adapters/httpapi/incoming.go: strict handlers for listIncomingWebhooks, createIncomingWebhook, revokeIncomingWebhook, postIncomingWebhook and postIncomingWebhookWithBearer; incoming_stage.go: the middleware for the new chain stage (Design note 2).
  • test/integrations/ (new folder under services/server/test/): cron_path_test.go, alertmanager_path_test.go, azm53_test.go, end to end through the real server assembly of internal/wire; register the folder in architecture/modules.yaml the way the other folders under test/ are registered, if make arch asks for it (M1-05).
  • internal/wire/configkeys_apps.go (extend it if M2-24 or M2-26 created it): webhook_in.max_body_bytes (KindInt, minimum 1) and webhook_in.formats_enabled (KindStringList), defaults from the 11 table, chapter constant for 11-extensions.md in configchapters.go if missing; a start check in configchecks.go: every entry of webhook_in.formats_enabled is native or alertmanager and appears once, else a configuration error naming the key.

Change:

  • internal/apps port: interface IncomingWebhooks (02 4.3) with the use cases above and their DTOs; doc comments.
  • internal/apps/app uninstall use case of M2-24: revoke all active incoming webhooks of the app in the same unit of work (Design note 11).
  • internal/identity port Tokens (M1-44): NewIncomingWebhookToken(ctx context.Context) (IncomingWebhookToken, error) returning the secret in the redacting secret type of M1-44 plus its hash, and HashIncomingWebhookToken(ctx context.Context, raw string) ([]byte, error) returning ErrTokenMalformed without any database call for every value that is not a well-formed token of kind whi (format, checksum, kind). Both use domain/apitoken and the HMAC hasher with auth.token_pepper_ref; tests in identity. The authenticator keeps rejecting whi everywhere else (M1-44).
  • internal/messaging port: the create command of Messages (and the thread-with-first-message command of createThread) gets SkipMentions bool and HashContext string; zero values keep the M1-56 behaviour. With SkipMentions the mention scanner does not run, no mentions row is written and MessagePosted.MentionedIDs is empty; HashContext, when set, replaces the context part of the request hash of M1-56 D3. Tests in messaging.
  • internal/authz/domain/requirements.go: the entry of webhook_in.post changes from Deferred to All(AppKind, Level(write)) (Design note 3); remove it from the golden list of deferred entries; internal/authz/README.md.
  • internal/platform/httpserver: a new stage constant StageWebhookToken directly after StageRateLimit and before StageMaintenance; the golden test of the chain order; an override of the body limit of StageRoute per operation, used by the two webhook operations with webhook_in.max_body_bytes; README.md.
  • internal/wire: StageAuthenticate leaves requests of operations with action webhook_in.post anonymous (Design note 2); installs the apps middleware at StageWebhookToken; registers chat.ChannelArchived -> apps.OnChannelArchived after the existing subscribers and updates testdata/txn_registrations.golden; mounts the two webhook operations on the internal listener in every role (Design note 12); binds the new configuration keys and the port.
  • api/operations.yaml: move listIncomingWebhooks, createIncomingWebhook, revokeIncomingWebhook and postIncomingWebhook from later to enabled and add postIncomingWebhookWithBearer to enabled (sorted); regenerate the server code (M1-17).
  • docs/api/openapi.yaml, docs/api/webhooks.md, docs/spec/11-extensions.md, docs/spec/06-api.md, docs/spec/02-architecture.md, docs/spec/16-operations.md, docs/observability/alerting.md, docs/security/threat-model.md: as Design note 15 lists; regenerate the tables of 06 with make api-tables and the configuration reference with make config-reference.
  • READMEs of apps, messaging, identity, authz and internal/platform/httpserver; services/server/CHANGELOG.md under "Unreleased"; docs/DECISIONS.md section 10 rows D-M2-27-<n> for every ruling of Design note 16.

This ticket creates no new top-level make target, no durable or ephemeral event type, no job and no metric. It creates one operation (postIncomingWebhookWithBearer) and one schema (AlertmanagerAlert), both by ruling (Design note 16).

Out of scope

  • Outgoing webhooks, webhook_subscriptions, webhook_deliveries, the job apps.webhook_deliver, getWebhookKeys, G10 and AZ-M49 (M2-26).
  • Manifest, installation, scopes, app tokens and AZ-M32 to AZ-M34 (M2-24); this ticket only extends its uninstall use case.
  • The operations suspendApp and resumeApp: no plan row names them (M1-17 lists them for M2 without a ticket; plan gap reported). Design note 11 fixes what a suspended installation means for posts.
  • The one-step "incoming app" of 11 8.1 (an owner creates a minimal app together with the webhook): IncomingWebhookCreate has no member for it and no plan row owns it (spec gap reported). Until then an owner installs a minimal app through installApp (M2-24) first.
  • Commands and interactions of apps (M2-25); the reference apps for code hosting and ticket systems, which receive their webhooks at their own address (M2-49, M2-50).
  • One thread per alert group or any mapping from an external key to a thread (11 8.3: an app's task, or a later table proposed in a report).
  • The Alertmanager configuration of the stack, the alert channel of the instance, the test alarm and the watchdog (M7-20).
  • Console screens for incoming webhooks (M6-08).
  • Notifications and push for webhook messages (M3-17 to M3-19); search of their content (M3-13).
  • The canary scenario across all outputs (M1-69, in the closure; Tests say how this ticket extends it); leak and timing tests over all routes (M1-70); maintenance mode (M7-08): webhook_in.post has effect write, so it answers 503 maintenance there.
  • The trigger path of agents (M2-04, M2-05, M2-08): only used to prove that nothing triggers.
  • Changes to the core contract.

Boundaries

  • A new migration needs its line in migrations.sha256: run make migrations-lock after creating the file and before make check; never edit a merged migration (M1-01, ADR 0007).
  • Work in a dedicated git worktree on the branch feat/M2-27-webhooks-incoming; one writer per worktree.
  • Conventional Commits with the module as scope (for example feat(apps): ...) and a DCO sign-off on every commit (git commit -s); squash into meaningful commits before the merge (docs/quality/push-discipline.md).
  • Never push to main, never tag and never create remote repositories; the only remote steps are the branch push and the pull request of AGENTS.md.
  • Do not modify files outside the scope above, except the CHANGELOG, README and docs/DECISIONS.md entries named in this brief.
  • Follow the host rules supplied by the launcher (ports, machine rules).
  • Tests bind only 127.0.0.1 and port 0; PostgreSQL runs only as a throwaway container on a random port through pgtest (P-09, P-10). Without a Docker-compatible runtime the integration tests fail; they never skip.
  • Stop only processes you started, by PID. Temporary directories only via mktemp -d, removed at the end.
  • Never read or print secrets. No network calls except package downloads for pinned tools and Go modules.
  • Never lower a threshold, disable a gate or delete a test.
  • If the core contract lacks something this ticket needs, stop with Verdict: BLOCKED <reason> instead of building a deviation (P-11).
  • This brief was written before milestone M1 closed (D-W-100). Where the merged code of an M1 ticket names a package, type, function, file or make target differently from this brief, follow the code and list each difference in the report; where it behaves differently from the specification, stop with Verdict: BLOCKED <reason>.
  • The same holds for M2-02, M2-05 and M2-24: their merged names win over the expected names here; list each difference.
  • No real Alertmanager, Prometheus or other third-party service in tests: the Alertmanager payloads are fixtures built from the structure documented in 11 8.2 (evidence retrieved 2026-10-10) with white-label values. No model call of any kind.
  • Tokens never appear in logs, problem bodies, audit details, event payloads or test failure messages; tests build tokens at runtime, and every committed token-shaped value (openapi examples, fixtures) carries the marker EXAMPLE in its random part (CONTRIBUTING.md, M1-44).
  • Time only through kernel/clock; tests move a fake clock and never sleep for a window.
  • A mutation used to prove a red run is never committed.

Design notes

  1. Start check. make check is green on the fresh worktree, else Verdict: BLOCKED base branch red. Check before writing code that the base has (all are in the dependency closure; the check guards against a capability the merged code lacks, while a differing name follows Boundaries): internal/platform/ratelimit with rl.webhook_in keyed by the path value webhook_id and the failed-credential hook (M1-19); the tables channels and threads (M1-53, M1-55); createMessage, createThread and MessagePosted (M1-56); apps and app_installations with an uninstall use case (M2-24); the acceptance subscriber runs.OnMessagePosted (M2-05). A missing one ends the ticket with Verdict: BLOCKED dependency <id> missing.

  2. Routes and order of checks. Two operations serve posts, with the same action webhook_in.post, class rl.webhook_in, exemption client_id and request body: postIncomingWebhook on POST /api/v1/hooks/{webhook_id}/{token} (security: []) and postIncomingWebhookWithBearer on POST /api/v1/hooks/{webhook_id} (security: bearerAuth). Ruling for docs/DECISIONS.md: the Bearer variant of 11 8.1 and 10.3 is its own operation, because one operation cannot have two paths and the plan row asks for both. A request passes, in this order, which is the order of 11 8.4 placed into the chain of 02 9.1: a. Stages 1 to 6 as for every route (M1-18). StageRoute sets the body limit of these two operations to webhook_in.max_body_bytes instead of http.max_json_body_bytes. b. StageAuthenticate: for every operation whose action is webhook_in.post the identity authenticator is not called and the actor stays anonymous, whatever Authorization holds (the token of these routes is not an API token; M1-44 would reject kind whi with 401). c. StageRateLimit: rl.webhook_in keyed by the path value webhook_id (M1-19); exhausted gives 429 rate_limited with Retry-After, before any database access and for unknown IDs too. d. StageWebhookToken (new, Design note 2): token check; failure gives 404 not_found. e. StageMaintenance, StageIdempotency (the exemption client_id skips it), StageBody (413 payload_too_large naming webhook_in.max_body_bytes, 400 malformed_request), StagePreconditions, StageValidate (400 validation_failed, unknown members included), StageProof. f. Handler: classification and format switch (400 validation_failed, Design note 4), body rules per format, target thread (Design note 5), authorize (Design note 3, every denial gives 404 not_found), write (Design note 6), 201 with IncomingWebhookResult, or 200 with Idempotent-Replay: true on a replay. Ruling for docs/DECISIONS.md: the token check is a chain stage between the rate limit and the body stages, so it runs after rl.webhook_in and before any body is read or validated, exactly as 11 8.4 orders it; 02 9.1 gets the new step.

  3. Token check (StageWebhookToken, middleware in internal/apps/adapters/httpapi, installed by internal/wire). It acts only on operations with action webhook_in.post; others pass through. a. Token source: postIncomingWebhook takes the path value token and ignores Authorization; postIncomingWebhookWithBearer takes Authorization and accepts exactly the scheme Bearer with one token. A missing or other scheme counts as a wrong token. b. identity.Tokens.HashIncomingWebhookToken: a malformed value fails without a database call. Otherwise AuthenticateIncoming loads the webhook by the path value webhook_id with its app's installation that is not uninstalled (GetIncomingForAuth) and compares token_hash with hmac.Equal. Unknown ID, wrong hash, revoked_at set, no installation or an installation in state suspended all give the same 404 not_found (11 8.1, 4.3; 07 4.1 app_not_installed). A known and an unknown ID take the same path (one HMAC, one indexed lookup), so AZ-P4 timing holds. c. A failed check is a failed credential (02 9.1, 05 3): it takes one token from rl.public_auth of the client IP through the hook M1-19 gave StageAuthenticate. At the start of the stage, while that bucket of the client IP is empty, the stage answers 429 rate_limited without checking the token, as StageAuthenticate does for other credentials. A failed check writes no audit entry (it is authentication, not an authorize decision). d. Success replaces the anonymous actor with the webhook actor (Design note 3) and stores the verified webhook ID for the handler. The handler uses the path value only after checking it equals the stored ID.

  4. Webhook actor and rights. Ruling for docs/DECISIONS.md: the actor of a verified post is the app principal (ID = incoming_webhooks.app_id, Kind app) with InstallationID of its active installation, GrantedScopes = the installation's granted_scopes intersected with messages:write, and no TokenID, SessionID or DeviceID (the fields of M1-28; M2-24 fills the same fields for app tokens). Reason: a stolen URL may only post into its channel (T-TK-03); message.create and thread.create need exactly messages:write (05 2.6). a. Ruling for docs/DECISIONS.md: the requirement of webhook_in.post is All(AppKind, Level(write)) on the resource {kind: channel, id: incoming_webhooks.channel_id}; it needs no scope (05 2.6, already in M1-28's scope table). An anonymous principal gets unauthenticated (fail closed; 07 4.2 step 1 allows anonymous only for the right public, and this right is "public with token; the app needs write"). b. The use case calls authorize(webhookActor, webhook_in.post, channel) on every post, also on a replay (02 9.3 proof). Then the messaging use case authorizes message.create (and thread.create for a new thread) for the same actor, which brings the scope check, the read-only check (05 rule 1.8) and the archive rule (07 4.5). c. Every outcome other than allow of these three checks (not_found, forbidden with any reason including scope_missing, channel_read_only and channel_archived, unauthenticated) becomes 404 not_found with exactly the body and headers of an unknown webhook (11 8.4: decision "authorize app: message.create on the channel", branch "no" to 404). Internal errors stay 500 internal_error. d. Audit: denials of webhook_in.post are written by the authz layer (Audit denials, aggregated per audit.deny_aggregate_window, R-AZ-04); message.create has Audit no; an allowed post writes no audit entry.

  5. Formats and body rules. Ruling for docs/DECISIONS.md: the format is decided per request from the body, not stored per webhook. Reason: 03 incoming_webhooks has no format column (core contract), openapi has no format member, and one closed request schema must cover both shapes because rule/oneof-needs-discriminator forbids a oneOf without discriminator and the Alertmanager payload has no discriminator member. The request body stays one schema, IncomingWebhookPost, with the native members (content_md, title, thread_id, client_msg_id) and the Alertmanager members (version, groupKey, truncatedAlerts, status, receiver, groupLabels, commonLabels, commonAnnotations, externalURL, alerts) and no required list. a. Classify: a body with at least one Alertmanager member is alertmanager; it must contain no native member, else 400 validation_failed with the pointer of the first native member and errors[].code format_mixed. Every other body is native (8.2: native is the standard). b. A format not in webhook_in.formats_enabled gives 400 validation_failed with an empty pointer and errors[].code format_disabled. c. native (M1-56 D2 rules, pointers into this body): content_md required (/content_md, required) with at least one non-white-space character (empty); more than message.max_content_bytes UTF-8 bytes gives 413 payload_too_large with detail naming message.max_content_bytes; client_msg_id, when present, a UUIDv7 (/client_msg_id, not_uuid_v7), when absent a new server UUIDv7 (03 messages.client_msg_id); title follows the title rules of M1-55 (/title); thread_id is a UUID (schema). Content is stored byte for byte. d. alertmanager (ParseAlertmanager): version must be 4 (schema enum), status and each alerts[].status firing or resolved (schema enum), alerts at least one item (schema), each alert needs status and labels (/alerts/<i>/labels, required); every value inside groupLabels, commonLabels, commonAnnotations, labels and annotations must be a string (pointer to the member, invalid_type). The client message ID is always a new server UUIDv7: retries of Alertmanager are not deduplicated (8.2 makes only client_msg_id of native idempotent). e. Unknown members give 400 validation_failed with unknown_member in StageValidate (SEC-IN-1, W7); the label maps use JsonObject, which M1-18 exempts from the closed-schema check.

  6. Target thread (ChooseTarget, 11 8.3). Facts: the webhook (channel_id, thread_id), the channel kind and its main thread, and chat.Threads.Info of a fixed or named thread. In this order:

    1. Fixed thread: used while it belongs to the webhook's channel; thread_id and title of the body are ignored. Ruling for docs/DECISIONS.md: when the fixed thread was moved to another channel (M1-62), the post gives 404 not_found, because a webhook writes only into its own channel (11 8.1 "for a channel", T-TK-03).
    2. thread_id of the body (native only) when that thread belongs to the webhook's channel; an unknown thread or one of another channel falls through to the next rule, so nothing about other channels is revealed.
    3. title (native only) creates a new thread. Forum: a forum post through the createThread use case with the message as its first message (M1-56 D8). Text channel: ruling for docs/DECISIONS.md: the message is written into the main thread and the new thread starts from it (parent_message_id = the message), like auto_thread of 09 3.7; reason: in a text channel a thread grows from a message (01 4, M1-55 D1 requires the parent), and 8.2 says one message per call. The answer names the new thread in thread_id.
    4. Main thread of a text channel. In a forum without rule 1 to 3: 400 validation_failed, pointer /title, errors[].code required_for_forum_post; for alertmanager the pointer is empty (10.3: Alertmanager posts into a text channel or a fixed thread).

  7. Writing (PostIncoming, one unit of work). a. Lock the webhook row with FOR SHARE (LockIncomingForPost) and check again revoked_at IS NULL and that the actor is its app; else 404 not_found. A revoke, an uninstall and an archive update the row and therefore wait for posts in flight; after their commit no post writes (Design note 10). Deadlocks are retried by the unit of work (database.tx_retry_max). b. Choose the target (Design note 5), call authorize (Design note 3), then the messaging port with the webhook actor in the context, SkipMentions = true, Silent = false (notifications follow the channel settings, 11 8.4), no reply, no attachments, kind = text: createMessage for rules 1, 2 and 4; createThread with first message for rule 3 in a forum; createMessage into the main thread followed by createThread with parent_message_id of that message for rule 3 in a text channel. c. Ruling for docs/DECISIONS.md: HashContext = "webhook:" + webhook_id + "|thread:" + <thread_id of the body or empty> + "|title:" + <title or empty>, so the hash covers the request as sent (04 5.1 point 3) and a client_msg_id repeated through another webhook of the same app gives 422 idempotency_key_reuse instead of a replay of a message of another webhook. A replay (same author, client_msg_id and hash) answers 200 with Idempotent-Replay: true, the original message_id and the thread of the first answer (the thread whose parent_message_id is the message in the text-channel case), and publishes nothing. The messaging port tells the caller that its answer is a replay (M1-56 D1 step 5); on a replay the use case creates no thread. d. Post-condition: the channel_id of the written message equals the webhook's channel_id, else roll back and answer 404 not_found (a thread moved concurrently). e. MessagePosted carries Origin = app and AuthorKind = app; runs.OnMessagePosted returns at its first step (M2-05), so no run, no anchor, no trigger_state (SR4). Events are those of M1-56 D7 (message.created with actor {id, kind: app}; thread.created when a thread is created), in its order. f. After the commit, TouchIncomingLastUsed sets last_used_at only when it is empty or older than identity.last_active_granularity (ruling for docs/DECISIONS.md: the same rule as api_tokens.last_used_at, 03); outside the unit of work so concurrent posts never upgrade their share lock; a failure is logged without the token and never fails the request.

  8. Rendering of an Alertmanager notification (RenderAlertmanager, golden tests). Ruling for docs/DECISIONS.md: the layout below, so the expected message of a payload is fixed by test (11 8.2 lists the contents, not the layout; docs/observability/alerting.md 2 adds severity and runbook link). a. Header line **FIRING** <title> (<n> alerts) or **RESOLVED** ... from the top-level status; <title> is commonLabels.alertname, else groupLabels.alertname, else Alerts; <n> is the number of items in alerts plus truncatedAlerts, written 1 alert for one. b. A blank line, then one list item per alert in payload order: - **<firing|resolved>** <alertname>, severity <severity>: <summary>, where <alertname> is the alert's label alertname (else the title), , severity <severity> appears only with the label severity, and <summary> is the annotation summary, else description, else the remaining labels except alertname and severity as name=value sorted by name and joined by ,, else no summary. Continuation lines indented by two spaces: Runbook: <URL> from the annotation runbook_url and Source: <URL> from generatorURL. c. When truncatedAlerts is above zero or items were dropped (d), a last item - and <k> more alerts. d. A blank line and Alertmanager: <URL> from externalURL. e. Values from the payload: line breaks and other control characters become one space, then every ASCII punctuation character is escaped with a backslash (CommonMark), so a label cannot add formatting, links or images. A URL is written only when it is an absolute http or https URL with a host and without white space, < or >, as an autolink <URL> whose visible text is the target itself (SEC-OUT-1); otherwise its line is left out. The server never fetches any of these URLs (11 1 point 4, SEC-OUT-2). f. When the text exceeds message.max_content_bytes: drop alert items from the end (counting them into the last item of c), then the footer, then cut the rest at a UTF-8 boundary so that ... still fits. The result is never larger than message.max_content_bytes and always valid UTF-8.

  9. createIncomingWebhook (POST /channels/{channel_id}/incoming-webhooks). a. Load the channel (unknown gives 404); authorize(actor, webhook_in.create, channel): invisible gives 404 not_found identical to an unknown channel, read without manage gives 403 forbidden, no fresh step-up gives 401 step_up_required; tokens of agents and apps cannot do step-up (R-AZ-03). b. Body (IncomingWebhookCreate): name non-empty after trimming, without control or bidi characters (kernel/textcheck, SEC-IN-4; /name); app_id an app principal with an installation in state active, else 400 validation_failed with /app_id and errors[].code app_not_installed; the app must be allowed message.create on the channel, checked with the webhook actor of Design note 3 in filter mode (authorize.Filter, no audit), else /app_id with app_cannot_write (11 8.1: it always writes as an app with write on the channel; a manager sees the same through "Who has access?"); thread_id, when present, a thread of this channel (chat.Threads.Info), else /thread_id with not_in_channel, the same for an unknown ID. Ruling for docs/DECISIONS.md: these three codes. c. Unit of work: identity.Tokens.NewIncomingWebhookToken; insert the row with created_by = the actor; audit allow webhook_in.create on the channel with details {webhook_id, app_id, thread_id} (no name, never token or hash); idempotency.MarkOneTimeSecret(ctx) (M1-20). d. 201 with IncomingWebhookCreated: webhook and url = server.public_url joined with /api/v1/hooks/{webhook_id}/{token} (ruling for docs/DECISIONS.md: absolute, as the schema says "Full URL"; Bearer senders drop the last segment and send the token in the header). No event (05 section 6 has none for incoming webhooks), no rights change.

  10. listIncomingWebhooks: load the channel; authorize(actor, webhook_in.read, channel) (404 without read right, 403 forbidden without manage); the active webhooks of the channel ordered by created_at, then id, as IncomingWebhook without token or hash. Ruling for docs/DECISIONS.md: revoked webhooks are not listed (the partial index of 03 serves exactly this list; their history is in the audit log).

  11. revokeIncomingWebhook: load the webhook (unknown gives 404); resource {kind: incoming_webhook, id, ChannelID} (M1-28 visibility row); authorize(actor, webhook_in.revoke, ...); already revoked: 204 and an audit allow entry with no_change: true (06 2 "No effect"); otherwise UPDATE ... SET revoked_at = now() WHERE id = $1 AND revoked_at IS NULL, audit allow with {webhook_id, app_id}, 204. The next post with its token gives 404 not_found.

  12. Lifecycle of the app and the channel. a. Uninstall (M2-24 use case, 11 4.3): in the same unit of work RevokeIncomingByApp sets revoked_at on every active webhook of the app; the audit entry of app.uninstall stays the record. b. Suspend (11 4.3: "remain"): the rows stay; posts give 404 not_found while the installation is not active (Design note 2b); after app.resume the same URL works again. Ruling for docs/DECISIONS.md: suspend is an emergency stop for everything the app writes (11 4.3, R-EX-05, 07 4.1 app_not_installed). c. Ruling for docs/DECISIONS.md: apps.OnChannelArchived (the subscription that 02 4.3 lists for apps) sets revoked_at on the active webhooks of the archived channel in the archive transaction, because archiving is irreversible (05 has no unarchive) and such a webhook could never post again. No audit entry of its own; the archive is audited.

  13. Internal listener (16 4.2, 02 3.2, docs/observability/alerting.md 2 point 3). In every role (api, worker, all) internal/wire mounts the two post operations on the internal mux of M1-11 through the same chain as on the public listener, built for this mux; nothing else of the API is mounted there. On the internal listener the client IP is the peer address and forwarded headers are never read (R-SE-09, M1-21 design note 1). Ruling for docs/DECISIONS.md: both post variants are served there with identical behaviour; health stays as it is.

  14. Logs, traces and redaction. The access log and spans carry the route pattern (M1-10); the problem writer writes instance with the literal {token} (M1-18); Authorization is never logged (SEC-LOG-2). The use cases never log a token, a hash, the content or the name. docs/spec/16-operations.md 4.3 gets the rule that reverse proxies redact the path after /api/v1/hooks/{webhook_id}/ in their logs (RK-20, docs/api/webhooks.md 8 point 5).

  15. Configuration. webhook_in.max_body_bytes (body limit of the two post operations), webhook_in.formats_enabled (Design note 4b, start check of its values), message.max_content_bytes (native content, rendering limit), ratelimit.rl.webhook_in.* and ratelimit.rl.public_auth.* (registered by M1-19), identity.last_active_granularity, server.public_url, server.internal_listen_addr. Code comments and docs name the keys, never the values.

  16. Specification and API changes (same change, spec first; none touches the core contract): a. docs/api/openapi.yaml: IncomingWebhookPost as Design note 4 (members of both formats, thread_id added, no required, a description of both shapes and of the classification, two request examples); new schema AlertmanagerAlert (closed: status enum firing, resolved; labels and annotations as JsonObject; startsAt, endsAt as string with format: date-time; generatorURL, fingerprint as string); top-level version (string, enum 4), groupKey, receiver, externalURL (string), truncatedAlerts (integer, minimum 0), status (enum), groupLabels, commonLabels, commonAnnotations (JsonObject), alerts (array of AlertmanagerAlert, at least one item); postIncomingWebhook gets the responses 200 (replay with Idempotent-Replay) and 422 and a description of the order of checks and of the classification; new operation postIncomingWebhookWithBearer as Design note 1 with the same request body and responses and security: [bearerAuth: []]; the description of bearerAuth names incoming webhook tokens on that operation only; IncomingWebhookCreate describes the three rules of Design note 8b; IncomingWebhookCreated.url "absolute URL from server.public_url"; IncomingWebhookResult.thread_id describes the text-channel case of Design note 5; IncomingWebhookList "active webhooks only"; the examples of TokenPath and IncomingWebhookCreated.url become token-shaped bothy_whi_ values with the marker EXAMPLE and a valid checksum (ADR 0061). make api-lint stays green. b. docs/api/webhooks.md 8: both call variants, the classification and the two formats, the target thread rules, the replay answer, the internal listener for senders in the stack, and a cron example that creates one client_msg_id per run and reuses it on retries. c. docs/spec/11-extensions.md (English, like the chapter): 8.1 the Bearer path POST /api/v1/hooks/{webhook_id}; 8.2 the member names of openapi (content_md, title, thread_id, client_msg_id), the classification per request, the layout rule of Design note 7 by reference to docs/DECISIONS.md; 8.3 the rules of Design note 5 (fixed thread only in the own channel, text channel with title); 8.4 the flowchart and text with the token check as chain step after rl.webhook_in, every denial 404, and the internal listener. d. docs/spec/06-api.md: 1 table with both paths; 3 row "Incoming webhook" with token in the path or Authorization: Bearer; 5 point 1 stays (only {token} of postIncomingWebhook is a secret path parameter); the generated tables through make api-tables. e. docs/spec/02-architecture.md 9.1: step 7 notes that operations with action webhook_in.post stay anonymous; a new step after the rate limit: apps checks the token of the incoming webhook (11 8.4), failure 404 and one token from rl.public_auth. f. docs/spec/16-operations.md: 4.2 both post paths on the internal listener in every role; 4.3 the redaction rule of Design note 13. g. docs/observability/alerting.md 2 point 2: the format is recognised per request; the URL of createIncomingWebhook serves both formats; Alertmanager in the stack uses the same path on server.internal_listen_addr. h. docs/security/threat-model.md T-TK-03: add the test names of this ticket to the tests column.

  17. Rulings for docs/DECISIONS.md (section 10, one row D-M2-27-<n> each with reason): the Bearer operation (note 1); the token check as chain stage after the rate limit (note 1); the webhook actor and its scopes (note 3); the requirement of webhook_in.post and the uniform 404 (note 3); classification per request with one closed schema, format_mixed and format_disabled (note 4); the fixed thread only in its channel (note 5); the text-channel title rule (note 5); the hash context (note 6c); last_used_at granularity (note 6f); the Alertmanager layout, escaping, URL and truncation rules (note 7); app_not_installed, app_cannot_write and not_in_channel at creation and the absolute URL (note 8); active webhooks only in the list (note 9); suspend refuses posts (note 11b); revocation on channel archive (note 11c); both variants on the internal listener (note 12). The uniform 404, origin = app, no mentions and no attachments are already in 11 and 03.

Tests (write them first)

Unit:

  • internal/apps/domain/incoming: TestClassify (native, Alertmanager, a mixed body with the pointer of the first native member, empty object is native); TestNativeRules (empty and white-space content, client_msg_id of version 4 rejected, absent client_msg_id gives a server UUIDv7); TestParseAlertmanager (missing labels, a label value that is a number, an empty alerts list); TestRenderAlertmanager_Golden over testdata/alertmanager: firing with two alerts of one alertname with severity, summary, runbook and source; resolved with one alert; mixed alert names without commonLabels.alertname; truncatedAlerts above zero; a javascript: URL and a relative URL left out; a label value with *, [, a line break and @alex escaped; no summary but description; neither, so the remaining labels are listed; TestRenderAlertmanager_FitsMaxContentBytes (with a small limit: items dropped from the end and counted, then the footer, then a cut at a UTF-8 boundary ending in ...; output never above the limit, always valid UTF-8); TestChooseTarget (table: fixed thread in the channel, fixed thread moved away, body thread of the channel, body thread of another channel falls through, unknown body thread falls through, title in a forum, title in a text channel, nothing in a text channel, nothing in a forum for both formats); TestHashContext (stable, differs per webhook, per body thread and per title).
  • FuzzRenderAlertmanager and FuzzClassify with the golden fixtures as seed corpus (the corpus runs in go test): no panic, output within message.max_content_bytes, valid UTF-8, no unescaped [ from a label value.
  • internal/identity: TestIncomingWebhookToken_MintAndHash (kind whi, length and checksum per ADR 0061, hash equals the HMAC of M1-34 with the pepper, the secret prints [redacted]); TestHashIncomingWebhookToken_RejectsWithoutLookup (wrong checksum, kind agt, app or pat, wrong length: ErrTokenMalformed, repository fake counts zero calls); the authenticator test of M1-44 for whi stays green.
  • internal/messaging: TestCreate_SkipMentions (no scanner run, no mentions row, empty MentionedIDs, content unchanged); TestCreate_HashContextOverride (same request with another context gives 422 idempotency_key_reuse; zero value keeps M1-56 D3).
  • internal/authz: TestWebhookInPostRequirement: app with write on the channel gives allow; app with read only gives forbidden level_insufficient; app without grant gives not_found; anonymous gives unauthenticated; a person with write gives forbidden with the reason of the AppKind atom of M1-28 (subject_kind_not_allowed); the golden list of deferred entries no longer names webhook_in.post.
  • internal/apps/app: TestWebhookActor_ScopesLimitedToMessagesWrite: for an installation granted messages:write and messages:read, the webhook actor carries messages:write only, so authorize of message.read for it gives forbidden scope_missing, while message.create gives allow (Red Team "exceeding the scopes of an app").
  • internal/platform/httpserver: the golden chain order with StageWebhookToken after StageRateLimit; TestBodyLimitOverride (the override wins only for its operations and detail names its key).
  • internal/wire: TestConfigCheck_FormatsEnabled (unknown value and duplicate fail the start with the key in the message; empty list allowed); the golden of subscriptions with chat.ChannelArchived -> apps.OnChannelArchived.

Integration (PostgreSQL 18 via pgtest, real modules through internal/wire, HTTP through the real chain, every answer checked by the response validator of services/server/test/apitest; world of 07 11.1: Owner, Alex with G-1, Sam, Robin, space Support with operator Lotse and channels Allgemein and Eskalation, app Tracker installed through installApp with scope messages:write only and grant G-6 write on Support/Eskalation, a forum channel in Support; step-up through the M1-35 flow helper):

  • TestIncomingWebhooksMigration: columns, foreign keys with ON DELETE RESTRICT, unique token_hash, partial index on channel_id, as 03.
  • TestCreateIncomingWebhook: the Owner with step-up gets 201; url is absolute from server.public_url and ends with a bothy_whi_ token; the row holds the HMAC of that token and no part of it; one audit allow entry with {webhook_id, app_id, thread_id} only. Sam, given read with manage on Support/Eskalation by a grant of the Owner, also gets 201. Without step-up 401 step_up_required; Alex with trigger but without manage 403 forbidden; a person without read right 404 identical to an unknown channel; an app without installation 400 validation_failed with app_not_installed; an app without write there app_cannot_write; a thread of another channel and an unknown thread not_in_channel; a name with a bidi character 400.
  • TestCreateIncomingWebhook_OneTimeSecret: the same Idempotency-Key again gives 409 conflict with reason: one_time_secret and Idempotent-Replay: true; one row only.
  • TestListIncomingWebhooks: active ones in creation order without token or hash, a revoked one missing; 403 forbidden without manage; 404 without read right, identical to an unknown channel.
  • TestRevokeIncomingWebhook: 204 with step-up and audit; the next post 404; a second revoke 204 with an audit entry carrying no_change: true; unknown ID and a webhook of an invisible channel give identical 404.
  • TestPostIncomingWebhook_PathAndBearer: the same native body through both operations gives 201 with message_id and thread_id; the stored message has origin = app, author Tracker, kind = text, silent = false; one message.created with actor {kind: app} and the echoed client_msg_id; MessagePosted with Origin = app.
  • TestPostIncomingWebhook_NotFoundEquality: unknown webhook ID, well-formed wrong token, malformed token, revoked webhook, uninstalled app, suspended installation, app whose grant was revoked, archived channel, fixed thread moved to another channel (through moveThread; M1-62 is in the closure), Bearer with scheme Basic, Bearer missing: all 404 with identical status, body (ignoring correlation_id and instance) and headers (ignoring Correlation-Id, RateLimit, RateLimit-Policy), for both operations.
  • TestPostIncomingWebhook_OrderOfChecks (11 8.4): with rl.webhook_in of the webhook exhausted a wrong token gets 429 rate_limited; a wrong token with a body above webhook_in.max_body_bytes gets 404 (token before size); a valid token with that body gets 413 payload_too_large naming webhook_in.max_body_bytes; a valid token with an unknown member gets 400 validation_failed (unknown_member); a valid token with a switched-off format gets format_disabled; a valid token for an app that lost write gets 404; otherwise 201.
  • TestPostIncomingWebhook_FailedTokensCountAgainstPublicAuth: wrong tokens from one client IP use up rl.public_auth; then a valid token from that IP gets 429 rate_limited without a token check (the verifier fake counts zero calls); another client IP still gets 201.
  • TestPostIncomingWebhook_RateLimitPerWebhook: two webhooks; exhausting rl.webhook_in of one leaves the other at 201; RateLimit-Policy names rl.webhook_in; bothy.ratelimit.rejected grows with bothy.ratelimit.class = rl.webhook_in.
  • TestPostIncomingWebhook_MentionsNotResolved (R-EX-09): content with @alex and @lotse: no mentions row, mention_ids empty, content stored unchanged.
  • TestPostIncomingWebhook_ContentRules: content of exactly message.max_content_bytes bytes 201, one byte more 413 payload_too_large naming message.max_content_bytes; a mixed body format_mixed; client_msg_id of version 4 not_uuid_v7.
  • TestPostIncomingWebhook_TargetRules: fixed thread; body thread_id of the channel; body thread_id of another channel lands in the main thread; title in the forum gives thread.created then message.created and the post in a new forum thread; title in Support/Eskalation gives the message in the main thread and a new thread with parent_message_id of it, the answer naming the new thread; forum without title 400 validation_failed with required_for_forum_post.
  • TestPostIncomingWebhook_RevokeRace: a post and a revoke behind a barrier: either 201 before or 404 after; no message is written after the revoke committed; no deadlock reaches the client.
  • TestPostIncomingWebhook_ChannelPostCondition: a thread moved concurrently between target choice and write leaves no message outside the webhook's channel.
  • TestUninstallRevokesIncomingWebhooks (11 4.3): after uninstallApp every webhook of the app has revoked_at and posts give 404.
  • TestSuspendedInstallationRefusesPosts (11 4.3): with the installation suspended (through suspendApp if it is enabled on the base, else through the apps test fixture) posts give 404; after resuming the same URL gives 201; the row was never revoked.
  • TestChannelArchiveRevokesIncomingWebhooks: archiveChannel revokes the channel's active webhooks in its transaction; a webhook of another channel stays active.
  • TestInternalListener_IncomingWebhook (16 4.2): both post operations answer 201 on server.internal_listen_addr in roles api, worker and all; createMessage there gives 404; failed tokens sent with changing X-Forwarded-For values all count against rl.public_auth of the peer address.
  • TestPostIncomingWebhook_TokenOnlyAsPattern (06 5 point 1, SEC-LOG-2, SEC-LOG-3, plan row): with a token built at runtime, the captured access log and the exported spans hold /api/v1/hooks/{webhook_id}/{token} and never the token; a 404, a 400 and a 413 answer carry instance /api/v1/hooks/<webhook_id>/{token}; the Bearer variant logs no Authorization; audit details and event payloads hold no token.
  • TestLastUsedAtGranularity: two posts within identity.last_active_granularity write last_used_at once; after the window it moves.
  • TestAZ_P1_RouteCount: passes with the five operations.

End to end (services/server/test/integrations, the assembled server on loopback, the standard library HTTP client as sender):

  • TestAZM53_IncomingWebhookNeverTriggers (AZ-M53, SR4, T-TK-03): Support/Eskalation gets free_response = true through putChannelBehavior; a native post of Tracker's webhook mentioning @lotse gives a message with origin = app, no trigger_state, no runs row, no anchor message and no run.state_changed or agent.invocation event; as a control Alex writes in the same channel without mention and one run of Lotse is queued, which proves the trigger path is live.
  • TestCronPath_NativeIdempotentPerRun (11 10.4, exit criterion 9): a test job that creates one client_msg_id per run and posts native with a title per run: run 1 gives 201; repeating run 1 gives 200 with Idempotent-Replay: true, the same message_id and thread_id, and exactly one message.created; run 2 gives 201 with a new message; run 1's client_msg_id with other content gives 422 idempotency_key_reuse; run 1's client_msg_id through a second webhook of Tracker gives 422 idempotency_key_reuse.
  • TestAlertmanager_ExamplePayloadGivesExpectedMessage (plan row): the firing fixture posted to the path URL gives 201 and a stored content_md byte-identical to its golden .md; the resolved fixture gives a second message; both origin = app; posted to the internal listener the same; with a fixed thread both land there.

Contract and gates: make api-lint (the new operation, schema and examples), make api-checks (catalogue sync, idempotency exemption list, problem codes), make codegen-check (generated server code and the closed-schema check of M1-18).

test/canary (M1-69, SEC-T10) is in the closure: add the webhook token in the path and in Authorization, and a webhook message text, to its scenario. bothy.message.created (M1-68) is in the closure: assert one count with bothy.message.origin = app per post.

Commands that must pass, from the repository root:

  • cd services/server && go test -race -count=1 ./internal/apps/... ./internal/messaging/... ./internal/identity/... ./internal/authz/... ./internal/kernel/... ./internal/platform/httpserver/... ./internal/wire/... ./api/... ./test/integrations/...
  • make migrations-lock, make migrations-lint
  • make codegen-check, make arch, make api-lint, make api-checks, make api-tables, make register-check
  • make config-reference, make config-doc-check
  • make docs
  • make check

Acceptance criteria

  • incoming_webhooks exists exactly as in 03; listIncomingWebhooks, createIncomingWebhook, revokeIncomingWebhook, postIncomingWebhook and postIncomingWebhookWithBearer are enabled and registered; AZ-P1 passes.
  • AZ-M53: a webhook message mentioning the operator in a channel with free_response has origin = app and creates no run, while a human message in the same channel does.
  • The Alertmanager example payload gives the expected message byte for byte (golden), resolved notifications too; rendering stays within message.max_content_bytes.
  • Cron path of 11 10.4: a test job sends native with one client_msg_id per run; a repeat of the same run creates no second message, a new run creates a new one.
  • The path token appears in the access log, in traces and in Problem.instance only as the pattern of 06 "Errors"; the Bearer token never appears.
  • rl.webhook_in limits per webhook and runs before the token check; failed token checks count against rl.public_auth.
  • The order of 11 8.4 holds: 429, then 404 for the token, then 413 for webhook_in.max_body_bytes, then 400 for the format, then 404 for authorize, then 201.
  • Unknown webhook, wrong or malformed token, revoked webhook, suspended or uninstalled app and every authorize denial answer the same 404.
  • The token is shown once, only its HMAC is stored, and a replay of the creation gives 409 conflict with reason: one_time_secret.
  • The target thread follows 11 8.3 with the rulings of Design note 5; mentions are never resolved.
  • Revoke, uninstall and channel archive end posting; no post writes after their commit.
  • Both post operations work on the internal listener in every role.
  • The spec and API changes of Design note 15 and the rulings of Design note 16 are in the same change.
  • make check is green.

Definition of done

  • Code and tests as specified; doc comment on every public symbol (purpose, parameters, return value, errors, example where useful); comments explain why.
  • READMEs of apps, messaging, identity, authz and internal/platform/httpserver updated (port IncomingWebhooks, the token check stage and its place in the chain, the webhook actor, SkipMentions and HashContext, the new identity token functions, the requirement of webhook_in.post, the subscription to chat.ChannelArchived, the internal listener routes).
  • Rulings of Design note 16 recorded in docs/DECISIONS.md (one row each with reason).
  • services/server/CHANGELOG.md entry under 'Unreleased'.
  • An ADR only if you take an architecture decision beyond this brief (MADR 4 as in docs/adr/0000-template-madr-4.md, headings and text in English, next free number, make adr-index).
  • make check green.
  • An independent code review over the diff, then the red team gate (docs/quality/red-team-gate.md, verdict JA or NEIN); on NEIN fix with a test per finding and have the delta checked again. Record both verdicts.
  • Land the change as AGENTS.md ("How work lands on main") describes: a pull request from this branch, squash-merged only when review, red team and make check, ci-ok and the review threads are green; then git worktree remove.

Report

Write to {{REPORT_PATH}}:

  1. First line exactly Verdict: DONE or Verdict: BLOCKED <reason>.
  2. Commits (SHA and subject), changed files with line counts, test results with counts (the red run before the change and the green run after), the summary of make check.
  3. Review and red team verdicts and the findings fixed.
  4. What is not proven.
  5. Follow-ups that could not be done in this scope, each with its reason, and spec gaps found.

Then write {{DONE_PATH}} with exit=0 sha=<sha> or exit=1 <reason>.