M1-65: Gateway hub, delivery, catch-up and revocation¶
Status: ready | Depends on: M1-45, M1-64 | Milestone: M1
Goal¶
After this ticket every process with role api delivers each durable event of the event log to
exactly those gateway connections whose principal may receive it at delivery time, in seq order,
without ever blocking on a slow reader. A connection that identifies or resumes with a cursor
catches up page by page over the WebSocket and then switches to live delivery without gap,
duplicate or interleaving. Revoking a grant, session, device, principal or token takes effect on
open connections immediately, also across processes, and shutdown closes every connection with
4013. This is the delivery half of B8 and the guarantee "permissions apply at every delivery"
(04 "Goals", items 1, 3 and 4).
Read first¶
docs/PLAN.md: section "Way of working per ticket"; section "M1 Foundation" with the rows M1-63, M1-64, M1-65, M1-66, M1-67, M1-68, M1-70, M1-72 and the list "Known contradictions that M1 resolves".docs/spec/04-sync-protocol.md: "Two kinds", "Envelope", "Frames on the WebSocket", "Lifecycle", "Close codes", "Catch-up" with "Cursor", "Flow", "resync_required" and "REST variant", "Visibility on permission changes" (all subsections), "Delivery guarantee", "Ordering", the sequence diagrams "Send a message, online", "Reconnect with catch-up" and "Access removal during an open connection", "Decisions of this chapter" (R-SY-02, R-SY-03, R-SY-04, R-SY-14, R-SY-15, R-SY-16), "Configuration values of this chapter".docs/spec/02-architecture.md: "Cross-cutting ports (internal/kernel)", "Allowed dependencies", "Event types and WebSocket ops per module", "Layers within a module", "Gateway", "Event path" ("Log tail per api process", "Permission cache", "Hub and delivery"), "Failure of a dependency", "Panics", "Health", "Shutdown", "Configuration values of this chapter".docs/spec/05-actions.md: "Binding rules" (rules 1, 2, 4, 7), "Decision", "Durable event types and receive action", "WebSocket ops and action". Action ids:gateway.connect,events.catch_up,grant.revoke,session.revoke,device.revoke,user.disable,token.revoke,agent.disable.docs/spec/07-permissions.md: "Flow" (pseudo-code ofauthorize), "Permission state (snapshot)", "Validity of the cache", "Delivery filter per recipient and immediate effect" (both subsections), "Required tests" (AZ-P9), test matrix rows AZ-M40 and AZ-M41.docs/spec/08-identity.md: "Refresh tokens: rotation, families, reuse", "Lost device", "Disabling a person".docs/spec/03-data-model.md: tableevent_log; "Transaction rules" ("Notifying the gateway processes", "Sync epoch and restore").docs/api/asyncapi.yaml: operationssend.dispatch,send.cursor,send.catchup.complete,send.resync_required;x-close-codes.docs/api/openapi.yamlschemasEventEnvelope,GatewayCursor,GatewayCatchupComplete,GatewayResyncRequired,AccessChangedPayload,AccessTarget,SessionRevokedPayload.- ADRs:
docs/adr/0008-*,0009-*,0017-*, and 0052 (one event envelope) and 0053 (no JSON null), written by M1-16. - Diagrams:
docs/architecture/flow-event-path.svg,seq-3-reconnect.svg,seq-4-revoke-grant.svg. docs/security/threat-model.md: "Gateway (WebSocket)", register rows T-AZ-03 and T-DS-01.- Code you build on:
internal/gateway(M1-64: endpoint, sessions, handshake, test client),internal/sync(M1-16: appender,Tail; M1-63:CatchUp,Snapshot,listEvents),internal/authz(M1-28: snapshot cache andAuthorize; M1-29:access.changed),internal/identity(M1-34, M1-44, M1-45: sessions, tokens, revocation,session.revoked),internal/kernel/*,internal/platform/lifecycleandinternal/platform/health(M1-11),internal/platform/pg/pgtest(M1-12),internal/wire. docs/quality/gates.md,CONTRIBUTING.md,services/server/internal/gateway/README.md.
Fixed requirements that apply¶
- No MVP. New from scratch and directly in the final state; stages are only the order.
- Strictly modular: clear module and service boundaries, dependencies only through defined interfaces, no cross access.
- Documentation in code: every public module, class, function and interface with a docstring/doc comment (purpose, parameters, return value, errors, example where useful); comments explain the why, not the what. API documentation is generated from code (OpenAPI from the routes, dartdoc, Python API docs in MkDocs).
- Documentation in the repo: README per module/service, ADR for every architecture decision, CHANGELOG per release, keep the documentation site current. A change without matching documentation is not done.
- Quality gates (SonarQube, linters, type checking, security scanners, test coverage) block the merge.
- Test first: every behaviour change has a test that was red before the change (record the red run in the report), then green, then clean up.
- No en dash or em dash (U+2013, U+2014) in any file you write; the
text-ruleshook rejects them. - White label: no company, person, host or domain names except the examples Alex, Sam, Robin,
example.comand the spaces Support, Marketing, Engineering, Research; no paths under/Users. - Limits, windows and intervals only through named keys from
docs/operations/configuration.md; no literal numbers in code or docs that a test does not hold. - Permissions apply at delivery time. A gateway process updates the permission cache of a
principal when its
access.changedarrives in log order and checks every later event with the new state (04 "Basic rule", R-SY-02; 02 "Permission cache"). - Every
dispatchpassesauthorize(recipient, receive_action, resource)first; personal events (audience = self) go only to the audience principal (05 "Durable event types and receive action"; 07 "Delivery filter"). - The hub never blocks on a connection; a connection whose queue exceeds
sync.outbound_queue_maxis closed with4014without delaying others (02 "Gateway", "Hub and delivery"; 04 "Flow" step 5). - Catch-up in pages of
sync.catchup_page_size;cursorframes for skipped events at the latest aftersync.cursor_advance_everyand at the end of every page; live events buffered and sent aftercatchup.complete {head_seq}inseqorder, never mixed (04 "Cursor", "Flow"). resync_required {reason, head_seq, sync_epoch}instead of events for the reasonsretention,too_far_behindandcursor_ahead(04 "resync_required"). The epoch check (epoch_changed) runs once atidentifyandresumein M1-64 throughsync.CatchUp.Check(M1-63), not here.- After
access.changednothing more from removed targets: buffered and queued frames of those targets are discarded (04 "What the client receives and does on access removal", items 1 and 2). - Revocation of a session, device, principal or token closes the affected connections with
4012(04 "Revocation of a session, device or principal"; 08 "Lost device", "Disabling a person"). - Shutdown of role
apicloses all connections with4013 server_restart(02 "Shutdown", step 3; R-AR-17). - AZ-P9: after the commit of a revocation in process A, nothing that depends on the revoked permission is allowed or delivered by process B (07 "Required tests").
- A panic in the log tail or the hub ends the process with an error code; a panic on one
connection closes it with
4000(02 "Panics").
Scope¶
Follow the package layout that M1-64 established in internal/gateway. Where M1-64 left no
place, use this layout (it satisfies 02 "Layers within a module"):
internal/gateway/domain/: pure types and functions: connection phase (catching_up,live,closing), the bounded outbound queue with scope metadata and purge, the candidate rule of the hub, cursor bookkeeping. No I/O.internal/gateway/app/:Hub(indices,Dispatch), the per-connection catch-up runner (catchup.go), revocation handling (revocation.go), shutdown (shutdown.go). Imports only its own port and domain,internal/kernel/*and the port ofsync(allowed edge in 02 "Allowed dependencies").internal/gateway/adapters/ws/(from M1-64): the writer goroutine drains the queue with deadlinegateway.write_timeout; the handshake registers the connection in the hub before it readshead_seq(see "Connection phases").internal/kernel/authorize/: addObserveto theAuthorizerinterface if M1-28 did not, the typeLogEvent, and the context markerDelivery(ctx), which implies the filter modeFilter(ctx)of M1-28 (no denial audit, not counted).internal/authz/: implementObserveand the delivery mode ofAuthorize(observe.go, tests), includingthread.moved(this ticket owns the cache side of moves; M1-62 builds the move itself, 04 "Basic rule").internal/kernel/actor/andinternal/identity/:Revalidate(see "Revocation") if theAuthenticatorhas no equivalent yet.internal/sync/port: expose what M1-63 built so the gateway can reuse it without copying: the resync check, a catch-up page with a fixed upper bound, and the per-event admission function (signatures below). Do not change REST behaviour oflistEvents.internal/wire/: wireTail->Authorizer.Observe->Hub.Dispatchin one goroutine per process, register the shutdown step and the liveness check.- Docs:
services/server/internal/gateway/README.mdandservices/server/internal/authz/README.md(English, like the existing files),docs/spec/02-architecture.mdtable "Cross-cutting ports" (already namesDeliveryandRevalidate; change it only where the build proves it wrong),docs/security/threat-model.mdrow T-AZ-03 (add the new test names),services/server/CHANGELOG.mdunder "Unreleased",docs/DECISIONS.md(rulings listed below).
Interfaces (Go pseudo-code; keep existing names where earlier tickets already chose them):
// internal/kernel/authorize
type LogEvent struct {
Seq int64
Type string // event type as in docs/spec/actions.yaml
PermissionVersion int64
Audience ids.ID // zero unless audience = self
Scope Scope // space, channel, thread ids; absent members are zero
Resource ResourceRef // kind, id, version
AccessChanged *AccessDiff // decoded payload of access.changed: added, removed, changed
}
type Authorizer interface {
// existing: Authorize, Scope
// Observe applies one durable event to the process-local permission cache. The log tail calls
// it in seq order, before the hub sees the event.
Observe(ctx context.Context, ev LogEvent)
}
// Delivery marks ctx as a delivery check (hub, gateway catch-up): cached state is validated
// against the Observe floor instead of reading sequence_counters, and it implies Filter(ctx) of
// M1-28: denials are filters, not requests, so they write no audit entry.
func Delivery(ctx context.Context) context.Context
// internal/kernel/actor (only if no equivalent exists)
// Revalidate re-checks an authenticated actor without a new proof: session not revoked,
// device active, principal active, API token not revoked. Access-token expiry stays with the
// reauth flow of M1-64 (4011). Returns an errs value of kind unauthenticated.
Revalidate(ctx context.Context, a Actor) error
// internal/sync port (names as M1-63 chose them; add what is missing)
Check(ctx, a actor.Actor, cursor int64) (head int64, resync *Resync, err error) // retention, too_far_behind, cursor_ahead
Page(ctx, a actor.Actor, after, upTo int64, limit int) (Page, error) // admitted envelopes in seq order, LastChecked
Admit(ctx, a actor.Actor, ev Envelope) (bool, error) // receive action and audience; the single admission rule
// internal/gateway/app
func (h *Hub) Register(c *Conn) // before head_seq is read
func (h *Hub) Dispatch(ctx context.Context, ev sync.Envelope) // tail goroutine only; never blocks
func (h *Hub) CloseAll(code int, reason string) // shutdown
Out of scope¶
- Focus, typing,
EphemeralPublisherand every ephemeral event (M1-66). - Frame conformance harness G9 (M1-67); metrics (M1-68; do not add instruments here).
- Agent consumers: durable cursor,
ack, redelivery,4015,agent_consumer_cursors(M2-07); invocation tokens andruns.Delivery(M2);run.cancel_requestedhandling (M2). - The epoch check: it runs once at
identifyandresumein M1-64 throughsync.CatchUp.Check(M1-63); the epoch only changes through a restore (M7-04). cluster.mode = multiandNOTIFY ephemeral(M7-17).- Load tests and
make perftargets (M1-72); leak tests (M1-70). - Changes to the REST behaviour of
listEventsorgetSnapshot(M1-63).
Boundaries¶
- Work in a dedicated git worktree on the branch
feat/M1-65-gateway-delivery. - Conventional Commits with the module as scope (for example
feat(gateway): ...,feat(authz): ...) and DCO sign-off (git commit -s). - Never push, tag, open pull requests or create remote repositories.
- Do not modify files outside the scope above, except the docs and changelog entries named here.
- Follow the host rules supplied by the launcher.
- Stop only processes you started, by PID. Temporary directories only via
mktemp, removed at the end. Tests bind127.0.0.1and port 0 only (P-10); PostgreSQL runs as a throwaway container throughpgtest(P-09). - Never read or print secrets. No network calls except package downloads for pinned tools.
Design notes¶
Event path (one goroutine per process, 02 "Log tail per api process")
sync.Tailreadsevent_loginseqorder. For each event: firstAuthorizer.Observe(ev), thenHub.Dispatch(ev). Nothing else consumes the tail in M1.Dispatchmust return without waiting on any connection: it only computes recipients, enqueues pre-encoded frames and schedules closes.- A panic in this goroutine is not recovered: the process exits with an error code (02
"Panics"). Register a liveness check with
internal/platform/healththat fails once the tail or hub goroutine has exited, so/healthzreflects 02 "Health".
Permission cache in delivery mode (internal/authz)
Observeonaccess.changedwith audience P: setfloor[P] = ev.PermissionVersionand drop the cached snapshot and revocation state of P.Observeonchannel.created,channel.updated(carriesread_only),channel.archived,space.archived,thread.moved(04 "Basic rule"): advance a process-wide structure generation; cached snapshots and resolved resource paths from an older generation are recomputed on next use. Reason: these events are rare, and a global generation is correct without knowing every affected principal. All other types: no-op.- In
Delivery(ctx)a cached snapshot of P is valid if its version is at leastfloor[P], its generation is current and the clock is beforevalid_until(07 "Validity of the cache"). A recomputed snapshot reads current database state, which can only be newer than the log position; that is allowed (permissions at delivery time). - REST mode is unchanged: it keeps reading
permission_versionper request (R-RE-02). - Delivery checks write no audit entry for denials (they are filters, 07 4.2;
Delivery(ctx)impliesFilter(ctx)of M1-28, whichlistEventsin M1-63 uses). Requests and ops keep R-AZ-04. - A delivery decision
unauthenticatedfor a connection means its credential was revoked: close that connection with4012.
Hub indices and candidates (02 "Hub and delivery")
- Indices: principal -> connections; space -> connections whose principal sees the space
(
visible_space_idsfromAuthorizer.Scope); channel -> connections (visible_channel_ids); instance-wide set (owner sessions, grants on the instance). - Candidates of an event: audience set -> connections of that principal; else
space_idset -> space index of that space, channel index ofchannel_id, instance-wide set; else (instance level, for exampleinstance.updated,user.updated,agent.updated) -> all connections. - The candidate set is only a performance filter and must be a superset of the allowed set; the
truth is
sync.AdmitinDelivery(ctx)for each candidate. A new channel in a visible space is reached through the space index without a new permission version (02 "Additions to the core contract" point 4). - Recompute the index entries of P's connections from
Authorizer.Scope(delivery mode) when P'saccess.changedis processed. - Encode each event once (
{"op":"dispatch","d":<envelope>}with the no-null encoder of M1-18) and enqueue the same bytes for every recipient. M1 has no delivery-only envelope fields.
Per-connection queue
- Bounded by
sync.outbound_queue_maxframes. Entries carry: frame bytes, class (dispatchorcontrol),seqif any, scope ids andoccurred_at. Purge(removed)drops queueddispatchentries whose channel or space is in the removed targets of anaccess.changed; control frames (cursor,catchup.complete,error,heartbeat_ack) are never purged.- Overflow: mark the connection
closing, drop its queue and close it with4014from the writer side. The writer writes each frame with deadlinegateway.write_timeout; a write that misses the deadline also closes with4014(04 'Close codes', 02 'Gateway').
Connection phases and catch-up (04 "Flow")
- Registration: the handshake calls
Delivery.Register(M1-64) before it readshead_seqforreadyorresumed(04 'Lifecycle' step 4); this ticket's implementation puts the connection into the hub in phasecatching_up. Call the value H. Every event withseq > Hthe tail delivers from now on is either buffered or sent; every event withseq <= His covered by the snapshot or the catch-up. - No cursor in
identify: no catch-up frames; switch tolivewith base H immediately afterready(04 "Lifecycle" step 4: catch-up only when a cursor was sent). - Cursor c (from
identifyorresume):sync.Check(c). On a resync reason sendresync_required {reason, head_seq: H, sync_epoch}and switch tolivewith base H (the connection stays open, 04 'resync_required'). Otherwise read pagesafter = c,upTo = H,limit = sync.catchup_page_sizeand enqueue each admitted event asdispatch. cursor {seq}frames: at the end of every page with the page's last checkedseq(equal to H after the last page), and in addition whenever the number of skipped sequence numbers since the last frame that carried aseqreachessync.cursor_advance_every.- End:
catchup.complete {head_seq: H}; then, atomically with respect toDispatch, run the live buffer throughAdmitinseqorder (onlyseq > H), enqueue the admitted ones and switch tolive. Admission at flush time applies the newest permissions, so events of a target removed during the catch-up never go out. - The live buffer during catch-up is bounded by
sync.outbound_queue_max; overflow closes with4014. The catch-up runner itself waits for queue space (it blocks only its own goroutine and stops when the connection closes). - A failing page read (database unavailable) is retried with backoff capped at
database.listen_reconnect_backoff_capwhile the connection is open; the gateway keeps connections and delivers nothing meanwhile (02 "Failure of a dependency").
Live cursor frames (04 "Cursor", R-SY-03)
- The hub counts processed events. Every
sync.cursor_advance_everyprocessed events, each live connection whose lastseq-carrying frame is older than the hub position getscursor {seq: hub position}. Not being a candidate implies not visible, because the candidate set is a superset.
Revocation
- On P's
access.changed, before it is dispatched: (a)Revalidateevery connection of P and close failures with4012; (b) recompute P's index entries; (c) purge the queues and live buffers of P's connections for theremovedtargets; (d) dispatchaccess.changedthrough the normal path, inseqorder (it must not overtake earlier frames, 04 "Delivery guarantee" item 2). - On
session.revoked(audience P, payloadsession_id): close every connection whose actor carries that session id with4012; it is delivered to P's other connections whose credentials are still valid. user.disable,device.revoke,session.revoke,token.revokeandagent.disableall raise the permission version and writeaccess.changed(05 column "Permissions"), so the recheck in (a) covers every revocation even without asession.revoked.
Agents and apps in M1
- They are delivered like people, by their own grants and the operator rule. Without a cursor
they are live from H. Durable consumer cursors,
ackand redelivery are M2-07.
Shutdown (02 "Shutdown", role api and all)
- Register a lifecycle step after
http.Server.Shutdownand before the tail stops:Hub.CloseAll(4013, "server_restart"), reject newidentifyandresumewith4013, and wait for the writers to flush close frames within the remainingshutdown.total_timeoutbudget (each write still bounded bygateway.write_timeout). Saving agent cursors is M2-07.
Configuration keys used (defaults in their chapter tables): sync.catchup_page_size,
sync.catchup_max_events, sync.event_retention, sync.cursor_advance_every,
sync.outbound_queue_max, sync.ws_max_frame_bytes, sync.tail_batch_size,
sync.log_poll_interval, gateway.write_timeout, database.listen_reconnect_backoff_cap,
authz.snapshot_cache_max_entries, shutdown.total_timeout.
Ruling to record in docs/DECISIONS.md (with its reason): structure events invalidate all
cached snapshots through a generation counter. The other decisions of this ticket are in the
specification: the connection stays open after resync_required (04 7.3), a missed write
deadline closes with 4014 (04 6.4), delivery checks write no denial audit (07 4.2, R-AZ-04), and
AZ-P9 runs with two independent server instances in one test binary (07 "Required tests").
If a change to 04 or 05 (core contract) turns out to be necessary, stop with
Verdict: BLOCKED (P-11). Changes to 02 named above are allowed.
Tests (write them first)¶
Unit tests with fakes (fake Authorizer, fake sync port, a writer that blocks until released,
the kernel/clock test clock):
TestQueue_BoundedAndPurgeByScope,TestQueue_ControlFramesSurvivePurge.TestHub_DispatchNeverBlocksOnSlowConnection: one writer blocked,Dispatchreturns within a test deadline, the other connection receives every frame.TestHub_SlowConsumerClosedWith4014: setsync.outbound_queue_maxlow in the test config.TestHub_PersonalEventsOnlyToAudience.TestHub_CandidatesSupersetOfAllowed: property test in random worlds (principals, groups, grants, spaces, channels, events) with a seededmath/rand/v2generator; compare hub recipients with brute-force admission over all connections; print the seed on failure.TestHub_UnauthenticatedDecisionCloses4012.TestHub_LiveCursorFrameAfterAdvanceEvery.TestCatchUp_PagesCursorFramesAndComplete: about 2.5 pages, half of the events invisible.TestCatchUp_LiveEventsBufferedThenFlushedInSeqOrder: events committed while pages are read; assert the frame stream isdispatch/cursor...,catchup.complete, buffered events, and theseqof alldispatchframes strictly increases with no duplicate.TestCatchUp_LiveBufferOverflowCloses4014.TestCatchUp_ResyncReasons: tableretention,too_far_behind,cursor_ahead; the connection stays open and receives later live events.TestCatchUp_NoCursorNoCatchupFrames.TestAuthzObserve_AccessChangedRaisesFloor,TestAuthzObserve_StructureEventInvalidates,TestAuthzDelivery_DenialWritesNoAudit(ininternal/authz).
Integration tests against PostgreSQL 18 with all real modules, using the in-process server and WebSocket test client from M1-64 and the step-up test authenticator of the identity tickets:
TestGateway_ReconnectWithCatchUp: sequence diagram 04 "Reconnect with catch-up".TestGateway_RevocationDuringOpenConnection: sequence diagram 04 "Access removal during an open connection" without the agent part (M2): the Owner revokes Sam's grant on channel Support viarevokeGrant; Sam receivesaccess.changedwith Support inremoved; frames for Support queued before it (writer held) are not written; a message posted to Support afterwards does not reach Sam; Sam still receives events of Marketing.TestGateway_RevocationCloses4012: table overrevokeSession,revokeDevice,disableUser,revokeToken(agent API token) anddisableAgent; the affected connections close with4012; in the session case a second session of the same person stays open and receivessession.revoked.TestGateway_AZP9TwoInstances: Sam connected to instance B; the Owner revokes on instance A; later events of the channel do not reach Sam on B, and a REST read on B returns404.TestGateway_NewChannelInVisibleSpaceDelivered.TestGateway_EchoToAuthor:message.createdwithclient_msg_idreaches the author's own connection.TestGateway_ShutdownClosesAllWith4013: three clients receive4013; shutdown ends withinshutdown.total_timeout.TestHealth_HubStoppedFailsHealthz.BenchmarkHubDispatch(many connections, one event): report the value; the gate is M1-72.
Commands that must pass:
cd services/server && go test -race ./internal/gateway/... ./internal/authz/... ./internal/sync/... ./internal/identity/... ./internal/wire/...
make test
make check
Acceptance criteria¶
- Every durable event reaches exactly the connections for which
Admitallows it at delivery time; personal events only the audience (unit and property tests green). - Catch-up sends pages with
cursorframes, thencatchup.complete {head_seq}, then the buffered live events inseqorder; no gap, no duplicate, no interleaving. -
resync_requiredwithretention,too_far_behindandcursor_aheadcarrieshead_seqandsync_epoch; the connection stays live. - A slow reader is closed with
4014while other connections keep receiving;Dispatchnever waits on a connection. - After
access.changedno frame of a removed target is written, including frames queued or buffered before it. - Revoking a session, device, person, API token or agent closes exactly the affected
connections with
4012. - AZ-P9 with two instances is green.
- Sequence diagrams 04 "Reconnect with catch-up" and "Access removal during an open connection" run as integration tests.
- Shutdown closes every connection with
4013withinshutdown.total_timeout;/healthzfails when tail or hub stop. - Delivery denials write no audit entries; request denials still do.
-
make checkgreen.
Definition of done¶
- Code and tests as above; doc comment on every public symbol (purpose, parameters, return value, errors); comments explain why.
- Module READMEs of
gatewayandauthzupdated (status, ports, actions, config keys, behaviour on revocation and shutdown); 02 table "Cross-cutting ports" updated; threat model row T-AZ-03 lists the new tests; rulings indocs/DECISIONS.md;services/server/CHANGELOG.mdentry under "Unreleased". No new ADR is expected; write one if you take an architecture decision beyond this brief. make checkgreen.- Independent review (
make reviewplus a code review over the diff) and the red team gate (docs/quality/red-team-gate.md) over the diff, verdictJAorNEINrecorded in the report. OnNEIN: fix, add a test per finding, re-check the delta. - Merge locally into the base branch only if review and red team are green and
make checkis green; thengit worktree remove.
Report¶
Write to {{REPORT_PATH}}: first line Verdict: DONE or Verdict: BLOCKED <reason>; then
commits, files with line counts, test results with counts (including the red runs before the
fixes), what is not proven, and follow-ups that could not be done in scope with the reason. Then
write {{DONE_PATH}} with exit=0 sha=<sha> or exit=1 <reason>.