04 Event and sync protocol¶
Status: draft M0, integrated after the feedback of all chapters. Normative for the gateway
(AsyncAPI 3.0), the REST catch-up interface (OpenAPI 3.1), the clients and the agent adapters.
All paths are under the prefix /api/v1 (06-api.md, R-API-01).
Terms: 00-glossary.md. Tables: 03-data-model.md.
Actions and event types: 05-actions.md.
All limits, windows and intervals are named configuration values. Their defaults are only in section 18.
1. Goals¶
- No event is lost: a client that comes back gets everything it may see, or the clear instruction to reload.
- Nothing is created twice: every write request can be repeated.
- The order in a thread is the same for everyone.
- Permissions apply at every delivery, not only at the request. A revocation takes effect immediately.
- An agent adapter is a normal client with a durable cursor. It misses nothing and starts nothing twice.
2. Events¶
2.1 Two kinds¶
| Kind | Stored | Sequence | Can be caught up | Examples |
|---|---|---|---|---|
| durable | yes, in event_log |
global seq |
yes, until sync.event_retention |
message.created, thread.closed, run.state_changed, access.changed |
| ephemeral | no | none | no | typing.started, run.updated, agent.presence_changed, notification.created, command.autocomplete_requested |
Rule: everything a client must know after a pause is durable. Ephemeral events are pure acceleration; their loss is healed by the next durable state or a fetch. The complete list of durable types with the receive action is in 05-actions.md, section 6.
Terminal (24-terminal.md): states of hosts (host.updated) and of own
terminal sessions (terminal_session.changed, personal) are durable events; the
connection of a host (host.presence_changed) is ephemeral. The terminal stream itself is
not an event: it runs end-to-end encrypted over its own WebSocket connections outside the
gateway and is never in the event log, in snapshots or in catch-up. Reason: the log is for
states, not for byte streams, and the server and backups read everything in it in plain text.
Live statistics (25-visualization.md): the live frame stats.live is
an ephemeral event that only connections with a subscription (stats.subscribe) receive
(section 12, item 4). It is never in the event log; the next frame replaces a
lost one.
2.2 Envelope¶
Every event, durable or ephemeral, has the same envelope, on the gateway, in
REST catch-up and in webhooks (ADR 0052). On the wire it is JSON. There is no JSON null:
a missing field means "no value" (ADR 0053).
| Field | Type | Required | Meaning |
|---|---|---|---|
id |
UUIDv7 | yes | unique event ID, dedupe key |
seq |
int64 | for durable ones | global sequence; missing for ephemeral events |
type |
string | yes | event type, for example message.created |
schema_version |
int | yes | version of the payload of this type; changes only additive |
occurred_at |
RFC 3339 | yes | time of the transaction (display only, never order) |
actor |
{id, kind} |
no | who triggered it; kind is user, agent, app, system; id is missing for system (system action of the server, 05 rule 10); actor is missing for events of jobs without a principal and without a system action (03, event_log.actor_kind) |
scope |
{space_id, channel_id, thread_id} |
yes | place; fields that do not apply are missing |
resource |
{kind, id, version} |
yes | affected object and its version after the change; version is missing where the event does not report a new version of exactly this object (section 9, item 3) |
audience |
"self" |
no | set for personal events |
payload |
object | yes | payload according to the schema <type>.v<schema_version> |
redacted |
bool | no | true: the payload was redacted after a deletion |
truncated |
bool | no | true: the payload was truncated to sync.event_payload_max_bytes; the client reloads via REST |
correlation_id |
UUID | no | trace ID of the triggering request (03, event_log.correlation_id) |
trace_context |
string | no | W3C traceparent of the triggering request; only in deliveries to agent adapters and only if the request was recorded |
client_msg_id |
UUIDv7 | no | echo of the client ID so that the sender can reconcile its outbox |
permission_version |
int64 | for durable ones | permission version at the time of the commit |
invocation_token |
string | no | delivery field: freshly issued invocation token (JWS, I3) for events to agents that carry an invocation (section 15.3) |
redelivered |
bool | no | delivery field: repeated delivery to an agent consumer after agent.ack_timeout |
sync_epoch |
UUIDv7 | no | delivery field: epoch of the timeline (7.5), only in webhook deliveries; a receiver that sees a new epoch resets its seq position (ADR 0059) |
Delivery fields are created per delivery and are never in the event log.
Rules for the payload:
- It contains only fields that a receiver with the receive action of the type may see.
Data that needs more permissions (for example channel prompt, skills, project binding) is never in
events. The event then names only the new version (
channels.settings_version); authorized receivers reload via REST. - It is smaller than
sync.event_payload_max_bytes. For larger objects (a long message) the producer builds a reduced payload with a reference and setstruncated: true(03,event_log.truncated); the appender rejects an oversized payload instead of truncating it. The client reloads via REST. - It contains no secrets and no tokens.
- Message contents are only in events with the resource
message(message.*;reaction.*carry the emoji). All other events name messages only by ID, includingagent.invocationandagent.followup; the receiver reads the content via REST with a check byauthorize. Reason: this way, redaction on deletion (03, section 6.5) reaches all content. - A redacted payload stays valid according to its schema:
message.createdandmessage.updatedcarry the tombstone of the message (content_mdempty, without attachments, mentions, components and reactions, withdeleted_at);reaction.addedandreaction.removedcarryemoji: ""andcount: 0;pin.addedandpin.removedstay unchanged because they carry no content. A new event type with the resourcemessagedefines its redaction with its schema; without it, the deletion fails (test).file.updated(schema version 1) of a deleted file keeps infileonlyid,uploaded_by,purpose,upload_state,scan_status,transcript_statusandcreated_at, withoriginal_name"",size_bytes0,mime_typeapplication/octet-streamandthumbnails[]; every other member is dropped. Another event type with resourcefiledefines its redaction with its schema; without it the deletion fails (03-data-model.md, R-DM-45).
2.3 Frames on the WebSocket¶
Every frame is a JSON text frame of the form {"op": "<name>", "d": {...}}. A frame is
at most sync.ws_max_frame_bytes in size. Events come as {"op": "dispatch", "d":
<envelope>}; the invocation token and redelivered are delivery fields in the envelope.
3. Global, commit-ordered sequence¶
3.1 Why bigserial is not enough¶
A PostgreSQL sequence assigns numbers at the call, not at the commit, and it is not
transactional. Example: transaction A gets seq = 10, transaction B gets seq = 11. B
commits first. A reader sees 11 and sets its cursor to 11. Then A commits with 10.
The reader asks for "everything after 11" and misses 10 forever. In addition, every
rolled-back transaction leaves a gap; a reader cannot tell a gap from a transaction that is
still open.
3.2 Pattern with a counter row¶
The row sequence_counters('event_seq') is locked in the same transaction that writes the
domain change, as the last step before the commit:
BEGIN; -- READ COMMITTED
-- 1. Domain change (example: new message)
UPDATE threads
SET last_message_seq = last_message_seq + 1,
last_activity_at = now(), message_count = message_count + 1,
version = version + 1
WHERE id = $thread_id
RETURNING last_message_seq; -- -> $thread_seq
INSERT INTO messages (id, thread_id, channel_id, thread_seq, author_principal_id,
client_msg_id, client_payload_sha256, origin, kind, content_md, ...)
VALUES ($id, $thread_id, $channel_id, $thread_seq, $author, $client_msg_id, $hash,
'human', 'text', $content, ...);
-- 2. As the last step: lock the counter and get n numbers
UPDATE sequence_counters
SET value = value + $n -- n = number of events of this transaction
WHERE name = 'event_seq'
RETURNING value; -- -> $last; numbers are $last - $n + 1 .. $last
INSERT INTO event_log (seq, id, type, ..., permission_version)
VALUES ($last - $n + 1, ...), ..., ($last, ...);
SELECT pg_notify('event_log', $last::text);
COMMIT;
Why this is correct:
- The row lock from the
UPDATEholds until the commit. A second transaction waits at the same row. - In
READ COMMITTEDthe waiting transaction reads the new counter value again after the commit of the first one. So it gets the next numbers. - PostgreSQL makes a transaction visible before it releases its locks. When B gets the
lock, A is already visible. Therefore: if a reader sees
seq = k, it also sees every smaller number. - If a transaction rolls back, the counter rolls back with it. There are no gaps. A reader may
therefore simply read
WHERE seq > $cursor ORDER BY seq.
Cost: all write transactions with events are serialized at the end. The lock holds
only from the last step to the commit (including the WAL flush). That is why the counter step is
always last. Write transactions with events must not run in REPEATABLE READ or
SERIALIZABLE; there the pattern breaks with serialization errors. The lock order
is in 03-data-model.md, section 6.3. A load test holds the throughput
as a quality gate: at least perf.event_commit_min_per_s in the reference profile
(02-architecture.md, section 17; ADR 0063).
3.3 Other counters¶
The same pattern applies to audit_seq (hash chain in the audit log) and to permission_version
(permission version). On permission changes the permission version is locked first, so that all
permission changes are serialized.
4. Thread sequence¶
Every message has thread_seq, gapless per thread, assigned through the row lock on
threads (pattern above, step 1). It determines the display order. The global seq
determines the delivery order. Both are ordered the same way within a thread, because both come
from the same transaction and the thread lock is taken before the counter lock.
Deleted messages keep their thread_seq as a tombstone. A client recognizes a real
gap in thread_seq as missing data and reloads the range via REST.
5. Idempotent writes¶
5.1 Messages: client_msg_id¶
- The client creates a UUIDv7 as
client_msg_idwhen composing. It stores it with the outbox entry before it sends. - The pair
(author_principal_id, client_msg_id)is unique, permanently (no expiry). - The server computes
client_payload_sha256over the canonical request (thread, content, reply reference, attachments). - Flow in the server: first a
SELECTby the pair. If the message exists, it answers with the original (200, headerIdempotent-Replay: true), without new events. If the hash is different, it answers422 idempotency_key_reuse. If it does not exist, the pattern from section 3.2 follows. If two identical requests arrive at the same time, the second violates the unique index; its transaction rolls back together with the thread number, and the server answers as for a repetition. - The server checks the UUID format (version 7), not the time in it. An outbox may therefore send entries that are days old.
Interactions use the same procedure with client_interaction_id.
5.2 All other writes: Idempotency-Key¶
Every other write REST operation accepts the header Idempotency-Key (UUIDv7).
Clients set it on every operation from the outbox. The server stores the request hash
and the response in idempotency_keys for sync.idempotency_key_ttl. A repetition with the same
hash returns the stored response; a different hash: 422 idempotency_key_reuse; first request
still running: 409 request_in_progress. Operations with a version check (If-Match) are
additionally protected by the version.
6. WebSocket gateway¶
6.1 Endpoint and version¶
One endpoint wss://<host>/api/v1/gateway. The protocol version is in identify and hello.
The supported versions are in one place: info.x-protocol-versions in
asyncapi.yaml; hello and the discovery name the same ones.
Changes within a major version are only additive: new ops, new fields, new
event types. Clients ignore what they do not know. A device holds exactly one connection: a new
identify or resume of the same device closes an older open connection with 4017
(section 6.4). Reason: focus, typing and "person is watching" apply per device, and a faulty
client should not hold any number of connections. With cluster.mode = multi this applies across all
api nodes (device ID through NOTIFY ephemeral, section 12 item 3;
02-architecture.md, section 11.6).
6.2 Ops¶
Client to server:
| Op | Purpose |
|---|---|
identify |
login of a new gateway session |
resume |
resuming a gateway session after a short disconnect |
heartbeat |
sign of life with the last processed seq |
reauth |
new access token during the connection |
focus |
list of the threads that are visible right now (for live box patches and typing), with idle (bool): true means "open but idle"; the connection keeps getting patches but does not count as "person is watching" (13-push-notifications.md, section 4.4). Optional background_thread_ids (addendum of 2026-10-24, run visualization, R-SY-27): threads whose runs the app shows without their messages (control room cards, a run graph opened from a card); they get the run document and run.updated (11.3) but no typing, never count as "Active in the thread" and are not part of the presence focus (13 4.4, R-DM-51); a thread in both lists counts as in thread_ids; both lists together hold at most sync.max_focused_threads distinct threads |
typing.start / typing.stop |
typing indicator; people only (section 12) |
ack |
agents only: acknowledgement up to seq |
run.update |
agents only: patch to an own run (equivalent to REST) |
command.autocomplete_result |
agents and apps only: suggestions for a request (equivalent to REST, section 15.7) |
stats.subscribe / stats.unsubscribe |
subscribe to live statistics (space_id optional, otherwise the instance) or end the subscription; at most one subscription per connection (section 12, item 4) |
Server to client:
| Op | Purpose |
|---|---|
hello |
first message after the upgrade |
ready |
session ready, with a snapshot |
resumed |
resumption confirmed |
dispatch |
an event |
cursor |
advance the cursor without an event (skipped events that are not visible) |
catchup.complete |
catch-up finished, live from now on |
resync_required |
catch-up not possible, reload |
heartbeat_ack |
response to heartbeat |
reauth_required |
the access token expires soon |
error |
error for an op, the connection stays |
Every op is mapped to an action in 05-actions.md. If an op fails on
its rate-limit class, on authorize or on its rules, the server answers with error and
the problem code from 06-api.md (for example rate_limited, forbidden,
validation_failed); the connection stays. The server closes only with the codes from 6.4.
6.3 Lifecycle¶
- Upgrade. The client opens the connection. At the upgrade the server checks nothing except
the origin. The rate-limit class
rl.gateway_connectcounts onidentifyandresumeas soon as the device or the token has been checked (05, section 3); when exhausted: close with4008. hello. The server immediately sends{heartbeat_interval_ms, resume_window_ms, protocol_versions, dpop_nonce, server_time, sync_epoch}(schemaGatewayHello; the values come fromsync.heartbeat_intervalandsync.resume_window).dpop_nonceis fresh and valid forauth.dpop_nonce_ttl(08-identity.md).identify. Withinsync.identify_timeoutthe client sends:- People:
access_token,dpop(DPoP proof according to RFC 9449 withhtm = "GET",htu= HTTPS form of the gateway URL,nonce=dpop_noncefromhello,ath= hash of the token),device_id,protocol_version,cursorandsync_epoch(both missing on the first connection),client(platform and version). - Agents and apps:
api_token, withdpop_jkton the token alsodpop,consumer(name of the consumer, defaultdefault),protocol_version. The server checks the token, signature, expiry,cnf.jktagainst the active DPoP key of the device, deviceactive, session not revoked, principalactive. A device in the statependingis rejected with4016. Then it callsauthorize(principal, gateway.connect, instance). The DPoP proof is inidentify, not in headers of the upgrade. Reason: not every client can set headers on the WebSocket upgrade; the server nonce fromhelloprevents replay.
- People:
ready. The server answers with{gateway_session_id, principal, device_id, permission_version, head_seq, sync_epoch, snapshot}.snapshotcontains: visible spaces and channels with versions, sidebar state, unread and mention counters, active runs (summary), presence of the visible agents, versions of settings and drafts, the feature flags evaluated for the principal. Thread contents are not included; the client loads them via REST. The server registers the connection for delivery before it readshead_seq; every later event is buffered or sent, every earlier one is in the snapshot or in the catch-up. This way nothing is lost betweenreadyand live operation. If the client has sent acursor, catch-up starts after that (section 7).- Heartbeat. The client sends a
heartbeat {last_seq}everyheartbeat_interval_ms. It delays the first heartbeat by a random fraction ofheartbeat_interval_ms, so that after a server restart not everyone sends at the same time. The server answersheartbeat_ack. Ifsync.heartbeat_miss_limitheartbeats in a row are missing, the server closes with4009. If the client does not get aheartbeat_ackwithinheartbeat_interval_ms, it closes itself and reconnects. - Token renewal. Before the access token expires, the server sends
reauth_required(lead timesync.reauth_lead_time). The client gets a new token through refresh and sendsreauth {access_token, dpop}with the current nonce. Without a validreauthby the expiry the server closes with4011. Agents and apps get noreauth_required; their API token has a fixedexpires_atat which the server closes with4011. They can only continue with a new token that the owner issues. resume. After a disconnect withinresume_window_msthe client sends, instead ofidentify, aresume {gateway_session_id, access_token, dpop, cursor, sync_epoch}. The server checks the token and the session as foridentify, restores focus and subscriptions and answersresumed; then the catch-up fromcursorfollows. Ephemeral events of the pause are lost. If the gateway session is unknown or expired, the server closes with4006; the client then doesidentify.- End. The client closes with
1000. The server closes with a code from 6.4.
6.4 Close codes¶
| Code | Name | Client reaction |
|---|---|---|
1000 |
normal | do not reconnect, unless the app is in the foreground |
1001 |
going_away | reconnect with backoff, try resume |
4000 |
unknown_error | reconnect with backoff |
4001 |
unknown_op | report the error, reconnect |
4002 |
decode_error | report the error, reconnect |
4003 |
not_identified | op sent before identify; reconnect |
4004 |
authentication_failed | renew the token; if that fails, log out |
4005 |
already_identified | report the error, reconnect |
4006 |
session_invalid | identify instead of resume |
4007 |
protocol_version_unsupported | show an app update, do not reconnect |
4008 |
rate_limited | only rl.gateway_connect on identify and resume: wait for the waiting time from the close reason |
4009 |
heartbeat_timeout | reconnect with backoff, try resume |
4011 |
token_expired | renew the token, then resume |
4012 |
revoked | session, device or principal revoked: delete local data, log out, do not reconnect |
4013 |
server_restart | reconnect immediately with a random delay, try resume |
4014 |
slow_consumer | outbound queue full or write deadline gateway.write_timeout missed: reconnect, catch-up from the cursor |
4015 |
consumer_replaced | agents only: another process has taken over the same consumer; do not reconnect |
4016 |
device_pending | the device is waiting for approval: do not reconnect until GET /api/v1/devices/{id} reports the state active |
4017 |
device_replaced | a newer connection of the same device is active: do not reconnect on its own, only on return to the foreground or on user action |
4018 |
forbidden | login valid, but gateway.connect denied (for example scope_missing of an app): show the error, do not renew the token, do not reconnect on its own |
The terminal connections (stream of the device, control and data connection of the host service) are
separate WebSocket channels with their own close codes 4100 to 4114
(24-terminal.md, section 13; asyncapi.yaml); the codes of this table
apply there only where the channel names them.
Mapping during connection setup (reason: one code per cause, so that the client reacts correctly):
| Cause | Code |
|---|---|
no identify and no resume within sync.identify_timeout |
4003 |
frame over sync.ws_max_frame_bytes, binary frame, invalid JSON or UTF-8, duplicate key, unknown field |
4002 |
token or DPoP proof invalid, device_id does not match |
4004 |
login valid, but another rejection forbidden (for example scope_missing); a new token does not help |
4018 |
| access token expired | 4011 |
| session, device, token, principal or app installation revoked or not active | 4012 |
| device or provisioned person waiting for approval | 4016 |
database not reachable while building ready |
4000 |
6.5 Reconnecting¶
Before attempt n (from 0) the client waits a time uniformly distributed between 0 and
min(sync.reconnect_backoff_cap, sync.reconnect_backoff_base * 2^n) (full jitter). After a
successful ready or resumed response, n starts at 0 again. A network change and a
return to the foreground trigger an immediate attempt that does not increase the counter.
7. Catch-up¶
7.1 Cursor¶
The cursor of a client is the highest global seq that the server has checked for this
principal, not just the highest delivered one. That is why the server sends cursor {seq} when
it has skipped events that are not visible, at the latest after sync.cursor_advance_every
skipped events and at the end of every page. The client stores the cursor durably after every
processed page.
7.2 Flow¶
- The server reads pages of
sync.catchup_page_size:SELECT ... FROM event_log WHERE seq > $cursor AND seq <= $head AND (channel_id = ANY($visible_channels) OR space_id = ANY($fully_visible_spaces) OR (channel_id IS NULL AND space_id = ANY($visible_spaces)) OR ($instance_wide AND space_id IS NOT NULL) OR audience_principal_id = $principal OR (space_id IS NULL AND channel_id IS NULL AND audience_principal_id IS NULL)) ORDER BY seq LIMIT $page.$visible_channels,$visible_spaces(spaces with at least one visible channel),$fully_visible_spaces(spaces with a grant on the space) and$instance_wide(owner or a grant on the instance) are provided byauthzfrom the permission cache. Reason for the spaces: a channel that is created in a visible space after the cache was built is not yet in$visible_channels; its events must still not be missing. - In addition, every event passes through
authorize(principal, <receive_action>, resource). Only then is it sent asdispatch. The SQL filter is the performance,authorizeis the truth. - New live events during the catch-up are buffered per connection and sent only after the
catch-up, in
seqorder. There is no mixing. - At the end the server sends
catchup.complete {head_seq}. After that the connection is live. - If the buffer exceeds
sync.outbound_queue_max, the server closes with4014. The client restarts from its cursor. - The server does not deliver ephemeral events before
catchup.complete, and it discards them when the outbound queue is full instead of closing the connection (never4014); forrun.updatedthe state according to 11.3 item 3 makes up for it. Reason: they are pure acceleration (2.1) and must not cost a connection.
7.3 resync_required¶
The server sends resync_required {reason, head_seq, sync_epoch} instead of events when:
- the sent
sync_epochdoes not match the current one (restore from a backup, section 7.5), reasonepoch_changed; cursor < min_retained_seq - 1(events were removed aftersync.event_retention), reasonretention;head_seq - cursor > sync.catchup_max_events, reasontoo_far_behind;cursor > head_seq, reasoncursor_ahead(second safety net, in case a client asks without an epoch).
The client then discards its server state in the cache (the outbox stays), loads
GET /api/v1/sync/snapshot, sets the cursor and the epoch to the returned values and loads threads
via REST when needed. On the gateway the connection stays open and is live from head_seq;
the client discards events up to the head_seq of the new snapshot according to section 9,
item 2. It does not have to reconnect. Reason: every close code would trigger a wrong reaction,
and the live delivery from head_seq loses nothing.
7.4 REST variant¶
GET /api/v1/events?after=<seq>&epoch=<sync_epoch>&limit=<n> returns the same events with
the same rules: {events, next_cursor, head_seq, sync_epoch} or 410 resync_required with
a reason. It serves SDKs, tests and adapters without WebSocket. limit is at most
sync.catchup_page_size.
7.5 Sync epoch¶
The global sequence applies within one timeline, named by instance.sync_epoch
(03-data-model.md, section 6.7). After a restore
bothy restore finalize sets a new epoch.
hello,ready,resumed, the snapshot andGET /api/v1/eventsname the epoch.- The client stores it together with the cursor and sends both on
identify,resumeand in the REST catch-up. - If the sent epoch does not match, the server answers
resync_requiredwith the reasonepoch_changed, even ifcursor <= head_seq. Reason: after the first writes of the new timeline it overtakes old cursors; without an epoch a client would silently skip foreign events with the same number. - The same command sets agent consumers, webhook subscriptions and worker consumers to the new
head_seq; they catch up nothing from the old timeline.
7.6 Consumers in the worker (push, webhooks)¶
Push and outgoing webhooks are created in the worker from the log, not in the write path
(02-architecture.md, section 11.5, ADR 0051). Per consumer there is one
row in event_consumer_cursors. A round reads from last_seq in seq order, creates one job per
event and receiver and advances the cursor in the same transaction. Before
sending, every job checks push.deliver or webhook_out.deliver again with the current
permissions.
8. Visibility on permission changes¶
8.1 Basic rule¶
The permissions at the time of the delivery apply, not those at the time of the event.
A gateway process updates the permission cache of a principal when it reaches that principal's
access.changed event in the log. It checks all events after that with the new state.
It also applies structure changes (channel.created, channel.updated with read_only,
channel.archived, space.archived, thread.moved) to the cache in log order, because they
change the path or the read-only state of a resource. channel.archived and space.archived are
in addition permission changes with their own access.changed (05, column "Permissions";
07-permissions.md, section 4.5); the others come without a new permission version.
8.2 What happens on a change¶
Every transaction that changes permissions (grant, group, owner, operator, backend policy,
tool class, status of a principal, device or session) increases the permission version and
writes a personal access.changed event for every affected principal. This also applies
to the expiry of grants and memberships: it takes effect immediately on every authorize,
and the job authz.expiry_sweep writes, every authz.expiry_sweep_interval, the permission version,
access.changed and run.cancel_requested for the expired ones (system actions
grant.expire, group.member.expire). Before an expiry the subject gets
access.expiring once (07-permissions.md, section 9.4).
{
"permission_version": 418,
"added": [{"kind": "channel", "id": "...", "level": "write"}],
"removed": [{"kind": "space", "id": "..."}],
"changed": [{"kind": "channel", "id": "...", "level": "read", "previous_level": "trigger"}]
}
An entry can have kind: "host": it reports the capability terminal on a host,
without level, with capabilities: ["terminal"] (24-terminal.md, section 8). Whoever
loses it, their terminal sessions on the host end in the same transaction
(access_revoked).
Affected are all principals whose visible targets or levels can change: for grants
the subject (for groups all current members), for an owner change and for disabling the person, for an
operator change the old and the new operator agent, for archiving a space or channel
everyone with a level above read on the target (grants on the target and its ancestors, group members,
owners, operators). If an action changes a target instead of a subject, authz determines the affected
principals itself from the target (02-architecture.md, section 8.1). authz computes
the content as the difference of the visible targets before and after the change; whoever has no difference
gets no access.changed.
8.3 What the client receives and does on access removal¶
- The server sends
access.changedimmediately, without waiting for other events. - After that it delivers no more events from the removed targets, ends their focus and discards buffered events of these targets.
- For removed targets the client deletes all content from the local cache: threads, messages, files, thumbnails, search index, drafts. It closes open views with the note "No access anymore".
- Outbox entries for removed targets fail on sending with
404. The client marks them as failed and keeps the text locally for copying. - If the principal has lost the level
triggeron a thread in which a run it triggered is active, the server writesrun.cancel_requestedwith the reasonaccess_revokedto the agent. The agent aborts and reportscancelled(fail closed, ruling L23). The same applies with the reasonchannel_read_onlywhen a channel becomes read-only. - Push: every push passes through
authorizebefore sending. There is nothing to delete.
8.4 On access extension¶
The client gets access.changed with added. It does not catch up older events of the new targets
before its cursor; it loads the structure of the new targets via REST and contents when needed.
8.5 Revocation of a session, device or principal¶
The server closes all affected connections immediately with 4012. The client deletes all
local data of this account. Access tokens carry the session ID sid; authorize checks
the revocation state from a cache that is tied to the permission version. A revoked token is
thus without effect before its expiry.
9. Delivery guarantee¶
- Durable events are delivered at least once (at-least-once). Duplicates arise on reconnecting and on retries.
- The client discards every event with
seq <=its stored cursor. Because a connection delivers strictly inseqorder, this comparison is enough. - In addition, the client applies a change to an object only if
resource.versionis greater than the local version. This way data from REST and from events can be mixed without conflict.resource.versionis present exactly when the table of the object has a columnversion(03) and the event reports its new value. It is therefore also missing for these types, whose object has a version that the event does not change:reaction.*andpin.*: they do not increasemessages.version(otherwise the author would get412when editing after every reaction); their payload carries absolute values (count per emoji, pinned yes or no).read_state.updated(resourcethread) andsidebar.updated(resourcespace): their payload carries the version of the personal sub-object; the client compares that one.channel.settings_changedandagent.config_changed(resourcechannel, payload withsettings_version),thread.moved_out(the source channel does not change),agent_session.changed(resourcethread),agent.invocation,agent.followupandrun.cancel_requested(resourcerun),command.invoked(resourcecommand). Withoutresource.versionthe client applies the event inseqorder. Reason: a foreign version in the field of the object would make later real changes get discarded. The catalogue (ADR 0049) records per event type whether it hasresource.version; the frame conformance (G9, 06-api.md section 15) checks every delivery against it.
- Ephemeral events are delivered at most once (at-most-once).
10. Ordering¶
| Scope | Guarantee |
|---|---|
| global, durable events | delivery in seq order = commit order |
| per thread | strictly by thread_seq; the same for all receivers |
| per object | strictly by resource.version |
| ephemeral | no guarantee among each other; run.updated is ordered through version |
| between processes | every gateway process reads the same log in the same order |
Timestamps never order anything.
11. Live box streaming¶
11.1 Run document¶
A run has a document (schema run_document.v1): state, version, status_line,
last_text_excerpt, model, metrics (including last_api_retry), cost,
billing_mode_used, plan (items with id, text, status), steps (object, the key is
the step ID, every step with ordinal, parent_step_id, kind, name, title,
status, times), followups (object, the key is the message ID), stall_warned_at (addendum of
2026-10-24), times. Which paths
the adapter may change is defined in 09-agents.md, section 5.3. Steps are
stored as an object, not as a list, so that patches use stable paths; the order
comes from ordinal.
11.2 From the adapter to the server¶
The adapter sends run.update {run_id, base_version, ops} (WebSocket) or PATCH
/api/v1/runs/{id} (REST). ops is a list of operations according to RFC 6902, limited to
add, replace and remove, with JSON Pointer paths into the run document. The server checks:
the principal is the agent of the run, the run is active, base_version matches, the paths are allowed
(for example not state; states change only through their own endpoints). It stores the change,
increases runs.version and answers with the new version. On a wrong base_version
it answers 409 base_version_conflict with the current document. The adapter throttles itself to at most one
change per agent.run_update_min_interval.
11.3 From the server to the clients¶
- State changes (
queued,running,waiting_input, end) are durable eventsrun.state_changedwith a summary (state, duration, cost, number of steps). The sidebar and "Running now" are fed from them. - Content changes are the ephemeral event
run.updated {run_id, base_version, version, ops}. It goes only to connections that have the thread infocusand may performrun.read; a thread inbackground_thread_ids(6.2) counts as in focus for this event and for item 5. A client focuses at mostsync.max_focused_threadsthreads (both lists together); afocuswith more IDs results inerrorwithvalidation_failed, and the previous focus stays. - Coalescing: the server collects changes per run and connection for
sync.run_patch_coalesce_interval. Then it sends one frame from the oldestbase_versionto the newestversion. Laterreplaceoperations on the same path replace earlier ones. If the outbound queue of the connection is full, the server discards the collected patches and instead sends the whole document asrun.updatedwithsnapshot. - Client rule: if
base_versionequals the local version, it applies the ops. Ifversionis less than or equal to the local one, it ignores the frame. Otherwise (a gap) it fetchesGET /api/v1/runs/{id}and replaces its document. It always draws only the newest state. - When a thread with an active run is focused, the server sends the whole document once.
- After the end the live box collapses to one line. The summary comes from
run.state_changed; the client loads details via REST when it is expanded. - Run graph and control room (addendum of 2026-10-24, run visualization,
09-agents.md section 5.6): both use only this path. The control room puts
the threads of its visible cards into
background_thread_idsoffocus(6.2), withinsync.max_focused_threadstogether with the threads the app already shows;thread_idsandidlekeep their meaning (the threads the person reads, the window state of 15 section 10), so the cards change no notification rule of 13 4.4. There is no further op and no further event.stall_warned_atis part of the document: the server sendsadd /stall_warned_atwhenruns.stall_checksets it (with a version step of its own) andremove /stall_warned_atin the normalized ops of the patch that clears it.
12. Typing and agent presence¶
- Typing. The client sends
typing.start {thread_id}at most once persync.typing_min_intervalwhile the person is typing, andtyping.stopwhen the input is cleared or sent. The server checkstyping.start(action in 05-actions.md; people only) and sendstyping.started {thread_id, principal_id, expires_at}to connections with focus on the thread and read permission, not to the connections of the typing person themselves. Without renewal, the indicator ends aftersync.typing_ttl; clients also end it at the next message of this principal in the thread.typing.stopcreates no event; it only resets the throttle of the connection, so that a newtyping.startapplies at once. An active run shows the typing of the agent; agents send no typing of their own (Ruling L9). - Presence of agents. Adapters send
POST /api/v1/agents/me/heartbeat(or implicitly through the gatewayheartbeat) with state, runtime version,reported_billing_mode,billing_modes_supported,billing_evidence, tool class and active runs (schemaagent_heartbeat.v1, 09-agents.md, section 4.8). Adapters that follow the configured mode per run omitreported_billing_modeand name both modes inbilling_modes_supported; the check per run is then binding (Ruling L8). A change of the reported mode or of the tool class is stored and audited at once. The server derivesonline,degradedoroffline;offlineapplies afteragent.presence_timeoutwithout a heartbeat. On a change it sends the ephemeralagent.presence_changed {agent_id, state, reported_billing_mode}to everyone withagent.read.readycontains the current state. People have no presence indicator. - Multiple nodes. With
cluster.mode = multi, a node also distributes typing, presence, notifications and live box changes throughpg_notify('ephemeral', ...), only with IDs and versions; the receiving node reads the content from the database (03-data-model.md, section 6.6). - Live statistics. A person with
stats.readsendsstats.subscribe {space_id?}. The server checksstats.readon the instance or the space and then sends, everystats.live_interval, the ephemeral eventstats.livewith a live frame (schemaStatsLiveFrame, 25-visualization.md, section 5) to this connection. A newstats.subscribereplaces the subscription,stats.unsubscribeends it,resumerestores it. Every frame passes through the delivery mode with the receive actionstats.readand is filtered to the visible channels of the receiver. If the receiver loses the permission, the subscription ends with anerror(forbidden). Agents and apps cannot subscribe (stats.readrequires owner ormanage). Withcluster.mode = multi, every node builds its frames itself; rates come fromstats_minutesand are therefore the same on all nodes.
13. Offline behavior of the client¶
13.1 Local cache¶
The client keeps the structure, recently opened threads, messages, thumbnails, its own
settings and drafts in a local, encrypted database (protection class per platform: see the
security chapter). The cache is only a copy; the truth lies on the server. Entries older than
client.cache_max_age without access are removed.
13.2 Outbox¶
- Every writing operation first lands in the outbox, with its own ID
(
client_msg_idfor messages,client_interaction_idfor interactions,Idempotency-Keyfor everything else), a timestamp and the expected version where needed. - The outbox is strictly FIFO per thread. The client sends the next entry of a thread only after the response to the previous one. Different threads run in parallel.
- The interface shows waiting messages at once with the state "sending" at the position
after the last confirmed
thread_seq. - Retry on a network error, a timeout,
5xxand429: backoff with full jitter betweenclient.outbox_retry_baseandclient.outbox_retry_cap. The same ID on every retry. - Permanent errors (
400,404,410,413,422) end the retries. The entry stays visible as "failed" with a reason; the person can edit it, send it again (new ID) or discard it. Nothing is discarded silently. - When the echo event
message.createdwith a matchingclient_msg_idarrives, the client replaces the waiting entry with the server message.
13.3 Editing and deleting¶
| Case | Rule |
|---|---|
| Editing a message that has not been sent yet | changes the entry in the outbox; no server request |
| Deleting a message that has not been sent yet | removes the entry; if it was already sent but not confirmed, a delete entry follows |
| Editing a sent message | PATCH with If-Match (version). If the version does not match: 412 version_conflict with the current revision (RFC 9110). The client shows both revisions; the person decides. No automatic merge. |
| Editing a message that has been deleted in the meantime | 410 message_deleted; the entry is discarded, the text stays available for copying |
| Deleting a sent message | DELETE is idempotent; an already deleted message yields 204 |
| Thread marked done in the meantime | sending reopens it (baseline); no conflict |
| Thread moved | the ID stays the same; sending works without a change |
13.4 Reconciliation after reconnecting¶
- Connect with
resumeoridentifyand the stored cursor. - Catch up until
catchup.complete. Echoes of the person's own messages confirm entries in the outbox. - After that, the client sends the outbox in order.
- On
resync_required, the outbox stays untouched and is sent after the reload.
14. Cross-device synchronization¶
Personal data changes through REST. Every change writes a personal durable event
(audience = self); all devices of the person receive it through the log. The winner rule is
the commit order in the server, expressed by the version.
| Data | Event | Rule |
|---|---|---|
| Read state | read_state.updated |
monotonic: last_read_seq = GREATEST(old, new). Resetting ("mark as unread") is a separate operation with If-Match. Imported content never counts as unread, through threads.unread_floor_seq (section 15.8). The server does not store unread and mention counters (03 read_states): the snapshot, listMyReadStates and read_state.updated give the computed value; in between, the client counts by itself from message.created, message.updated and message.deleted (mention_ids). A new mention writes no personal event. The server does not advance the read state of the author on sending; the author's client does that through REST like any other change of personal data. |
| Drafts | draft.updated, draft.deleted |
the last write wins in server order; every write increments version. The client sends base_version; if it is older than the current one, the server answers with overwrote_newer: true, and the client shows a notice. A device takes over incoming changes from another device only when the person is not currently typing in this draft on it; otherwise the next local write wins. Successful sending with clear_draft: true deletes the draft in the same transaction. An empty draft (no text except whitespace, no files) is deleted; a PUT with an empty draft answers 200 with the empty final state. On saving, the server checks file_ids only in form and number (message.max_attachments); ownership and state are checked on sending (12-files.md, section 10). |
| Settings | settings.updated |
per key, the last write wins; changes to different keys do not interfere with each other. The server does not store default values, because some depend on the device or the instance; the client applies the defaults from 01 and 13. Deleting a key writes settings.updated with deleted: true. |
| Sidebar | sidebar.updated |
per space (user_space_states), the last write wins |
| Notification, quiet hours | notification_pref.updated, quiet_hours.updated |
per row, the last write wins |
A device recognizes its own echo by updated_by_device_id and ignores it.
Lists of personal data (listMyReadStates, listMyDrafts, snapshot) name only threads
and channels that the person may still read (07-permissions.md, section 7.4, rule 1).
The rows stay stored until their retention ends; if the person gets the read permission
back, they appear again.
15. Agent adapters¶
15.1 Connection and cursor¶
- An adapter connects like a client, with
api_tokenandconsumer. - For each
(agent, consumer)there is a durable cursor inagent_consumer_cursors. On the first connection, the server sets it to the currenthead_seq. An adapter therefore never catches up on events from the time before its activation. - There is exactly one connection per consumer. A new
identifywith the same consumer writes its gateway session intoagent_consumer_cursors.holder_session_idand closes the older connection with4015. Reason: two processes on the same stream would do the work twice. Across several api nodes, fencing holds this: a node storesacked_seqonly withWHERE holder_session_id = <own session>; if it gets no row back, it closes its connection with4015. Until then, the event ID and the run start by compare-and-swap prevent duplicate work. - The server delivers from
acked_seq + 1inseqorder, filtered as for people (the agent sees what its grants and the operator rule allow), plus personal events to it (agent.invocation,agent.followup,run.cancel_requested,interaction.created,command.invoked,agent.config_changed,access.changed) andagent.updatedabout itself (a change of the configured backend policy or of the configured tool class). - Ephemeral, only to one active connection of the agent:
command.autocomplete_requested(section 15.7).
15.2 Ack and redelivery¶
- The adapter sends
ack {seq}after durable acceptance. An event with an invocation token is accepted when the token passes the check in the adapter (expiry excepted) and the event together with the token is stored in the inbox of the adapter; the run start follows after that (09-agents.md, section 4.3). The ack applies to all numbers up toseq. - At most
agent.max_inflight_eventsunacknowledged events are in flight. After that, the server waits. - Without an ack within
agent.ack_timeout, the server delivers again fromacked_seq + 1, with the delivery fieldredelivered: truein the envelope. - The server stores
acked_seqat the latest afteragent.ack_persist_interval. After a server restart there can therefore be duplicates; the adapter deduplicates by the event ID and by run IDs. If the duplicate carries a different invocation token and the run has not started yet at the adapter, the new token replaces the stored one, because the delivery replacedruns.invocation_jti.
15.3 Invocation and run start¶
- Acceptance. A triggering message (decision in 09-agents.md,
section 3.4) writes in its transaction: the message with
trigger_state = received, a runqueuedwithtrigger_kind, the live box anchor and the personal eventagent.invocationto the agent (without a token). Exception: if the message has attachments that are not final yet (scan running), or if it is a voice message whose transcript the server creates, the event is written only by the job that makes the last attachment final (ReleaseInvocationin the moduleruns, flagruns.invocation_held_until, 03-data-model.md, R-DM-44), at the latest afterfile.invocation_hold_maxorvoice.transcript_hold_maxrespectively (12-files.md, section 10). Reason: an agent never receives unchecked bytes, and a waiting invocation does not block the other events of the consumer. - Delivery. On every delivery of
agent.invocation, the server checks immediately before itauthorize(trigger_principal, agent.invoke, thread)with the current permissions, the backend policy (I2) and the guard rules SR2 and SR3;trigger_principalis the triggering person without session, device and token (05-actions.md, section 5, "Check without a request", R-AZ-29). If the check fails, it does not deliver, sets the run tofailedwith a reason, setstrigger_state = failedwithtrigger_state_reasonand writes an audit entry (fail closed). If it passes, the server creates a fresh invocation token (JWS, EdDSA, claims in 05-actions.md, section 5) with the validityagent.invocation_token_ttl, stores itsjtiinruns.invocation_jti, setsruns.billing_mode_expectedto the current configured mode and puts the token as the delivery fieldinvocation_tokeninto the envelope. With the first successful delivery,trigger_statechanges fromreceivedtoqueued. - Check in the adapter. The adapter checks the header (
typ = agent-invocation+jwt,alg = EdDSA), the signature against the JWKS of the server, the expiry, the audience (itself), that its reported mode for exactly this run equalsbilling_modeof the token and that its tool class is not higher thantool_class. If something does not match, it reportsPOST /api/v1/runs/{id}/finishwithstate = failedand a reason, for examplebilling_mode_mismatchortool_class_mismatch. If only the expiry is exceeded, it fetches a fresh token withPOST /api/v1/runs/{id}/invocation-token(actionrun.token.renew): the server checks the old token without the expiry, requires the run inqueuedorwaiting_input(otherwise409 already_started),jtiequal toruns.invocation_jtior equal toruns.renewed_from_jti(retry after a lost response) andruns.started_jtiNULL or not equal tojti(not consumed, in SQLstarted_jti IS DISTINCT FROM jti; otherwise403 invocation_token_invalidwithreason: superseded), repeats the checks from step 2 with the current permissions and issues a new token that replacesruns.invocation_jti(09 section 4.5.1). - Start. Otherwise it fetches the context (
run.context.read) and starts withPOST /api/v1/runs/{id}/start {invocation_token, billing_mode_used, tool_class_used, external_session_ref, model, billing_evidence};external_session_refis missing when the runtime creates the reference only during the run, and then the firstagent_session.reportcreates the session row. The server checks the signature, the expiry,jtiequal toruns.invocation_jti, the reported mode equal to the configured mode andtool_class_usednot higher thantool_classof the token, and performsqueued -> runningas a compare-and-swap; it storesruns.started_jtiand setsruns.renewed_from_jtitoNULL. A retry with the same token (response lost) returns200with the same result as long as the run isrunning; inwaiting_inputthe token is consumed (403 invocation_token_invalidwithreason: superseded, 09 section 4.5). Another, older delivery of the same invocation gets409 already_started; the adapter then starts nothing. Ifbilling_mode_useddeviates from the configured mode, the server rejects the start and sets the run tofailedwithbilling_mode_mismatch; iftool_class_usedis higher, likewise withtool_class_mismatch(409). An error of the token yields403 invocation_token_invalidwithreason:expired(the adapter renews and starts again),superseded(a newer delivery applies; the adapter starts with its token) orinvalid(runfailed). - Resumption. While a run waits (
waiting_input), deliveries ofagent.followupandinteraction.createdalso replace the token inruns.invocation_jti. The adapter sets the run back torunningwith this token (POST /api/v1/runs/{id}/resume, actionrun.start, the same checks). Deliveries duringrunningdo not changeruns.invocation_jti; afterrunning -> waiting_input, it therefore still holds the consumed start token, which neitherrun.startnorrun.token.renewaccepts. Every delivery that replacesruns.invocation_jtisetsruns.renewed_from_jtitoNULL. - With the start,
trigger_statechanges toworking, at the end todone,failedorcancelled.
15.4 Follow-ups¶
- Only a message that would trigger the agent of the active run according to
09-agents.md, section 3.4 becomes a follow-up: a row in
run_followups(queued) and, if the adapter can feed follow-ups into the running turn (followups = steer), the personal eventagent.followupto the agent. Other messages stay normal messages and reach the agent only as context of the next run. If the message would trigger another agent, it is rejected (run_active_other_agent). - The delivery checks as in 15.3 step 2 (a follow-up is a trigger) and attaches a
token (
trigger_kind = followup). - The adapter reports
delivered(handed to the runtime) andabsorbed(taken in by the runtime) as a patch onfollowups/<message_id>. Ondelivered, the server setstrigger_state = followup_delivered. - If the run ends with follow-ups in the state
queuedordeliveredwithoutabsorbed, the server sets them todeferredand creates a new run: the first of these messages is the triggering one, the others are its follow-ups. If it ends withcancelledand the reasonuser_cancelled, they becomediscarded(09-agents.md, section 7). - A follow-up to a run in
waiting_inputis the answer to the open question; the adapter resumes the run with the token of the follow-up (15.3 step 5). - Control commands (
commands.control = true, for example stop) are never follow-ups; they go to the agent at once ascommand.invoked.
15.5 Cancellation¶
run.cancel (person), access removal (8.3), disabling the agent or the triggering person, and
the channel becoming read-only write run.cancel_requested with a reason to the agent. The agent
ends the runtime and reports cancelled. If it reports nothing within agent.cancel_grace,
the server sets the run to cancelled itself with the reason cancel_timeout; later patches of
the adapter for this run are rejected.
15.6 Webhook as transport¶
An adapter without a running process (for example a function) can receive events by webhook
instead of through the gateway (09-agents.md, section 4.2). The subscription is
bound to the agent (webhook_subscriptions.agent_id). The same filters and checks apply as
in 15.1 to 15.3. Per subscription, the next event goes out only after a 2xx for the previous
one; 2xx counts as an ack up to this seq. Every delivery, including a retry, carries a
freshly issued token. Ephemeral events do not exist on this path.
15.7 Autocomplete¶
- A client requests suggestions for an option with
autocomplete = true(command.autocomplete). The server checks as foragent.invokewithout a run (including I2) or, for app commands,required_leveland scope. - It sends the ephemeral event
command.autocomplete_requested {request_id, command_id, channel_id, thread_id, user_id, option, value, options, expires_at}to an active gateway connection of the command's owner, for agents with a token (trigger_kind = autocomplete, withoutrun_id). For agents, token and payload name asthread_idthe thread of the request, without one, in a text channel, the main timeline (03-data-model.md, R-DM-11). A forum channel has no main timeline: if a request for an agent command there names no thread (new post), the check from item 1 applies as for a new thread of the channel (05-actions.md, section 5). If it passes, the server does not ask the agent, issues no token and returns an empty list at once (R-AZ-26). For apps without a gateway, the server asks through their interaction address (11-extensions.md, section 9). - The command's owner answers with the op
command.autocomplete_result {request_id, choices}or through REST (command.autocomplete.respond), with at mostcommand.autocomplete_max_choicesentries. - Without an answer within
command.autocomplete_timeout, the server returns an empty list. A handler never calls a model.
15.8 Writes of an import¶
An import batch (import.apply, 22-import-migration.md) affects
exactly one thread and is one transaction. Instead of message.created per message, it writes
one thread.updated for the thread and one import.progress; clients load the thread through
REST when they display it. At the end of the batch, the server sets threads.unread_floor_seq to
the end of the batch; imported content therefore counts as unread for nobody (03, read_states). Imported messages have origin = import, never trigger agents and create
no notifications.
16. Sequence diagrams¶
16.1 Send a message, online¶
sequenceDiagram
autonumber
participant C as Client (Alex)
participant API as API process
participant DB as PostgreSQL
participant GW as Gateway processes
participant O as Other clients (Sam)
C->>C: create client_msg_id, put into the outbox
C->>API: POST /api/v1/threads/{id}/messages (client_msg_id, DPoP)
API->>API: authorize(Alex, message.create, thread)
API->>DB: SELECT by (author, client_msg_id)
DB-->>API: no row
API->>DB: BEGIN, thread counter, INSERT message
API->>DB: UPDATE sequence_counters event_seq, INSERT event_log, pg_notify
API->>DB: COMMIT
API-->>C: 201 message (thread_seq, version)
DB-->>GW: NOTIFY event_log (seq)
GW->>DB: SELECT event_log WHERE seq > last position
GW->>GW: authorize per receiver (message.read)
GW-->>C: dispatch message.created (echo, client_msg_id)
GW-->>O: dispatch message.created
C->>C: entry in the outbox confirmed
16.2 Send offline, then reconnect¶
sequenceDiagram
autonumber
participant C as Client (Alex)
participant API as API process
participant GW as Gateway
C->>C: no network: message with client_msg_id into the outbox
C->>C: display "sending"
Note over C: network back
C->>GW: connect, hello
GW-->>C: hello (dpop_nonce)
C->>GW: resume (cursor = 900)
GW-->>C: resumed, dispatch 901..950, catchup.complete
C->>API: POST message (client_msg_id)
API--xC: timeout (response lost, but message stored)
C->>API: POST message (same client_msg_id), backoff
API->>API: pair exists, hash equal
API-->>C: 200 original, Idempotent-Replay: true
GW-->>C: dispatch message.created (seq 951, client_msg_id)
C->>C: entry confirmed, exactly one message
16.3 Reconnect with catch-up¶
sequenceDiagram
autonumber
participant C as Client
participant GW as Gateway
participant AZ as authz
participant DB as PostgreSQL
C->>GW: Upgrade
GW-->>C: hello (heartbeat_interval_ms, resume_window_ms, dpop_nonce)
C->>GW: identify (access_token, dpop, device_id, cursor = 1200)
GW->>AZ: authorize(gateway.connect), visible channels
AZ-->>GW: permission snapshot (permission_version)
GW-->>C: ready (head_seq = 5300, snapshot)
alt cursor older than retention or too far back
GW-->>C: resync_required (reason), connection stays live from head_seq
C->>C: discard cache, keep outbox
C->>GW: GET /api/v1/sync/snapshot, cursor to its head_seq
else catch-up possible
loop pages of sync.catchup_page_size
GW->>DB: SELECT event_log WHERE seq > cursor AND visible
GW->>AZ: authorize per event (receive_action)
GW-->>C: dispatch ... (visible ones only)
GW-->>C: cursor (last checked seq)
end
Note over GW: live events buffered during this time
GW-->>C: catchup.complete (head_seq)
GW-->>C: buffered live events in seq order
end
C->>GW: heartbeat (last_seq), send outbox
16.4 Access removal during an open connection¶
sequenceDiagram
autonumber
participant O as Owner
participant API as API process
participant DB as PostgreSQL
participant GW as Gateway
participant S as Client (Sam)
participant A as Agent adapter
O->>API: POST /api/v1/grants/{id}/revoke (step-up)
API->>DB: BEGIN, lock and increment permission_version
API->>DB: set grants.revoked_at, check I1
API->>DB: event_log access.changed (audience Sam), run.cancel_requested (audience agent)
API->>DB: audit_events, COMMIT
API-->>O: 200
DB-->>GW: NOTIFY
GW->>GW: on access.changed: rebuild permission cache of Sam
GW-->>S: dispatch access.changed (removed: channel Support)
GW->>GW: end focus, discard buffer for channel Support
S->>S: delete cache for channel Support, close view
GW-->>A: dispatch run.cancel_requested (access_revoked)
A->>API: POST /api/v1/runs/{id}/finish (cancelled)
Note over GW,S: later events from channel Support no longer reach Sam
16.5 Agent run with follow-up¶
sequenceDiagram
autonumber
participant H as Client (Alex)
participant API as API process
participant GW as Gateway
participant A as Agent adapter
participant R as Agent runtime
H->>API: POST message (mention or free-response channel)
API->>API: authorize(Alex, message.create) and trigger check
API-->>H: 201 (trigger_state received)
Note over API: same transaction: run queued, live box anchor, agent.invocation
GW->>GW: before delivery: authorize(Alex, agent.invoke) and I2
GW-->>A: dispatch agent.invocation + invocation token (JWS)
GW-->>H: message.updated trigger_state queued
A->>A: signature, reported mode equals configured mode (I3), tool class, into the inbox
A->>GW: ack (seq)
A->>API: POST /api/v1/runs/{id}/start (token, billing_mode_used, tool_class_used)
API-->>A: 200 running
GW-->>H: dispatch run.state_changed running, trigger_state working
A->>R: start turn
loop throttled
R-->>A: tool, status, text
A->>API: run.update (base_version, ops)
API-->>GW: ephemeral run.updated
GW-->>H: run.updated (coalesced)
end
H->>API: POST message (follow-up)
API-->>H: 201
GW->>GW: authorize(Alex, agent.invoke) and I2
GW-->>A: dispatch agent.followup + token
A->>R: feed in follow-up
A->>API: Patch followups/ID delivered
GW-->>H: message.updated trigger_state followup_delivered
A->>API: Patch followups/ID absorbed
R-->>A: answer
A->>API: POST message (origin agent), POST /api/v1/runs/{id}/finish (completed)
GW-->>H: message.created, run.state_changed completed, trigger_state done
A->>GW: ack (seq)
17. Decisions of this chapter¶
| No. | Decision | Reason |
|---|---|---|
| R-SY-01 | DPoP proof in the identify frame with a server nonce from hello, not in upgrade headers. |
The same path for all clients; the nonce prevents replay. |
| R-SY-02 | Permissions apply at delivery time; the permission cache follows access.changed in log order. |
Simple, effective at once, the same for all gateway processes. |
| R-SY-03 | The cursor is "last checked", not "last delivered"; the server sends cursor frames. |
Someone who sees little does not rescan the same foreign traffic on every connect. |
| R-SY-04 | resync_required also on cursor > head_seq. |
A second safety net next to the sync epoch (R-SY-18) for clients without a stored epoch. |
| R-SY-05 | Live box patches are ephemeral and go only to focused threads; states are durable. | A high patch rate burdens neither the log nor uninvolved devices; after pauses, the state is enough. |
| R-SY-06 | The patch format is RFC 6902 with add, replace, remove; steps as an object by ID. |
Standard, stable paths, simple coalescing. |
| R-SY-07 | Invocation tokens are created anew on every delivery, not on writing. | I2 checks directly before the invocation; a redelivery after expiry stays possible. |
| R-SY-08 | Invocation tokens are signed asymmetrically (EdDSA), keys through JWKS. | The adapter can verify a token but cannot forge one (study A proposed HMAC). |
| R-SY-09 | One consumer per (agent, consumer), the newest wins. |
One stream, no duplicate work. |
| R-SY-10 | The run start is a compare-and-swap with a jti check. |
A redelivery never starts a second run. |
| R-SY-11 | Access removal cancels the runs of the affected triggering person, for all agents. | Fail closed; study C allowed the run to finish writing, which is harder to check. |
| R-SY-12 | Drafts: the last write wins, with the notice overwrote_newer; no merge. |
Drafts are personal and mostly active on one device; merging would be expensive and error-prone. |
| R-SY-13 | The read state is monotonic; resetting only with a version check. | Two devices cannot throw each other back to "unread". |
| R-SY-14 | On a device or session revocation, the client deletes all local data. | A lost device keeps no content. |
| R-SY-15 | Distribution to gateway processes through LISTEN/NOTIFY and reading the log, no broker. |
One dependency path fewer; the log is the source anyway. |
| R-SY-16 | No JSON null in the envelope; missing fields mean "no value" (ADR 0053). |
Generated clients (dart-dio) know no null type; Go omitempty fits. |
| R-SY-17 | The invocation token and redelivered are delivery fields in the same envelope, never in the log (ADR 0052). |
One envelope for gateway, REST and webhooks; tokens are created per delivery (R-SY-07). |
| R-SY-18 | A sync epoch after every restore; a mismatch yields resync_required with epoch_changed (ADR 0059). |
Detects a new timeline even when its numbers have already overtaken old cursors; R-SY-04 stays as a second safety net. |
| R-SY-19 | A repeated run.start with the same token returns the result again (runs.started_jti) as long as the run is running; a repeated run.token.renew with the token from before the last renewal issues a token again (runs.renewed_from_jti). |
A lost response must not leave a run orphaned; the token changes on every delivery. A consumed token does not resume a waiting run. |
| R-SY-20 | agent.invocation is written later as long as attachments or a server transcript are not final. |
Agents never see unchecked bytes; holding it back in the delivery would block the stream of the consumer. |
| R-SY-21 | trigger_state changes to queued only with the first successful delivery. |
If the agent is offline, the person sees "the server has it, the agent does not yet". |
| R-SY-22 | Only triggering messages become follow-ups; another agent is rejected. | A channel that requires mentions does not push conversations between people into a running turn. |
| R-SY-23 | Fencing of the agent consumers through holder_session_id. |
The rule "one consumer, one connection" also holds across several api nodes. |
| R-SY-24 | Import batches write thread.updated instead of message.created per message and set threads.unread_floor_seq. |
An import can write a multiple of the normal traffic; clients should not receive every message individually. |
| R-SY-25 | Ephemeral events between api nodes through NOTIFY ephemeral only with IDs, only in the multi mode. |
NOTIFY stays a wake-up signal without content (R-Stack); a single node does not need the channel. |
| R-SY-26 | Live statistics as a subscription op with an ephemeral frame per interval on the gateway, not as a durable event and not as a separate WebSocket channel. | A frame is worthless after one interval; the gateway already brings login, permission checks and close codes (25-visualization.md, R-VZ-05). |
| R-SY-27 | Live details of the control room come through the optional focus member background_thread_ids within sync.max_focused_threads: run document and run.updated only, never "Active in the thread"; no overview op and no overview event (addendum of 2026-10-24). |
The focus already delivers the whole document and its patches under run.read; idle is one flag per connection and follows the window state (15 section 10), so card threads need their own list to stay out of the notification rules (13 4.4); a card that nobody sees needs no live data, and a second fan-out path would need its own receive rule. |
18. Configuration values of this chapter¶
| Name | Default | Meaning |
|---|---|---|
sync.event_retention |
30d |
durable events stay available for catch-up this long |
sync.event_payload_max_bytes |
65536 |
largest payload of an event |
sync.ws_max_frame_bytes |
1048576 |
largest WebSocket frame |
sync.identify_timeout |
10s |
time between hello and identify |
sync.heartbeat_interval |
30s |
interval of the heartbeats (reported in hello) |
sync.heartbeat_miss_limit |
2 |
missed heartbeats until the connection is closed |
sync.resume_window |
5m |
resume is possible this long after a disconnect |
sync.reauth_lead_time |
60s |
lead time of reauth_required before the token expires |
sync.reconnect_backoff_base |
1s |
base of the backoff when reconnecting |
sync.reconnect_backoff_cap |
60s |
largest interval when reconnecting |
sync.catchup_page_size |
500 |
events per catch-up page |
sync.catchup_max_events |
50000 |
a larger backlog yields resync_required |
sync.cursor_advance_every |
1000 |
one cursor frame after this many skipped events |
sync.outbound_queue_max |
10000 |
frames in the outbound queue of a connection until 4014 |
sync.idempotency_key_ttl |
24h |
retention of Idempotency-Key |
sync.max_focused_threads |
20 |
threads in focus per connection |
sync.run_patch_coalesce_interval |
250ms |
collection window for run.updated |
sync.typing_min_interval |
5s |
typing.start at most this often |
sync.typing_ttl |
8s |
typing indicator without renewal |
client.cache_max_age |
30d |
unused cache entries |
client.outbox_retry_base |
2s |
base of the backoff in the outbox |
client.outbox_retry_cap |
5m |
largest interval in the outbox |
agent.invocation_token_ttl |
60s |
validity of an invocation token |
agent.ack_timeout |
30s |
without ack: redelivery |
agent.ack_persist_interval |
1s |
acked_seq is stored at least this often |
agent.max_inflight_events |
100 |
unacknowledged events per consumer |
agent.run_update_min_interval |
500ms |
adapter throttle for run.update |
agent.presence_timeout |
90s |
without a heartbeat, the agent counts as offline |
agent.cancel_grace |
30s |
time for the adapter to report a cancellation |
Values of other chapters that this chapter refers to: auth.dpop_nonce_ttl
(08-identity.md), perf.event_commit_min_per_s
(02-architecture.md), file.invocation_hold_max and
voice.transcript_hold_max (12-files.md), command.autocomplete_timeout and
command.autocomplete_max_choices (09-agents.md),
authz.expiry_sweep_interval (07-permissions.md), stats.live_interval
(25-visualization.md).