Skip to content

M4-14 Sync engine: catch-up and reconciliation

No Date Ruling Reason
D-M4-14-0 2026-10-16 Base int/m1-wave30 at c0ad78b, branch feat/M4-14-sync-catch-up in its own worktree; no local merge, no push and no pull request from this build, the PR lane lands it. Brief and spec were read from main; the brief relies on no core rule or contract change that the base lacks (R-SY-14 and R-CL-03 are in the German chapters 04 and 15 of the base, D-W-115 was ported earlier, D-W-100 and D-W-114 are planning rulings), so nothing is ported. The base names the chapters 04-sync-protokoll.md and 15-client-architektur.md; text this ticket adds there is English. Names that differ from the brief (D-W-100, D-W-114): the sign-out notice is SignedOutNotice (SignedOut is an event of auth), the REST reads are the port Fetcher with ApiFetcher over the generated API, Dispatcher.apply takes a DecodedEnvelope (the raw envelope with the DecodedEvent of api_client, null when malformed) plus the stored cursor and an AfterCommit, Fetcher.myReadStates takes a channel because the generated listMyReadStates requires channel_id, the write path of drafts calls CatchUpListener.applyDraftPutResult, and a failed gap fill is retried by CatchUpListener.onFocus. The wire mappers (wire.dart) and the identity port (LocalIdentity, AuthIdentity) are files beyond the brief's list. Instructions of the launcher, which override the brief's landing step; merged names win over the brief.
D-M4-14-1 2026-10-16 A gateway page is the run of dispatch frames up to the next cursor frame or catchup.complete; it is applied in one commitPage with the seq of that frame (never below the stored cursor) and the epoch of the session. In the live phase every durable dispatch is its own transaction with cursor = seq; ephemeral events apply at once without a transaction. A disconnect drops an unfinished page; its events come again from the stored cursor. Design note 1; 04 7.1 (the server sends cursor at the end of every page) and 04 9 item 2.
D-M4-14-2 2026-10-16 ready without a stored cursor empties the mirror and applies the whole snapshot with cursor = head_seq and the epoch in one transaction and goes live at once, because an identify without a cursor gets no catch-up and no catchup.complete (04 6.3 item 4; red team of M4-14); with a cursor only the parts that are not events apply (active run summaries by version, the counters, the agent presence, the feature flags into a stored instance row). After a whole snapshot the drafts are loaded with listMyDrafts when a draft reference is newer than or missing from the local drafts, the read states with listMyReadStates for the channels with unread threads, and the instance with getInstance (its flags replaced by the snapshot's). Design note 1; without a cursor the mirror belongs to no known time line; the snapshot names drafts only by version and carries counters without a read state version.
D-M4-14-3 2026-10-16 The outbox, the run store and the outbox start are hooks (onEcho, onRunUpdated, onLive), constructor parameters with no-op defaults, not dependencies. onEcho runs after the commit for a message.created with client_msg_id whose author is the person (or when the person is not known yet, after a resume in a new process); a truncated message.created echoes after its reload. Design note 2; M4-17 wires them; outbox IDs are UUIDv7 of this device, so an echo of another author never matches an entry.
D-M4-14-4 2026-10-16 access.changed in the middle of a page applies in its seq position (removal inside the page transaction, file bytes and TargetsRemoved after the commit) and the rest of the page applies after it. Only 4012 and session.revoked of this session stop a page: the page rolls back, the database is wiped (LocalDatabase.wipe), SignedOutNotice(deleteLocalData: true) is emitted and nothing is applied after that. 4012 reaches the listener as the connection state signedOut/revoked (watchConnection). Design note 3; 04 8.1 and 8.3 item 2: the gateway withholds only events of removed targets, so dropping the rest would lose events that are never sent again.
D-M4-14-5 2026-10-16 session.revoked wipes only when its session_id is this device's session (LocalIdentity.sessionId, the sid of auth) or the own session is unknown; a revocation of another session of the person (a logout on the phone) changes nothing here. session.revoked has the audience self and reaches every device of the person (05 6); the revoked device itself is also closed with 4012 (04 8.5).
D-M4-14-6 2026-10-16 Gaps: a message.created that continues a loaded range, or arrives in a thread with no loaded range, or lies below every loaded range joins the loaded ranges; otherwise it is stored without joining them, the gap from the end of the highest range below it up to it is recorded, and Fetcher.range loads it after the commit (listThreadMessages with after_seq, page by page); the fetched messages and the whole range up to the new message join the loaded ranges in one transaction. MessagesDao.watchContiguous therefore shows only the contiguous range ending at the newest loaded message. A failed fill keeps the gap recorded; onFocus retries it. Design note 4; 04 4: a real gap is missing data, and a thread never opened here shows from its newest message without loading its history.
D-M4-14-7 2026-10-16 Personal data: the echo of this device (updated_by_device_id) is ignored for drafts, settings and the sidebar; the read state is merged also for the own echo, because mergeReadState is monotonic and versioned and the counters come only from the server. A draft.updated or draft.deleted for a draft the person is typing on this device (DraftEditing) is not applied; the next local write wins. Design note 5 and 04 14.
D-M4-14-8 2026-10-16 A page whose transaction fails (a payload that breaks its mapping, a database error) leaves rows and cursor as they were; the listener then applies nothing until the next ready or resumed, which catches up from the stored cursor. An envelope that does not fit the schema of its type is skipped like an unknown type and the cursor passes it. A later page must not move the cursor past events that were never applied; SessionControl has no close, and a reconnect repeats the page.
D-M4-14-9 2026-10-16 When getSnapshot fails during a resync, the listener keeps mirror and cursor and retries the snapshot after the reconnect backoff (sync.reconnect_backoff_base, sync.reconnect_backoff_cap, full jitter) on the same connection; frames that arrive meanwhile wait in memory and apply after the snapshot with the duplicate rule. A disconnect ends the retries; the next session catches up from the stored cursor and gets resync_required again. onLive runs once per session, also when a catchup.complete follows a resync. Code review of M4-14: dropping the mirror and halting left the app empty on an open connection until the next reconnect; after a restore every client asks for the snapshot at once.
D-M4-14-10 2026-10-16 packages/local_store gets three methods beyond the brief's list, each with its test: PersonalDao.deleteSettingIfNewer and deleteNotificationPreferenceIfNewer (for deleted: true, 04 14) and MessagesDao.reactionsOf (reading the me flag of a reaction inside the page transaction, where a stream would wait for the transaction's end). Fixed requirement 17 and Design note 5 need the deletions; no existing method removes those rows.
D-M4-14-11 2026-10-16 decideApply decides per event between the version rule and the seq rule and drops a versioned type that arrives without resource.version; the comparison with the stored version is the conditional upsert of the DAOs (D-M4-12-3), one statement per row. Event payloads, the snapshot and REST answers go through one set of wire mappers from JSON to the domain types; the generated models of REST answers are serialized back to JSON first. The page size for the isolate decision is an estimate over the decoded frames (string lengths plus a few bytes per value). One rule in one place; the gateway frames are already JSON-decoded by the connection, so their raw size is not at hand.
D-M4-14-12 2026-10-16 REST reads that run after a commit (a truncated object, a gap fill, the newest page of a focused thread, the structure of added or changed targets, drafts and read states after a snapshot) store their result only when, inside the transaction of the write, the target is still held: content needs its channel row and neither the channel, its space nor the thread may have been removed in this process (SyncContext.removedTargets); a view of changed needs its row. An access.changed with added (for its spaces and channels and for every thread its structure load lists), a thread.moved and a snapshot that lists a target take it out of the removed set (follow-up review of M4-14: a moved-out thread stayed blocked after its new channel became visible). Code review of M4-14, reproduced: a getMessage in flight during access.changed with removed stored the message and its search row again (04 8.3 item 3, T-DV-01).
D-M4-14-13 2026-10-18 The ticket was ported from its integration base onto main with its own commits and the integration fix INT-W33/3 (the doc comments of fetcher.dart and sync_events.dart name ApiClient and AuthSession, so make docs resolves them); the English chapter 15 of main takes the page boundary of 7.3. The event catalogue of main has two durable types more than the base, device.pending and gdpr.request_due; neither has a local table, so they join the skipped set of catalogue_test.dart and the README. Tickets are built in parallel on integration bases, main takes each as one pull request with the integration fixes it needs; the other fixes of that wave belong to M6-01.
D-M4-14-14 2026-10-19 The snapshot retry of a resync (D-M4-14-9) checks its session after the snapshot fetch and after the follow-up steps of its commit (markSnapshot, _afterCommit, which always run once the snapshot is committed, because only their reads restore the instance row and the read states) and ends when a new session started; only the retry of the current session sets lastProcessedSeq, clears _resyncing, sets live, reports Resynced and drains the held frames. The drain flag names its session (_drainingSession): a drain of an ended session no longer makes the next session hold its frames. CodeRabbit on the pull request of M4-14, reproduced with resync_session_test.dart: a disconnect and a new ready while the retry awaited its commit set the new session live during its catch-up and reported Resynced for it, and frames of a new session that arrived while the old drain was inside a frame stayed in the hold list after that drain ended, so the next cursor or catchup.complete moved the cursor past them (04 9, D-M4-14-1).