M3-07: Attachments and release of the agent invocation¶
Status: ready | Depends on: M1-58, M2-11, M3-04 | Milestone: M3
Goal¶
After this ticket a person, an agent or an app sends a message with up to message.max_attachments of its own files that were uploaded for the channel of that message and whose upload session is completed: createMessage and createThread (first_message) accept MessageCreate.attachment_file_ids (until now 400 validation_failed with errors[].code not_enabled, M1-56), require +upload, bind every file to exactly one message in message_attachments with the copies thread_id and channel_id of that message, set messages.has_attachments, and refuse every other file with 409 conflict and reason file_not_attachable (12 10, SEC-UP-9, R-DM-22, R-DM-43). An attached file is read through the channel of its message, appears in every representation of the message, follows its thread when the thread moves (same transaction) and goes with its message when the message is deleted (same transaction, has_attachments back to false). A message that triggers an agent while one of its attachments is not final keeps trigger_state = received and writes no agent.invocation: the transaction that makes the last attachment final writes it (ReleaseInvocation, subscriber of files.FileFinalized), and the job runs.invocation_hold_timeout writes it after file.invocation_hold_max with the attachments in their current state, exactly once either way (04 15.3 point 1, 12 10, R-FI-09, RK-43). No agent ever receives an unchecked byte; after a timeout it receives an invocation whose attachments it can see as pending, never their content. The attachment facts this ticket writes are what the search coordinator (M3-14) filters on.
Read first¶
docs/PLAN.md: 3 "Way of working per ticket"; 5 "Test strategy" (row "Test doubles": clamd container); 9 "M3 Platform services" (Goal, Scope, Out of scope, Exit criteria 1 and 2, Test strategy, Red team "a file across space boundaries in the context of another agent (SEC-UP-9)", Dependencies, Risks "a long outage of clamd holds back invocations with attachments up tofile.invocation_hold_max"); the rows M3-03 and M3-04 (what exists), M3-07 (this ticket, the contract), M3-05 and M3-06 (beside it, not in its closure), M3-08, M3-09, M3-12, M3-14, M3-15 (who builds on it); 8 rows M2-05, M2-07, M2-08, M2-11, M2-13, M2-19, M2-31; 7 rows M1-56, M1-57, M1-58, M1-62, M1-70; 14 row M8-03.- Dependencies: the header names M1-58, M2-11 (follow-ups and the follow-on run) and M3-04. Implied through them and therefore not in the header: M2-08, M1-62, M3-03 (tables
files,file_upload_sessions,file_upload_parts,getFile, thefile.updatedbuilder,authorize.FileFacts,file.readno longer deferred), M3-01 (object store), M2-07 (pollAgentEvents), M2-06, M2-05 (Open, acceptance), M2-04, M2-02, M1-71 (shared test harness), M1-70 (test/leakand the SQL visibility linttools/lint/sql_visibility.py), M1-69 (test/canary), M1-68 (instrument registry), M1-65, M1-64, M1-63, M1-61 (drafts andclear_draft), M1-60, M1-56, M1-55, M1-54, M1-51, M1-28, M1-24, M1-16, M1-14, M1-13, M1-12 and the rest of their closure. Not in the closure but used if merged (each place says how): M1-57 (history), M1-59 (reactions and pins in the delete path), M2-13 (run context), M2-15 (the owner's switch pattern), M2-19 (agent answer branch), M3-05 (download), M3-06 (images). docs/spec/12-files.md: 1 "Goals and limits" point 1; 2 "States of a file" (definitions "servable" and "final", the rule that every state change writesfile.updated); 3.4 "Completion and verification pass" step 3 (a completed upload may be attached at once); 3.7 "Purpose of a file"; 10 "Attaching and agent invocation" completely; 12 "Retention, deletion, deduplication" row "Message deleted"; 15 "Tests" rows "Attaching the same file a second time" and "Agent invocation with an attachment while clamd is down"; 16 R-FI-08, R-FI-09; 18file.invocation_hold_max.docs/spec/03-data-model.md(core contract, read only): 1 "Principles" item 12; 4.5threads(paragraph "A move changeschannel_id"); 4.6messages(has_attachments, its CHECK, the paragraph "Deletion emptiescontent_md, setshas_attachmentstofalse"); 4.7files,file_upload_sessions,message_attachments(all columns and the paragraphs after the table); 4.8runs(includinginvocation_held_until, R-DM-44, Design note 2, and the partial index of active runs(state, agent_id)); 6.3 "Lock order"; 6.5 "Redaction on deletion"; 8 R-DM-22, R-DM-43, R-DM-44; 9message.max_attachments.docs/spec/04-sync-protocol.md(core contract, read only): 2.2 "Envelope" (actorabsent for jobs without principal, payload rules 4 and 5); 9 "Delivery guarantee" point 3; 15.3 "Invocation and run start" points 1 and 2; 15.4 "Follow-ups" points 1, 2 and 4 (Design note 11 e); 17 R-SY-20.docs/spec/05-actions.md(core contract, read only): 2.2 "Target of a resource" (rowsfile, the paragraph "An attached file always follows its message"); 4.7 rowmessage.create("with attachments additionally+upload"); 4.8 rowsfile.read,file.delete; 5 "Agent invocationagent.invoke" (only to see that the release writes no new check); 6 rowsagent.invocation,file.updated,message.created.docs/spec/02-architecture.md: 4.1 "Rules" point 4; 4.3 "Ports, events and sources per module" (rowsfiles,messaging,runs); 4.4 "Allowed dependencies" (rowsfiles,messaging,runs, "Tiers"); 8.1 to 8.3 (subscribers, veto); 8.5 "What never happens in a transaction"; 12.3 "Job catalogue" (rowsruns.invocation_hold_timeout,files.purge_objects); 12.4 "Rules for jobs"; 13 "Configuration loading" item 5.docs/spec/09-agents.md: 2.2 "Features" (row "Sending files",supports_files_out); 2.3 "Grants of an agent"; 3.4 "Decision per message" point 7; 4.6 "Run context" (attachmentswithscan_status, the paragraph "The adapter loads attachments withfile.read"); 4.7 row "Send file"; 5.2agent_unavailable; 6 "Trigger state (trigger_state)" (receiveduntil the first delivery).docs/spec/07-permissions.md: 4.1 (capability_missing), 4.3 "Visibility", 7.4 "Query pattern" rules 1 and 3; 11.2 AZ-P4, AZ-P12.docs/spec/19-security.mdSEC-UP-9.docs/spec/20-privacy.md4.2 "Deleting a message", 4.3 "Files".docs/spec/14-search.md4.3 to 4.5 (the consumer ofmessage_attachmentsandhas_file).docs/spec/06-api.md5 "Errors" points 3, 5 and 7.docs/api/openapi.yaml: operationscreateMessage,createThread,getFile,moveThread,deleteMessage,pollAgentEvents,listEvents,updateAgent(andgetMessage,listThreadMessages,downloadFile,createFileTicket,getRunContextwhere their tickets are merged); schemasMessageCreate,ThreadCreate,Message,Attachment,File,FilePayload,FileUpdatedEvent,MessageCreatedEvent,AgentInvocationPayload,AgentInvocationEvent,Problem(x-problem-codes:conflict,validation_failed,forbidden,feature_disabled),FieldError.docs/api/asyncapi.yaml: messagesevent.file.updated,event.agent.invocation.- ADRs 0007, 0008, 0017, 0023, 0033, 0050, 0052, 0053, 0054.
docs/DECISIONS.md: D-FI-07, D-FI-08, D-LS-14, D-K-A6, D-K-G6 (the mechanismfiles.FileFinalizedplusruns.invocation_hold_timeoutinstead of a port call), D-W-100, D-W-104 (R-DM-43, R-DM-44 and the further amendments), D-W-106, section 10 (format of the rowsD-M3-07-<n>).docs/security/threat-model.mdT-AI-01 ("files only in their own channel (SEC-UP-9)"), T-AI-03, T-UP-01;docs/RISKS.mdRK-43.docs/operations/configuration.md:file.invocation_hold_max,message.max_attachments,run.queued_max_age,database.tx_retry_max;docs/operations/upgrade.md4 "Rules for migrations (Expand/Contract)" (M3, M4, M5, M6).- Briefs (merged names win, D-W-100):
M3-04-clamd-scan.md(files.FileFinalized,domain.Final,domain.Deliverable, the outcome write of its Design note 6,clamdtestwith the fake clamd server),M3-03-upload-sessions.md(Design notes 15 and 16: visibility of kindfile,FileFacts, thefile.updatedbuilder ininternal/files/app/events.go, audience before attaching),M2-08-invocation-delivery.md(acceptance inOnMessagePosted, Design notes 10 f and 14: the order inside acceptance and theagent.invocationbuilder;runs.Delivery; the jobruns.queued_expiry),M2-05-runs-state-machine.md(Open,OpenCmd,Transition,SetTriggerState),M2-07-agent-consumers.md(pollAgentEvents),M1-58-messaging-edit-delete.md(the delete use case, its steps 3 to 9 and the lock order of its note 4),M1-62-chat-thread-move.md(chat.ThreadMoved, the subscribers ofmessagingandsync),M1-56-messaging-send.md(D1, D2, D3, D7, D8, D9,MessagePosted),M1-55-chat-threads.md(AppendToThreadreturnsAppended.Thread),M1-28-authz-core.md(Resource.HasAttachments,CapIf(upload, HasAttachments)),M1-13-unit-of-work.md(txn.Subscribe,txn.Publish,testdata/txn_registrations.golden),M1-14-background-jobs.md(jobs.Args,jobs.Register,Inserter.InsertTx,InsertOptions.ScheduledAt),M1-16-event-log.md(eventlog.Appender,AudiencePrincipalID),M1-51-agents-records.md(operator_config,OperatorConfigV1),M1-12-postgres-migrations.md(pgtest, migration numbering),M1-61-sync-personal-data.md(note 12:clear_draftdeletes the draft in the message transaction),M1-70-leak-tests.md(SQL lint rules 1 to 7 and the exception categories),M1-01-server-gates.md(migration rules ofmake migrations-lint),M2-11-run-followups.md(note 8 c: the follow-on run uses the invoke step of the acceptance). If merged:M2-15-components-interactions.md(note 6 b: the owner's switchsupports_components). Consumers:M3-14-search-coordinator.mdDesign note 15,M3-15-file-text-extraction.mdDesign note 10,M3-06-image-metadata-thumbnails.mdDesign note 16,M2-13-run-context.mdrequirement 18. - Code (all under
services/server/):internal/files(port,domain,app/events.go,app/get.go, the outcome write of M3-04,adapters/postgres,adapters/jobs,adapters/clamd/clamdtest),internal/messaging(create, delete, representation,MessagePosted),internal/runs(acceptance,Open, theagent.invocationbuilder,adapters/jobs),internal/chat(ThreadMoved,Threads.Info),internal/agents(port),internal/kernel/authorize,internal/authz(visibility of kindfile),internal/wire,migrations/,architecture/modules.yaml,tools/archgen.
Fixed requirements that apply¶
Owner's fixed build rules (apply verbatim):
- No MVP. New from scratch and directly in the final state; stages are only the order.
- Strictly modular: clear module and service boundaries, dependencies only through defined interfaces, no cross access.
- Documentation in code: every public module, class, function and interface with a docstring/doc comment (purpose, parameters, return value, errors, example where useful); comments explain the why, not the what. API documentation is generated from code (OpenAPI from the routes, dartdoc, Python API docs in MkDocs).
- Documentation in the repo: README per module/service, ADR for every architecture decision, CHANGELOG per release, keep the documentation site current. A change without matching documentation is not done.
- Quality gates (SonarQube, linters, type checking, security scanners, test coverage) block the merge.
Rules for every brief:
- Test first: write the failing test, run it and keep the red output for the report, then make it green, then clean up.
- No en dash and no em dash in any file you write; the
text-rulespre-commit hook rejects them. Use a hyphen or rephrase. - White label: examples and test data use
example.comand the people Alex, Sam and Robin; no company, host or personal names, no paths under a home directory. - Limits, windows and thresholds only through named configuration keys listed in
docs/operations/configuration.md; code comments and docs name the key, never the number. - Specification first: a change to behaviour, API, event, right or configuration updates
docs/spec/ordocs/api/in the same change. The core contract (docs/spec/00-glossary.md,03-data-model.md,04-sync-protocol.md,05-actions.md) is not edited by this ticket.
Product requirements implemented here:
- Attaching (12 10 points 1 to 3, SEC-UP-9, R-DM-22): every file was uploaded by the author (
uploaded_by), haspurpose = 'attachment',scope_channel_idequal to the channel of the message, an upload session in statecompleted,upload_stateother thanfailed, nodeleted_atand no attachment yet; otherwise409 conflictwithreasonfile_not_attachable. At mostmessage.max_attachmentsfiles per message; a file hangs on at most one message (unique index onmessage_attachments(file_id)). message.createwith attachments needswriteand+uploadon the thread's channel (05 4.7 rowmessage.create);attachment_file_idsis accepted bycreateMessageand bycreateThreadinfirst_messageand no longer answers400 validation_failedwitherrors[].codenot_enabled(06 5 point 3, M1-56 D2).message_attachmentsexactly as 03 4.7 with the copiesthread_idandchannel_id(R-DM-43):fileswrites them at attach from the message thatmessaginginserted in the same transaction and passes through thefilesport;fileschangeschannel_idin the transaction ofmoveThread(subscriber ofchat.ThreadMoved); no foreign key on the copies;filesnever readsmessagesand never usesfiles.scope_channel_idfor the copy (03 1 item 12).messages.has_attachmentswith the CHECKdeleted_at IS NULL OR NOT has_attachments:trueexactly when the message was created with at least one attachment and is not deleted;messagingsets it in the INSERT and resets it in the UPDATE that deletes the message; in that same transaction the attachment rows go and the files getdeleted_at(03messages, 12 12, 20 4.2 and 4.3).- Hold (04 15.3 point 1; 12 10 "Invocation only after the check"; R-FI-09): a triggering message with an attachment that is not final (12 2 "final") writes no
agent.invocationat acceptance and keepstrigger_state = received; the transaction that makes the last attachment final writes it (files.FileFinalized, subscriber inruns,ReleaseInvocation); the jobruns.invocation_hold_timeoutwrites it afterfile.invocation_hold_maxwith the attachments in their current state; every run gets exactly oneagent.invocation. - A released invocation carries no file content; the agent learns attachment states only through reads that
authorizechecks (getFile, the message representation, the run context) and gets bytes only whenscan_status IN ('clean','skipped')(12 1 point 1, 09 4.6, 04 2.2 rule 4). - An attached file is read through the channel and thread of its message (05 2.2 rows
file);files.scope_channel_idcounts only before attaching and never changes (03threads,files). file.updatedof an attached file carries the scope of its message and no audience; before attaching it goes to the uploader only (05 6 rowfile.updated; M3-03 Design note 16).- Every representation of a message (
Message.attachmentsin REST answers,message.created,message.updated) lists its attachments in position order with theFileobject; no JSONnull(ADR 0053); a deleted message lists none (04 2.2 rule 5). - An agent sends files only with
+upload(09 2.3) and only while the owner's switchsupports_files_outinagents.operator_configis notfalse(09 2.2); otherwise403 feature_disabled(06 5 point 5). - Lock order (03 6.3, 02 8.2): thread row, then message rows, then
filesrows in ascendingid, thenmessage_attachments, thenruns; no path takes them in another order.
Scope¶
All paths under services/server/ unless they start with docs/ or infra/. Where M3-03, M3-04, M2-05, M2-08, M1-56 or M1-58 already created a file named below, extend it.
Create:
migrations/<next>_files_message_attachments.sql,migrations/<next + 1>_messaging_has_attachments.sql,migrations/<next + 2>_messaging_has_attachments_validate.sql,migrations/<next + 3>_runs_invocation_held_until.sql(Design note 2), their lines inmigrations.sha256throughmake migrations-lock.internal/files/attachments.go(port):Attachments,AttachCmd,Attachment,AttachmentRef(Design note 3).internal/files/domain/attach.go:Attachable(AttachFacts) bool,EventTarget(attached bool) TargetandBecameFinal(before, after StateFacts) bool(Design notes 4 and 10), standard library andkernel-coreonly.internal/files/app/attach.go(the five port methods),internal/files/app/resource.go(fileResource, Design note 9),internal/files/app/filechange.go(recordFileChange, Design note 10, unless the merged code already has one function that appendsfile.updatedfor every state change; then extend that one),internal/files/app/subscribers.go(OnThreadMoved).internal/files/adapters/postgres/queries/attachments.sqland the regenerated sqlc code.internal/runs/domain/hold.go:HoldUntil,ReleaseCause(ReleaseFinal,ReleaseTimeout),ReleaseVerdict(Design notes 11 and 12).internal/runs/app/hold.go:invokeOrHold,ReleaseInvocation,OnFileFinalized, the hold timeout use case;internal/runs/adapters/jobs/hold.go(argsInvocationHoldTimeout{RunID ids.ID}, kindruns.invocation_hold_timeout, handler, scheduler overjobs.Inserter);internal/runs/adapters/postgres/queries/hold.sql.
Change:
internal/messaging: port (MessagePosted.HasAttachments, the create command withAttachmentFileIDs),domain(body rules),app(create, delete, representation), postgres queries (has_attachmentsin the INSERT, in the delete UPDATE and in the reads) (Design notes 5 to 7).internal/runs:OpenCmd.InvocationHeldUntilof M2-05'sOpen; the acceptance subscriberOnMessagePosted(M2-05accept.go, extended by M2-08; follow the merged file) for the hold (Design note 11) and the agent branch of Design note 15.internal/files:app/events.go(scope and audience, Design note 10),app/get.goand every other caller offile.readinfiles(Design note 9), the outcome write of M3-04 (publish moves intorecordFileChange).internal/authz: only if the merged visibility of kindfilelacks the attached row of M3-03 Design note 15 (read on the path of the hintsChannelIDandThreadID): add it with a test.internal/agentsport: only if no read returnsoperator_config(M2-15 adds one where it is merged):Agents.OperatorConfig(ctx context.Context, agentID ids.ID) (OperatorConfigV1, error)with the type of M1-51, its query and a test.internal/wire: subscriptionsfiles.FileFinalized -> runs.OnFileFinalizedandchat.ThreadMoved -> files.OnThreadMoved(aftermessaging.OnThreadMovedandsync.OnThreadMoved); jobruns.invocation_hold_timeout(queuedefault, rolesworkerandall); thefiles.Attachmentsport formessagingandruns; config keys of Design note 16 inconfigkeys_files.goand the messaging key file; regeneratetestdata/txn_registrations.goldenand the jobs golden file if one exists.architecture/modules.yaml:filesinruns.depends_on; regenerate.go-arch-lint.yml(cd services/server && go generate ./tools/archgen).tools/lint/sql_visibility.yamlat the repository root, where M1-70 created it: the exceptions of Design note 16.- The M1-56 test that asserts
not_enabledforattachment_file_ids(TestCreateMessage_NotEnabledMembers400or its merged name): change only the attachments row to the new rule; keep the other rows. docs/spec/02-architecture.md4.3 and 4.4,docs/architecture/c4-3-server-components.mmdand its.svg(Design note 1);docs/spec/12-files.md2 and 10,docs/spec/09-agents.md2.2,docs/api/openapi.yaml(descriptions only) (Design note 17).docs/operations/configuration.mdregenerated (make config-reference).- READMEs of
files,messaging,runs(andagentsif its port changed);services/server/CHANGELOG.mdunder "Unreleased";docs/DECISIONS.mdsection 10 rowsD-M3-07-<n>(Design note 18).
This ticket enables no operation in api/operations.yaml (createMessage, createThread, getFile, moveThread and deleteMessage are enabled), adds no action, no event type, no metric and no top-level Makefile target.
Out of scope¶
- Upload sessions and
getFileitself (M3-03); the scan pass,files.scan,domain.Finaland the in-transaction eventfiles.FileFinalized(M3-04); download, fetch tickets,409 file_not_readyand409 file_blocked(M3-05); metadata removal and thumbnails (M3-06). This ticket only changes their call sites where Design notes 9 and 10 say so. - Deleting objects of deleted files, the jobs
files.purge_objectsandfiles.cleanup_pending, never attached files,deleteFile, the redaction offile.updatedpayloads of deleted files and DP-T01 for files (M3-08); this ticket setsfiles.deleted_atand leaves the objects to the purge job. - Voice messages,
kind = voice(stays400 validation_failedwitherrors[].codenot_enabled), transcripts andvoice.transcript_hold_max(M3-09); it extends the hold condition of Design note 11. - Reply attachments of the Claude Code runner (M3-12); files of the Hermes adapter (plan gap of M2-33); import files and
IngestFile(M8-03). - Search over attachments and
has_file(M3-14); text extraction and its deletion with the file (M3-15). - The attachment part of AG-20 in the conformance suite (its line
AG-20 attachment M3-07intestdata/deferred.goldenof M2-31): it needs M2-31, M2-13 and M3-05, none of which is in this ticket's closure; M3-09 closes it (D-W-108). - Holding
agent.followupof a follow-up whose attachments are not final: not held (Design note 11 e; the core contract defines the hold only foragent.invocation, 04 15.3 point 1, 12 10). Bytes stay protected by the download rule of 12 9.3. The follow-on run of M2-11 is held like an acceptance (Design note 11 d). - The adapter half of the capability
files_out(heartbeatcapabilities.files_out, M2-20, which is not in this ticket's closure); this ticket enforces only the owner's switch, as M2-15 does forsupports_components. Reason: the capability gates what an agent sends, and an adapter that sends a file shows by the send that it can; the owner's switch is the half that can forbid it (09 2.2 "Reason"). - Ending runs whose triggering message was deleted (M2); the deletion journal (M7-07); client rendering of attachments (M4, M5); notifications, which never wait for attachments (12 10 last sentence).
- Changes to the core contract.
Boundaries¶
- A new migration needs its line in
migrations.sha256: runmake migrations-lockafter creating the file and beforemake check; never edit a merged migration (M1-01, ADR 0007). - Work in a dedicated git worktree on the branch
feat/M3-07-attachments-invocation; one writer per worktree. - Conventional Commits with the module as scope (for example
feat(files): ...,feat(runs): ...) and a DCO sign-off on every commit (git commit -s); squash into meaningful commits before the merge (docs/quality/push-discipline.md). - Never push to
main, never tag and never create remote repositories; the only remote steps are the branch push and the pull request ofAGENTS.md. - Do not modify files outside the scope above, except the CHANGELOG, README and
docs/DECISIONS.mdentries named in this brief. - Follow the host rules supplied by the launcher (ports, machine rules).
- Tests bind only
127.0.0.1and port 0; PostgreSQL runs only as a throwaway container on a random port throughpgtest(P-09, P-10). Without a Docker-compatible runtime the integration tests fail; they never skip. - Stop only processes you started, by PID. Temporary directories only via
mktemp -d, removed at the end. - Never read or print secrets. Never log, print or persist an invocation token. No network calls except package downloads for pinned tools and Go modules.
- Never lower a threshold, disable a gate or delete a test.
- If the core contract lacks something this ticket needs, stop with
Verdict: BLOCKED <reason>instead of building a deviation (P-11). The attachment facts are in the core contract since D-W-104 (03message_attachmentsandmessages, R-DM-43), and so is the marker of a held invocation (03runs.invocation_held_until, R-DM-44); build them as written. - This brief was written before milestone M1 closed (D-W-100). Where the merged code of an M1 or M2 ticket names a package, type, function, file or make target differently from this brief, follow the code and list each difference in the report; where it behaves differently from the specification, stop with
Verdict: BLOCKED <reason>. - The same holds for the merged code of M3-01, M3-03, M3-04 and M2-11 (and M3-05, M3-06, M2-13, M2-19 where they are merged): follow their names (repositories, the
file.updatedbuilder, the outcome write,FileFinalized,Open, the acceptance file, theagent.invocationbuilder) and list each difference. - clamd runs in tests only as the fake clamd server of
clamdteston a Unix socket in amktemp -ddirectory, and in the one test marked "real clamd" as the throwaway container ofclamdtest(pinned image, no signature download, loopback only); stop only containers you started, by id. No real third-party service, no agent runtime and no model call: the adapter side ispollAgentEventswith an agent token. - Time only through
kernel/clock; tests move a fake clock and call job handlers directly, they never sleep for a window. - The SQL visibility lint of M1-70 is on the base branch: every new reading query on a content table carries the visibility predicate or an exception entry with category and reason (Design note 16); never widen an existing exception.
Design notes¶
-
Dependencies and ports (02 4.1, 4.3, 4.4; ADR 0008, ADR 0050).
runssubscribesfiles.FileFinalized(02 4.3 rowruns) and asksfileswhether the attachments of a message are final, sorunsimports thefilesport; 02 4.4 does not list that edge yet. Ruling for docs/DECISIONS.md: addfilesto "May use" ofrunsin 02 4.4 (reason: "write the held invocation onfiles.FileFinalized; read the attachment state"), toruns.depends_oninarchitecture/modules.yaml, the edgeruns --> filestodocs/architecture/c4-3-server-components.mmd(render withmake diagrams) and to the dependency line of therunsREADME;make register-checkholds the four equal (M1-06 C7, C11, C12). The edge points down (filesis belowmessaging,runsabove it), so the graph stays acyclic. Ruling for docs/DECISIONS.md: the source portAttachmentLocatorof 02 4.3 is not built, because the copies of R-DM-43 givefilesthe path of every attached file in its own table; remove it from the rowsfilesandmessagingof 02 4.3. -
Migrations (goose; sequential numbers and slugs of
make migrations-lint; use the next free numbers when you rebase before the merge and renumber yours if a parallel ticket took them).a.
<next>_files_message_attachments.sql: tablemessage_attachmentsexactly as 03 4.7:message_id uuid NOT NULL,file_id uuid NOT NULL,position smallint NOT NULL,thread_id uuid NOT NULL,channel_id uuid NOT NULL; PK(message_id, position); FKmessage_id -> messages(id) ON DELETE CASCADE, FKfile_id -> files(id) ON DELETE RESTRICT; no foreign key onthread_idorchannel_id(R-DM-43); UNIQUE(file_id); index(thread_id). The table is new and empty, so constraints and indexes are created with it. No column, CHECK or index that 03 does not name.b.
<next + 1>_messaging_has_attachments.sql:ALTER TABLE messages ADD COLUMN has_attachments boolean NOT NULL DEFAULT false(constant default, no rewrite, upgrade.md M3) andADD CONSTRAINT messages_has_attachments_deleted_check CHECK (deleted_at IS NULL OR NOT has_attachments) NOT VALID(M5).<next + 2>_messaging_has_attachments_validate.sql:VALIDATE CONSTRAINTof it. Remove the deviation comment of M1-56 in the documentation ofmessaging(its migration stays unchanged).c.
<next + 3>_runs_invocation_held_until.sql:ALTER TABLE runs ADD COLUMN invocation_held_until timestamptz(NULL allowed, no default, upgrade.md M3). The column is core contract (03runs, R-DM-44, D-W-104): set by the INSERT ofOpenexactly when acceptance holds theagent.invocationof the run, to the latest release time; set toNULLbyReleaseInvocationin the transaction that writes the event, which raisesruns.versionby one like every stored change of a run (03runs.version; as M2-08 does forinvocation_jti) but writes norun.state_changed;NULLfor every run that was never held; a run that ends while held keeps the value and is never released. No new index: the held-run lookup of Design note 13 a filters the partial index of active runs(state, agent_id) WHERE state IN ('queued','running','waiting_input')(03runs), because a held run is alwaysqueuedand active runs are few. No CHECK beyond 03. -
Port
files.Attachments(02 4.3 rowfiles:Attachments). Every method runs inside the caller's unit of work (internal error otherwise) and never callsauthorize, because the caller authorized the action (doc comments say so):// Attachments binds uploaded files to messages and answers attachment facts // (12 section 10, 03 message_attachments). type Attachments interface { // Attach checks every file against 12 10 point 1 and inserts one row per file in // request order with the thread and channel of the message just inserted. Attach(ctx context.Context, cmd AttachCmd) ([]Attachment, error) // ForMessages returns the attachments of the given messages in position order. ForMessages(ctx context.Context, messageIDs []ids.ID) (map[ids.ID][]Attachment, error) // DeleteForMessage removes the attachments of a message deleted in the same unit of // work and marks their files deleted (12 12). DeleteForMessage(ctx context.Context, messageID ids.ID) error // MessageOf reports where a file is attached; ok is false for an unattached file. MessageOf(ctx context.Context, fileID ids.ID) (ref AttachmentRef, ok bool, err error) // Pending reports whether an attachment of the message is not final (12 2). Pending(ctx context.Context, messageID ids.ID) (bool, error) } type AttachCmd struct { MessageID, ThreadID, ChannelID, AuthorID ids.ID FileIDs []ids.ID // request order; position = index + 1 Pointer string // JSON pointer of the list in the request body } type Attachment struct { Position int; File File } type AttachmentRef struct { MessageID, ThreadID, ChannelID ids.ID; Position int }messaging.MessagePostedgetsHasAttachments bool(doc comment: "the message was created with at least one attachment"); M2-05'sOpenCmdgetsInvocationHeldUntil time.Time(zero: not held), written by the INSERT ofOpen. -
Attach (12 10 points 1 to 3; Ruling for docs/DECISIONS.md for c and d).
a.
SELECT ... FOR UPDATE OF fileson thefilesrows of the distinct IDs in ascendingid, joined with the state of their upload session (the session row is not locked:completedis final for a session, M3-03); then, in a later statement, themessage_attachmentsrows of these files. Locking first makes a concurrent attach of the same file and the outcome write of M3-04 wait for this transaction.b.
domain.Attachable(AttachFacts{...})is true exactly when the row exists,deleted_atis NULL,uploaded_byequalsAuthorID,purpose = 'attachment',scope_channel_idequalsChannelID, the session state iscompleted,upload_state <> 'failed'and no attachment row exists. Blocked files (infected,error) and files whose scan still runs are attachable, as 12 10 point 1 and 12 3.4 step 3 say.c. Walk the list in request order; the first entry that fails ends the call with
409 conflictwithreasonfile_not_attachableanddetail=<Pointer>/<index>; an ID listed a second time fails as already attached (12 10 point 3). Unknown, foreign, deleted, other-channel, open, aborted, expired and failed files give the same status, code, reason and headers, so the answer tells nothing about files of other people.d. Insert the rows (
position= index + 1,thread_idandchannel_idfrom the command). A unique violation on(file_id)is the last line and maps to the same409withdetail=Pointer; the unit of work rolls back, so the message is not stored and its thread sequence is given back (03 6.2).e. Return the attachments with the attached form of
File(Design note 6). -
Sending (M1-56 D1, D2, D8; Ruling for docs/DECISIONS.md for the empty content).
a. Body rules: more than
message.max_attachmentsIDs give400 validation_failedwitherrors[].codelimit_reachedand pointer/attachment_file_ids(/first_message/attachment_file_idsincreateThread). Thenot_enabledrule forattachment_file_idsgoes. With at least one attachmentcontent_mdmay be empty or white space only (a picture without caption); without attachments the M1-56 rule (errors[].codeempty) stays.kind = voicestaysnot_enabled.b.
authorize(actor, message.create, Resource{..., HasAttachments: len(ids) > 0})at the place M1-56 authorizes (forcreateThreadthe check ofmessage.createon the new thread): withoutuploadthe answer is403 forbiddenwithreasoncapability_missing.c. In the unit of work, after the replay lookup (a replay attaches nothing and answers the stored message with its attachments):
AppendToThread; INSERT the message withhas_attachments = (len(ids) > 0); mentions; thenfiles.Attachments.Attachwith the message id, itsthread_id, thechannel_idof the row just inserted (the channelAppendToThreadreturned under the thread lock), the author and the list; thenmessage.createdandMessagePosted(withHasAttachments) in the merged order. The request hash of M1-56 D3 already holds the attachment IDs in request order; its encoding does not change. A send withclear_draft: trueremoves the files from the author's draft because M1-61 deletes that draft in the same transaction (12 10 point 4; M1-61 note 12); nothing else is needed.d. A message without attachments costs no extra statement (M1-56 D9,
perf.message_create_p99): nofilescall and no flag query. Every caller of the create use case (sends, agent replies, voice, import) inherits the flag and the attach. -
Representation (
Message.attachments, 04 2.2 rule 5). The single representation function of M1-56 fillsattachmentsfromfiles.Attachments.ForMessages, called only for messages withhas_attachments = true, once per request or page (one batched call per page where the merged history of M1-57 builds pages);message.createdand the201use the result ofAttach. EachAttachmenthaspositionand theFileof thefilesrepresentation (M3-03) with two rules: Ruling for docs/DECISIONS.md:scope_channel_idis absent once the file is attached, because after a move it names a channel the reader may not see (05 2.2: the channel of the message counts);sha256only whenupload_state = 'complete'(M3-03). A deleted message showsattachments: [](tombstone of M1-57 and M1-58). Messages carry the file state at the time of the read; later changes arrive asfile.updated(note 10). Editing (M1-58) never changes attachments (03messages). -
Deleting a message (M1-58 steps 3 to 9; 03
messages, 12 12, 20 4.2 and 4.3).a. Step 4 becomes
UPDATE messages SET content_md = '', has_attachments = false, deleted_at = now(), deleted_by = caller, version = version + 1 ...and remembers whether the flag wastrue.b. After revisions and mentions (and reactions and pins of M1-59 where merged) and only when the flag was
true:files.Attachments.DeleteForMessage: lock the files of the message's attachmentsFOR UPDATEin ascendingid, delete themessage_attachmentsrows of the message, setfiles.deleted_atto the clock time. Lock order thread, message, files,message_attachments(Design note 14).c. Ruling for docs/DECISIONS.md: no
file.updatedfor these files (12 2 names state changes, andmessage.deletedalready removes the message with its attachments from every client); objects stay untilfiles.purge_objectsof M3-08 removes them withinprivacy.object_delete_delay;getFilealready answers404for a file withdeleted_at(M3-03). Keep thedeleted_atwrite in one function offiles/appand name it in the README, because M3-08 and M3-15 add their deletions to it. -
Moving a thread (03
threads, 4.7; R-DM-43).files.OnThreadMoved(ctx, chat.ThreadMoved):UPDATE message_attachments SET channel_id = <ToChannelID> WHERE thread_id = <ThreadID>(index(thread_id)); no version change, no event;files.scope_channel_idstays. Registered ininternal/wireaftermessaging.OnThreadMovedandsync.OnThreadMoved, so the move transaction locks thread, messages, the rows ofsync, thenmessage_attachments. A send into the thread holds the thread row (M1-55AppendToThread) before it inserts attachment rows, so a send and a move of the same thread run one after the other and end withmessage_attachments.channel_id=messages.channel_id=threads.channel_id. A write is not linted by the SQL visibility lint (07 7.4 rule 1). -
Reading an attached file (05 2.2; M3-03 Design note 15). One function
fileResource(f File, ref *AttachmentRef) authorize.Resourcebuilds everyfile.readdescriptor infiles: unattached{Kind: file, ID, ChannelID: scope_channel_id, OwnerID: uploaded_by, File: &FileFacts{Purpose, Attached: false}}; attached{Kind: file, ID, ChannelID: ref.ChannelID, ThreadID: ref.ThreadID, OwnerID: uploaded_by, File: &FileFacts{Purpose: "attachment", Attached: true}}.getFileloads the attachment row byfile_idnext to the file row and uses it; where M3-05 (downloadFile,createFileTicket,downloadThumbnail) or M3-06 are merged, switch their call sites tofileResourcetoo. Foreign and unknown IDs keep answering the same404(AZ-P4). -
file.updatedandFileFinalized(12 2, 12 10; 05 6; M3-03 Design note 16; M3-04 Design note 6).a.
recordFileChange(ctx, before, after File)is the one place that appendsfile.updatedfor a change ofupload_state,scan_status, thumbnails or transcript. It reads the attachment row of the file (the caller holds the file row lock): unattached givesAudiencePrincipalID= uploader and scope{space_id, channel_id}ofscope_channel_id; attached gives no audience and scope{space_id, channel_id, thread_id}of the message, withchannel_idandthread_idfrom the copies andspace_idfromchat.Threads.Info(thread_id)(domain.EventTarget). Receive actionfile.read, resource{kind: file, id}withoutversion, payloadFilePayloadwith the form of note 6; the actor stays as each caller sets it today (the request actor for the abort of M3-03, none for the jobs of M3-04 and M3-06).b. In the same function:
txn.Publish(ctx, files.FileFinalized{FileID})whendomain.BecameFinal(before, after)(not final before, final after, 12 2). Ruling for docs/DECISIONS.md: the publish moves out of M3-04's outcome write into this function, so every finalizing write publishes exactly once: M3-04's outcome, M3-03's abort, and M3-06's thumbnail job and its failure transaction where M3-06 is merged (an image is final only after them, M3-06 Design note 16). Route every merged writer of these columns throughrecordFileChange. -
Hold at acceptance (04 15.3 point 1; 09 3.4 point 7; M2-08 Design note 10 f).
invokeOrHoldreplaces the last step of the acceptance, which wroteagent.invocation:a. Before
Open:pending := ev.HasAttachments && files.Attachments.Pending(ctx, ev.MessageID); the attachments were bound earlier in the same unit of work and their file rows are locked by it, so the answer is current.b.
domain.HoldUntil(now, pending, file.invocation_hold_max): zero when not pending, elsenow + file.invocation_hold_max, with onekernel/clockreading taken byinvokeOrHold; pass it asOpenCmd.InvocationHeldUntiland use the same value asScheduledAtof the hold job. Ruling for docs/DECISIONS.md: the release time is fixed at acceptance, so a later change offile.invocation_hold_maxdoes not move held runs.c. Not held: append
agent.invocationas M2-08 does. Held: noagent.invocation; insertInvocationHoldTimeout{RunID}withjobs.Inserter.InsertTxandScheduledAt= the release time. Everything else of the acceptance (runqueued, anchor,SetTriggerState(received),runs.queued_expiry) stays as M2-08 built it.d. Every path that writes the first
agent.invocationof a run with a triggering message goes throughinvokeOrHold: the acceptance ofcreateMessageand ofcreateThreadwithfirst_message, and the follow-on run of M2-11 (M2-11 note 8 c builds it with the invoke step of the acceptance; that path has noMessagePosted, so it asksfiles.Attachments.Pendingfor its triggering message unconditionally, one query per follow-on run).e. Follow-ups are not held (Ruling for docs/DECISIONS.md): a follow-up of 04 15.4 whose attachments are not final gets its
agent.followupas M2-11 writes it. Reason: the core contract holds onlyagent.invocation(04 15.3 point 1, R-SY-20); the adapter reads attachments only throughfile.read, which never serves bytes beforescan_status IN ('clean','skipped')(12 9.3, 09 4.6), so no unchecked byte reaches the agent; holding a follow-up would need a marker onrun_followupsthat 03 does not have and would reorder the follow-ups of a run. Add the sentence to 12 10 (Design note 17). -
ReleaseInvocation(ctx, runID, cause)ininternal/runs/app(04 15.3 point 1 names it; D-K-G6), always inside the ambient unit of work:a. Lock the run
FOR UPDATE. For causeReleaseFinalreadfiles.Attachments.Pending(triggering_message_id)after the lock (a new statement, so it sees every finalization committed before the lock was granted); forReleaseTimeoutthe state of the files does not matter.b.
domain.ReleaseVerdict(state, heldUntil, cause, pending): release only whenstate = 'queued',invocation_held_untilis set and, forReleaseFinal, nothing is pending. A held run that ended meanwhile (runs.queued_expiry, a cancel) keeps its column and is never released (Ruling for docs/DECISIONS.md).c. Release:
UPDATE runs SET invocation_held_until = NULL, version = version + 1 WHERE id = $1 AND state = 'queued' AND invocation_held_until IS NOT NULL; then appendagent.invocationwith the builder of M2-08 Design note 14 and the facts of now: scope of the run's thread now (chat.Threads.Info; the thread may have moved while held), current targetbilling_mode, currentsettings_version,triggered_byfrom themessagingport; audience the agent, receive actionagent.events.consume, resource{kind: run, id}withoutversion. Ruling for docs/DECISIONS.md: the envelope has noactorand nocorrelation_id, because in every path of this ticket the writer is a job without principal (the scan job of M3-04, the thumbnail job of M3-06, the hold job; 04 2.2 rowactor, 03event_log.actor_kind); the trigger stays intriggered_by. A later finalizing write inside a request (the transcript of M3-09) sets the actor of its unit as usual.d. Never
authorizehere:runs.Delivery.Prepareof M2-08 checksagent.invokewith the rights of the moment immediately before the hand-over (04 15.3 point 2; the human without request as principal, R-AZ-29). The release never touchestrigger_state; the first successful delivery moves it toqueued(M2-08).e. Exactly once: both causes serialize on the run row and the column is cleared by the first release; a second attempt finds it
NULL. A causeReleaseTimeoutlogs one Info line withbothy.run.idonly. -
Subscriber and job.
a.
runs.OnFileFinalized(ctx, files.FileFinalized):files.Attachments.MessageOf(e.FileID); not attached: return. OtherwiseSELECT id FROM runs WHERE state = 'queued' AND triggering_message_id = $1 AND invocation_held_until IS NOT NULL ORDER BY id FOR UPDATE(served by the partial index of active runs, note 2 c; the run may live in another thread than the message, 09 3.7) andReleaseInvocation(ctx, id, ReleaseFinal)for each. It runs in the finalizing transaction (02 8.3), so the last attachment's final state, itsfile.updatedand theagent.invocationcommit together.b. Job
runs.invocation_hold_timeout(02 12.3: moduleruns, scheduled, queuedefault), args only the run id (02 12.1). Handler in one unit of work: run notqueuedor columnNULL: returnnil;nowbefore the stored release time (clock skew between nodes): insert itself again at that time and returnnil; elseReleaseInvocation(ctx, id, ReleaseTimeout). Repeated execution has no second effect (02 12.4 rule 1); database errors go to River's retry (02 12.4 rule 5). -
Concurrency and lock order (03 6.3; 02 8.2; deadlocks are retried by the unit of work,
database.tx_retry_max).- Send: thread, message (insert), files ascending,
message_attachments(insert),runs(insert), the message again (SetTriggerState, already locked). - Delete: thread, message, files ascending,
message_attachments(delete). - Move: thread, target channel and space (share, M1-62), messages,
read_states,drafts,message_attachments. - Finalizing write (M3-04, M3-06): file, session, then the run (subscriber of note 13 a).
- Hold job: the run only; it reads files without a lock.
- A file attached and finalized at the same time: the attach locks the file first, so the finalizing write waits and then sees the attachment and the held run; or the finalizing write commits first, so the attach reads a final file and acceptance does not hold. Two attachments finalized in parallel: both subscribers lock the same run; the second reads
Pendingafter the first committed and releases, without waiting for the timeout.
- Send: thread, message (insert), files ascending,
-
Agents and the switch
supports_files_out(09 2.2, 2.3; Ruling for docs/DECISIONS.md). Inruns.OnMessagePosted, before every other branch, forOrigin = agentwithHasAttachments: read the author'soperator_config(OperatorConfigV1of M1-51) through theagentsport;supports_files_out = falsereturns akernel/errserror with codefeature_disabled, which vetoes the unit of work (02 8.3) and which thecreateMessageandcreateThreadhandlers render as403 feature_disabledlike any other error of the use case, so neither message nor attachment rows stay; a missing key allows (09 2.2). The check lives inrunsbecausemessagingandfilesmay not useagents(02 4.4), the same place M2-15 note 6 b chose forsupports_components. Then the merged agent branch (M2-19checkAnswerwhere merged) runs as before. The heartbeat half is not evaluated (Out of scope). Add the sentence to 09 2.2 row "Sending files". -
Configuration and observability. Add to the schema, with chapter defaults (02 13 item 5):
file.invocation_hold_max(chapter 12) andmessage.max_attachments(chapter 03) if absent;TestConfigSchemaBuildslists them; regeneratedocs/operations/configuration.md. No new metric (docs/observability/signals.mdhas none for this path); logs carry onlybothy.run.id,bothy.file.idandbothy.message.id, never file names (12 14). SQL visibility lint (M1-70,tools/lint/sql_visibility.py, AZ-P12): every new reading query on a content table carries the predicate or gets one exception entry intools/lint/sql_visibility.yamlwith its category and a reason:after_authorizefor the file lock read ofAttach(keyed by the listed IDs and the author aftermessage.createwas allowed),ForMessages(messages the caller already read),Pending(the attachments of one message whose creation or run was already decided) and the lock read ofDeleteForMessage(aftermessage.delete);authorize_resolutionfor the attachment read byfile_idthatgetFileuses to build the resource and thatMessageOfandrecordFileChangereuse;system_jobfor the run lock ofReleaseInvocation, the held-run lookup and the read of the hold job. TheUPDATEofOnThreadMovedand the inserts are writes and are not linted (rule 2).make sql-visibilitystays clean. -
Specification and API (rule 10; nothing in 00, 03, 04 or 05 changes; texts in English like the chapters, YAML in English):
docs/spec/12-files.md10: the order and the single answer of note 4 (one409 conflictwithreasonfile_not_attachablefor every cause,detailwith the pointer; blocked and pending files are attachable);limit_reached; empty content with attachments;+uploadand the switchsupports_files_out;scope_channel_idabsent once attached; the release withruns.invocation_held_until(R-DM-44), exactly once, withoutactor, release time fixed at acceptance; follow-ups (agent.followup) are not held, the follow-on run of deferred follow-ups is (note 11 d and e); deletion with the message withoutfile.updated. 12 2 last paragraph: the write that appendsfile.updatedpublishesfiles.FileFinalizedwhen the file becomes final; attached files carry the scope of their message without audience.docs/spec/09-agents.md2.2 row "Sending files": "Ifsupports_files_outis off, the server rejects messages of an agent with attachments with403 feature_disabled."docs/spec/02-architecture.md4.3 and 4.4 as note 1.docs/api/openapi.yaml(descriptions only):MessageCreate.attachment_file_ids(own files uploaded for this channel with a completed session; at mostmessage.max_attachments; needsupload; otherwise409 conflictwithreasonfile_not_attachableor400 validation_failedwitherrors[].codelimit_reached),MessageCreate.content_md(may be empty only with attachments),createMessage(an agent invocation with attachments waits until they are final orfile.invocation_hold_max),File.scope_channel_id(only before the file is attached),Message.attachments(position order). Thenmake codegen-check,make api-lint,make api-checks, andmake api-tablesif a generated block of 06 changes.
-
Rulings for
docs/DECISIONS.md(section 10, one rowD-M3-07-<n>each, with reason and place): the edgeruns -> filesand noAttachmentLocator(note 1); one409for every attach failure with the pointer indetailand the lock before the check (note 4); empty content with attachments (note 5 a);scope_channel_idabsent once attached (note 6); nofile.updatedon deletion with the message and onedeleted_atfunction (note 7 c);FileFinalizedpublished byrecordFileChange(note 10 b); the file lockFOR UPDATE OF fileswithout the session row (note 4 a); release time fixed at acceptance from one clock reading (note 11 b); the follow-on run asksPendingunconditionally (note 11 d); follow-ups are not held (note 11 e); ended runs are never released (note 12 b); noactoron a released invocation andruns.versionplus one at the release (note 12 c); the hold job reschedules itself when early (note 13 b); no new index for the held-run lookup (note 2 c); the switchsupports_files_outin therunssubscriber with403 feature_disabled(note 15); the categories of the SQL lint exceptions (note 16). The columnruns.invocation_held_untilitself is core contract (R-DM-44) and needs no row.
Tests (write them first)¶
Test world: 07 10 and 11.1 (space Support with channels Allgemein and Eskalation, operator Lotse, organization, chat; G-1 Alex trigger with upload on Support; group Leser with Sam, G-4 read on Support/Allgemein; G-5 Lotse write and upload on Support; Robin without a grant on Support; the Owner). Files are uploaded through the real chain with createUpload, uploadPart and completeUpload (M3-03) and scanned by calling the files.scan handler of M3-04 directly; the scanner is the fake clamd server of clamdtest (answers stream: OK, the EICAR reply, or is stopped), except the test marked "real clamd". The adapter side is pollAgentEvents with Lotse's agent token (M2-07); the fake clock drives file.invocation_hold_max (small value in the test configuration).
Unit:
internal/files/domain:TestAttachable(one row per condition of note 4 b: foreign uploader, purposeavatar, other channel (SEC-UP-9), sessionopen,aborted,expired,upload_state = 'failed', deleted, already attached;infected,errorandpendingscan are attachable);TestEventTarget(unattached: audience uploader, scope channel; attached: no audience, scope of the message);TestBecameFinal(every pair of 12 2 states; only not final to final is true; a deleted clean file is not deliverable, M3-04TestDeliverableAndFinalstays).internal/runs/domain:TestHoldUntil(not pending: zero; pending: the given clock reading plusfile.invocation_hold_max);TestReleaseVerdict(statequeued,running,failed,cancelledtimes held or not times causefinalortimeouttimes pending or not; release only for queued, held, and final without pending or timeout).internal/messaging/domain:TestAttachmentBodyRules(overmessage.max_attachmentsgiveslimit_reachedwith pointer/attachment_file_ids, increateThread/first_message/attachment_file_ids; empty content with one attachment passes, without givesempty;kind = voicestillnot_enabled).internal/messaging/appwith a fakefilesport:TestCreate_NoFilesCallWithoutAttachments(noAttach, noForMessagesfor a message without attachments; oneForMessagesper page of messages with the flag).
Integration (PostgreSQL 18 via pgtest, real modules through internal/wire, HTTP through the real chain, M3-01's file system driver in a mktemp -d root):
TestMigrationMessageAttachments: columns, types, PK, both FKs with theirON DELETE, UNIQUE(file_id), index(thread_id), no FK onthread_idorchannel_id;TestMigrationHasAttachments(column, defaultfalse, constraint validated after the second migration);TestMigrationRunsInvocationHeldUntil(nullabletimestamptzcolumn without default,NULLon existing rows, no new index); a secondmigrate upchanges nothing.TestAttach_SendWithAttachments: Alex sends a message with two files in Support/Allgemein:201withattachmentsin request order (position1 and 2), eachFilewithoutscope_channel_idand without JSONnull; two rows withthread_idandchannel_idequal to the message's;has_attachments = true; themessage.createdpayload validates againstMessageCreatedEventand lists both; a message without files hashas_attachments = falseandattachments: []. Variant withclear_draft: trueafter Alex saved a draft of the thread with both file IDs (putThreadDraft): the draft is gone in the same transaction (12 10 point 4, M1-61).TestAttach_SecondAttach409(plan, 12 15 row "Attaching the same file a second time"): F is attached to M1; a new message with F answers409 conflictwithreasonfile_not_attachable; M2 is not stored and the next message of the thread gets the nextthread_seqwithout a gap.TestAttach_NotAttachable_SEC_UP_9: a file of Alex uploaded for Support/Eskalation sent in Allgemein (SEC-UP-9), Lotse's file sent by Alex, an unknown ID, a file with an open session, an aborted upload, an upload failed bychecksum_mismatch, a row withpurpose = 'avatar'(fixture through thepgtestconnection), the same ID twice in one request: each409 conflictwithreasonfile_not_attachable, identical status, code, reason and headers,detailnaming the entry; nothing stored.TestAttach_ConcurrentSameFile: two sends with F at the same time: exactly one201, one409.TestAttach_LimitCapabilityNotEnabled: overmessage.max_attachmentsgives400 validation_failedwitherrors[].codelimit_reached; Robin with awritegrant withoutuploadon Support/Eskalation (granted by the Owner in the test) gets403 forbiddenwithreasoncapability_missing; the M1-56not_enabledanswer forattachment_file_idsis gone.TestAttach_AttachmentOnlyAndPending: a message with only a file and emptycontent_mdis201; a file whose scan has not run yet is attachable and showsscan_status: pending.TestAttach_ReplayAndHash: the same request again answers200withIdempotent-Replay: true, the same attachments and no new row; the sameclient_msg_idwith the files in another order answers422 idempotency_key_reuse.TestAttach_CreateThreadFirstMessage: Alex callscreateThreadin Support/Allgemein withparent_message_id(his message in the main thread) andfirst_message.attachment_file_ids(files uploaded for Allgemein): thread, message and attachment rows are created in one transaction, the rows carry the new thread'sthread_id; overmessage.max_attachmentsand a foreign file answer with the/first_message/attachment_file_idspointer.TestAttachedFile_ReadPath(Alex's message in a thread of Allgemein): Sam (readon Allgemein) gets404for Alex's file before it is attached and200after (getFile, noscope_channel_id); Robin always404, identical to an unknown ID; after the Owner moves the thread to Support/Eskalation Sam gets404and the Owner200(the copies decide). With M3-05 merged,downloadFileandcreateFileTicketfollow the same rule.TestAttachedFile_UpdatedEventScope: F attached while pending; the scan outcome writesfile.updatedwithout audience in the scope{space_id, channel_id, thread_id}of the message; Sam receives it throughlistEvents; a file of Alex that is never attached sends itsfile.updatedto Alex only.TestFileFinalized_OncePerTransition: a test subscriber countsfiles.FileFinalized: once for clean, infected and checksum failure, none for a retry round of M3-04, once for an abort of an unattached upload; with M3-06 merged, once for an image after its thumbnail transaction and never in the scan for that type.TestHasAttachments_Lifecycle(plan):trueafter a send with a file,falsewithout,falseafterdeleteMessage, with themessage_attachmentsrows gone andfiles.deleted_atset in the same transaction (a test subscriber ofMessageDeletedthat vetoes leaves flag, rows and file unchanged);getFilethen answers404for Alex and Sam; a second delete is204without writes.TestHasAttachments_CheckRejectsDeletedWithFlag(pgtest): a directUPDATE messages SET deleted_at = now(), content_md = ''on a message withhas_attachments = truefails the CHECK.TestMoveThread_MovesAttachmentCopies(plan): the Owner moves a thread with attachments from Allgemein to Eskalation: everymessage_attachments.channel_idof the thread equals the target in the same transaction;thread_idunchanged; unattached files keepscope_channel_id; a test subscriber registered afterfiles.OnThreadMovedthat vetoes rolls back the move and leaves the copies.TestMoveThread_ConcurrentSendWithAttachment: many parallel sends with attachments into the thread while it moves: at the endmessage_attachments.channel_id=messages.channel_id=threads.channel_idfor every row.TestHold_ReleasedOnClean(plan, 12 15 row "Agent invocation with an attachment while clamd is down"):auto_threadon Allgemein; the fake clamd is stopped; Alex mentions Lotse with F (pending):201withtrigger_state = received; the run in the new thread isqueuedwithinvocation_held_until= the fake clock at the send plusfile.invocation_hold_max; oneruns.invocation_hold_timeoutjob scheduled at that time; Lotse'spollAgentEventsshows noagent.invocation, andtrigger_statestaysreceivedacross further polls and a clock step below the limit. The fake clamd starts; the duefiles.scanmakes F clean: in that transaction exactly oneagent.invocationis appended, the column isNULLandruns.versionis one higher; the next poll delivers it withinvocation_tokenandtrigger_statebecomesqueued(M2-08); the hold job, run after the limit, writes nothing.TestHold_TimeoutReleasesPending(plan): the fake clamd stays stopped; the clock passesfile.invocation_hold_max; the hold job appends exactly oneagent.invocation(noactor, scope of the run's thread, payload valid againstAgentInvocationPayload); F is stillpending; Lotse'sgetFileof F answers200withscan_status: pending(with M3-05 merged,downloadFileanswers409 file_not_ready; with M2-13 merged,getRunContextlists the attachment withscan_statuspending); when F becomes clean later, no secondagent.invocationappears.TestHold_RealClamdStoppedThenClean(real clamd,clamdtestcontainer): the same sequence asTestHold_ReleasedOnCleanwith the container stopped and started (M3-04TestScanClamdStoppedStaysPendingpattern).TestHold_NoHoldWhenFinal: F is clean before the send:agent.invocationin the message's transaction, columnNULL, no hold job.TestHold_LastOfTwoAttachments: F1 and F2 pending; F1 clean: still held; F2 clean: released once. Variant: both outcome writes in parallel: exactly oneagent.invocation, before the limit.TestHold_BlockedOrFailedAttachmentReleases: an EICAR reply makes Finfected(final): released at once; Lotse'sgetFileshowsinfected.TestHold_SameTransaction: a test subscriber offiles.FileFinalizedregistered afterruns.OnFileFinalizedvetoes once: neither the clean state of F, nor itsfile.updated, noragent.invocationis stored; the retry stores all three.TestHold_RunEndedWhileHeld: withrun.queued_max_agebelowfile.invocation_hold_maxin the test configuration,runs.queued_expiry(M2-08) fails the held run withagent_unavailable; later finalization and the hold job write noagent.invocation.TestHold_ThreadMovedWhileHeld: the thread of a held run moves to Eskalation; the releasedagent.invocationhas the scope of Eskalation.TestAgentAttachments_FilesOutSwitch: Lotse uploads a file in Support and sends it in its thread:201; after the Owner setsoperator_config.supports_files_out = falsewithupdateAgent, the next send with a file gives403 feature_disabledand stores nothing; a send without files still passes.- With M1-57 merged:
TestHistory_ShowsAttachments(getMessageandlistThreadMessagesshow the attachments; a deleted message shows none). For M2-11: a follow-on run whose first deferred message has a pending attachment is held like an acceptance and released once; a follow-up with a pending attachment to a running run gets itsagent.followupat once (not held, note 11 e). internal/wire: the golden registrations showfiles.FileFinalized -> runs.OnFileFinalizedandchat.ThreadMoved -> files.OnThreadMovedafter the subscribers ofmessagingandsync; the jobruns.invocation_hold_timeouton queuedefault;TestConfigSchemaBuildslistsfile.invocation_hold_maxandmessage.max_attachments.
Commands that must pass, from the repository root:
cd services/server && go test -race -count=1 ./internal/files/... ./internal/messaging/... ./internal/runs/... ./internal/chat/... ./internal/agents/... ./internal/authz/... ./internal/wire/... ./migrations/...make migrations-lock(once after creating the migrations),make migrations-lint,make codegen-check,make arch,make register-check,make api-lint,make api-checks,make config-reference(no diff afterwards),make config-doc-check,make sql-visibilitymake diagrams,make docs,make linkcheckmake check
Acceptance criteria¶
-
createMessageandcreateThreadacceptattachment_file_idswith+upload(no longer400 validation_failedwitherrors[].codenot_enabled); a second attach of the same file answers409 conflictwithreasonfile_not_attachable, and so does every file that 12 10 point 1 refuses, including a file of another channel (SEC-UP-9). -
message_attachmentsexists as 03 4.7 says; every row carriesthread_idandchannel_idof its message; aftermoveThreadthe rows of the thread carry the target channel in the same transaction, a rolled back move leaves them, and a concurrent send ends consistent. -
messages.has_attachmentsistrueafter a send with a file,falsewithout, andfalseafter deleting, with the attachment rows gone and the files deleted in the same transaction; the CHECK rejects a deleted message with the flag. - With clamd stopped, a message with an attachment that triggers Lotse keeps
trigger_state = receivedand writes noagent.invocationuntil the attachment isclean(written in that transaction) or untilfile.invocation_hold_max(with the attachment stillpending); each run gets exactly oneagent.invocation, andruns.invocation_held_untilis set exactly while it is held (R-DM-44). - An attached file is read through its message's channel,
file.updatedcarries the message's scope without audience, and every representation of a message lists its attachments. - An agent with
supports_files_out = falsegets403 feature_disabledfor a message with attachments. - 02 4.3 and 4.4,
architecture/modules.yaml, the C4 diagram, 12 2 and 10, 09 2.2, the openapi descriptions,configuration.md, the READMEs, the CHANGELOG and the rowsD-M3-07-<n>are updated;make register-check,make archandmake sql-visibilityare green. -
make checkis green.
Definition of done¶
- Code and tests as specified; doc comment on every public symbol (purpose, parameters, return value, errors, example where useful); comments explain why.
- READMEs of
files(tablemessage_attachmentswith retention classcontent, portAttachments, the subscriber ofchat.ThreadMoved,recordFileChangeand the publish ofFileFinalized, thedeleted_atfunction, query exceptions),messaging(flaghas_attachments, attach in the create path, attachments in the representation, the delete step) andruns(dependency onfiles, the hold,ReleaseInvocation, the subscriber, the job, the switchsupports_files_out) updated;agentsif its port changed. - Rulings of Design note 18 recorded in
docs/DECISIONS.md(one row each with reason). services/server/CHANGELOG.mdentry under 'Unreleased'.- An ADR only if you take an architecture decision beyond this brief (MADR 4 as in
docs/adr/0000-template-madr-4.md, headings and text in English, next free number,make adr-index). make checkgreen.- An independent code review over the diff, then the red team gate (
docs/quality/red-team-gate.md, verdictJAorNEIN); onNEINfix with a test per finding and have the delta checked again. Record both verdicts. - Land the change as
AGENTS.md("How work lands onmain") describes: a pull request from this branch, squash-merged only when review, red team andmake check,ci-okand the review threads are green; thengit worktree remove.
Report¶
Write to {{REPORT_PATH}}:
- First line exactly
Verdict: DONEorVerdict: BLOCKED <reason>. - Commits (SHA and subject), changed files with line counts, test results with counts (the red run before the change and the green run after), the summary of
make check. - Review and red team verdicts and the findings fixed.
- What is not proven.
- Follow-ups that could not be done in this scope, each with its reason, and spec gaps found.
Then write {{DONE_PATH}} with exit=0 sha=<sha> or exit=1 <reason>.