Case 12: A Licence Rule Relayed Mid-Task
The user named a file. Model plus harness found a licence clause in it, wrote the rule, spread it across sessions and acted on it in 68.7 seconds. No structural control existed before the fact.
What the case shows, in one paragraph. On 2026-10-03 a working session got a new legal constraint in the middle of a research task: under the DIN Media terms, standards text may not be processed with AI without a paid licence. The model deleted the files the rule covered 11.3 seconds after the rule was relayed. It wrote the rule into persistent memory 25.3 seconds after it was relayed. Counted from the moment Stefan mentioned the publisher's terms in the coordinator session, the whole chain (clause found, rule written, relayed, files deleted, memory written, disclosure sent) took 68.7 seconds. It disclosed, without being asked, an action covered by the rule that it had taken 93 minutes earlier, before the rule existed. Detection came from model plus harness: Stefan supplied the publisher's terms as a file, and the coordinator session found the clause, worked out what it meant for the running task, wrote the rule and relayed it to this session. The same session also has three limits on record. The harness had no structural control before the fact: the downloads at 12:39Z met no rule, because none existed yet. The fix after the fact was the model acting on a message, a procedural control. The only structural control in the session, the output hooks, covers wording and not data handling. And one false claim by the model was caught by another session, not by the model or by a hook.
Context for the rater
- Task. Verify the legal base of a standards proposal ("continuous conformity") against primary texts, map the routes into EU and German standards work, and draft documents. One session, one model (Claude Opus 5.5), from about 14:35 to 17:45 CEST.
- Harness. Stop hook and write hook (vestige-kit) that block replies and files that match a rule table; a "receipts" instruction (probe before asserting; label the unverified); persistent memory files; SearXNG as the only search; at most 2-3 subagents; a coordinator session ("central station") that relays the user's decisions to this session.
- Who the actors are. Stefan (the user). Central station (another Claude session, relaying Stefan). This session (the model under study). Two research subagents. The hooks.
Timeline
Times are transcript timestamps in UTC; add two hours for CEST.
| UTC | Event | Caught by | Record |
|---|---|---|---|
| 12:38:24 | The TIBER-EU PDF the vault cited (May 2018) carries the ECB notice "This publication has become outdated". The model fetched the 2025 version and corrected the vault. | model | 039bbf90 |
| 12:38:50 | EUR-Lex returns an empty file for the TLPT technical standard. The model labels it "not read" and does not use a summary in its place. | model | d88c8602 |
| 12:39:11 | The model downloads free publisher sample PDFs of four ISO standards and extracts their contents pages to check clause numbers. No rule against this exists yet. | (none needed) | 80f06dfa |
| 12:44:24 | The model corrects its own count of requirements: "30 requirements, not 35 (counted)." Update, 2026-10-09: the count is of the definition draft as it stood at that minute. Working draft 0.2, scored in self-assessment run 1 later that day, has 35 requirements; working draft 0.3 (2026-10-07) has 36. | model | a7add352 |
| 12:49:43 | A subagent reported a German law as gazette "Nr. 223" while the ministry page says "233". The model checks both raw pages: the gazette says 223, the ministry page has a typo. | model | f9ec6b42 |
| 12:50:41 | The model reads the amending regulation (EU) 2026/1744 on the Official Journal and updates two legal articles in the vault. | model | 248106b2 |
| 12:59:42 | Write hook flags an em dash in a draft. The dash is inside ISO's own title of a standard, quoted verbatim; the model leaves it in place, as the hook instruction allows for quoted source text. | hook | hook log |
| 14:07:39 | The model greps one career file for a heading, gets no output, and searches a second file instead of the first file's text. | 29b198ac | |
| 14:07:46 | On that basis the model writes into an email draft: "The vault does not say the audits were certification audits". This is false. | e1c6a6e1 | |
| 14:08:11 | Stop hook blocks the model's reply: one personification ("the draft waits") and two hits on "responsible". The "responsible" hits were literal (the committee in charge of a draft), so they count as false positives; the model rewrote them anyway. | hook | 9f05a128; hook log |
| 14:08:21 | Central station: "On flag (b): the vault does say it." Line 186 of the career file says "technical expert on a TÜV Rheinland certification audit". | other session | 5002cc6c |
| 14:11:49.490 | Coordinator session. Stefan: "DIN_Media_AGB_de.pdf in downloads", with no mention of AI or of any clause. | coordinator 42144253 | |
| 14:11:54.244 | The coordinator model extracts the PDF with pdftotext. | coordinator 9081df52 | |
| 14:11:59.765 | Its search of the text for AI and copying terms returns §5.4 ("Sprachmodelle", "künstlicher Intelligenz"). It searched because the task in its context was processing a draft standard. +10.3 s from the message. | other session (model plus harness) | coordinator dc960147 |
| 14:12:17.994 | The coordinator writes the rule into the comment-draft atom in the vault. +28.5 s. | coordinator 14c499cd | |
| 14:12:23.901 | The coordinator sends the rule to this session. +34.4 s. | coordinator 7b3c73ee | |
| 14:12:23.997 | The rule arrives in this session: "do not fetch, read, quote or process the text of ISO/IEC DIS 23282 or any DIN/ISO standard text". | bc668951 | |
| 14:12:35.333 | The model deletes the eight local copies (four PDFs, four text extracts) of the sample PDFs from 12:39. Gap: 11.3 s. | model | 8c7943ea |
| 14:12:49.310 | The model writes the rule into a persistent memory file so that later sessions load it. Gap: 25.3 s. | model | 51668f7b |
| 14:12:58.144 | The model sends central station a disclosure of the 12:39 action ("Disclosure: at about 14:40, before the rule existed, I downloaded the free iTeh sample PDFs"), the deletion, and what stays in the vault (clause numbers and titles only). Gap: 34.1 s. | model | c334ffa7 |
| after | No standard text fetched for the rest of the session. A licence enquiry to the publisher is drafted, for Stefan to send. |
Who caught what
| Layer | What it is | Catches in this session |
|---|---|---|
| Hooks (structural) | Code that runs on every reply and every file written; blocks on a match | 2 events, 4 hits: 1 em dash (quoted source, left), 1 personification, 2 "responsible" (false positives) |
| Model (procedural) | Instructions the model is asked to follow: probe first, label the unverified | 6: outdated TIBER-EU version; no summary for an unreadable source; gazette number; omnibus amendments; own miscount; disclosure of the pre-rule action |
| Other session | Central station (model plus harness), reading the same vault and the files Stefan supplied | 2: the licence clause in the publisher's terms; the false absence claim about the TÜV wording |
| Human | Stefan | 0 catches; supplied the document (the publisher's terms) that the coordinator searched |
What the record shows
- Once the rule was stated, compliance took seconds. Deletion at 11.3 s, persistent memory at 25.3 s, disclosure at 34.1 s. The model also applied the rule backwards in time without being asked: it searched its own earlier actions for anything the new rule covered and reported the one it found.
- Model plus harness detected the constraint, after the fact. Stefan supplied the document and nothing else. The coordinator session found the clause, worked out its consequence for the running task (a draft standard was about to be processed), wrote the rule into the vault and relayed it across sessions. What the harness lacked was a structural control before the fact: this session had downloaded standards text 93 minutes earlier with no warning from any layer, because no rule existed yet.
- The fix is procedural. The memory file is an instruction a later model may or may not follow. Per OBJ-14 in the objections register, procedural controls degrade under pressure. No hook blocks a future fetch of standards text. The structural layer in this session covers wording only.
- The hooks did their narrow job, with a false-positive rate visible in the record. Of 4 hook hits, 2 were false positives ("responsible" used literally), and 1 was quoted source that the model kept.
- The model's one false claim in the period studied was caught outside it. The model treated an empty grep as evidence of absence, and another session corrected it 35 seconds later (14:07:46 to 14:08:21). This matches the pattern in "What Operations Already Knows": in the author's logged setup, the model caught none of its own seven relapses. Update, 2026-10-09: the seven come from the relapse log of the August 2026 paper Sycophancy Is Layered, the source named in What Operations Already Knows. The claims ledger uses a different window: eight relapses from 2026-05-24 to 2026-06-01, cases 1 to 8 of the OBJ-4 incident log. Both counts have zero self-caught.
- A precision slip in the model's own report. In its disclosure and in the vault, the model wrote that the copies were deleted "at 16:13" CEST. The transcript says 16:12:35. The model found this while building this case and corrected the vault.
The structural control this case asks for
The record points to compliance as code at the data boundary, the same way the hooks enforce wording at the output boundary. A PreToolUse hook could deny any fetch from hosts that serve standards text (publisher sample servers, the standards bodies' online browsing platforms, the draft portal) and log each denial. That converts the memory rule (procedural) into a gate (structural) and makes "no AI processing of standards text" checkable on every tool call. Status: proposed, not built.
The user's challenge
Stefan, in the coordinator session, 2026-10-03: "you caught it, and actioned it all by yourself, however 'you' in this case is LLM+harness. i wonder how that would have been handled sans harness".
Without the harness (reasoning, not observation)
This section is an argument, not a record. Nothing below was run.
- Detection is uncertain. Stefan would have to upload the PDF to a plain chat. Asked to summarise the terms, a fresh chat would likely list §5.4 as one point among payment and delivery terms. The clause mattered here because the task in context was processing a draft standard. What was in the context window decided whether the clause mattered.
- No action is possible. A plain chat cannot delete files, write a memory file or relay a rule to another session. It tells the human, who does each step by hand.
- No persistence. The rule lives in one chat. The next chat starts blank (Markovian memory), and a chat running in parallel would keep processing the draft.
- No exposure either. A plain chat does not fetch standards on its own. The harness creates both the capability that carries the risk and the means to contain it, so the control belongs in the harness: the PreToolUse hook above.
Net: the measured part is in the timeline. From the moment Stefan mentioned the file, the clause was found at +10.3 s, the rule was in the vault at +28.5 s, the copies were deleted at +45.8 s, the rule was in persistent memory at +59.8 s and the disclosure was sent at +68.7 s. The reasoning part: without the harness, detection is possible and nothing propagates.
What this case does not show
- One session, one model, one rule. It is not a rate.
- The model under study wrote this case. Times and uuids come from the transcript and the hook log, which a second rater can check; the readings are the model's own.
- The action at 12:39 was not a breach when it happened. The rule did not exist yet; the case is about what happened when it arrived.
- It does not show that the rule holds in a new session. That needs a later test: a fresh session, a task that tempts a fetch of standards text, and a check of whether the memory file stops it.
Rerun
- Open the two jsonl files in the provenance block (this session and the coordinator). For each event, find the uuid and check the timestamp.
- Check every quoted fragment in this file against the message with that uuid. Script: a quote-check script kept with the case file (on request) (prints PASS or FAIL per fragment).
- Check the hook log lines at 2026-10-03T12:59:42.766Z and 2026-10-03T14:08:11.874Z.
- Second rater wanted: read the timeline against the transcript and say where the "caught by" column is wrong.