From 39fdac34765ba5fb79b3acab84f505def1b8038e Mon Sep 17 00:00:00 2001 From: agentloop Date: Mon, 8 Jun 2026 06:19:17 +0000 Subject: [PATCH] plan: refresh plan for issue #539 --- ISSUE-539.md | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 ISSUE-539.md diff --git a/ISSUE-539.md b/ISSUE-539.md new file mode 100644 index 0000000..a0b2bfa --- /dev/null +++ b/ISSUE-539.md @@ -0,0 +1,72 @@ +## Background + +The issue is real but the proposed mechanism (predicted-mail table + Message-ID match at next sync) is more complex than needed. Two IMAP facilities cover this directly: + +- **RFC 4315 / 6851 — `UIDPLUS` + `MOVE` response code `COPYUID`**: the server returns the destination UID(s) inline with the `MOVE` (or `COPY`+`EXPUNGE`) response. `enough_mail`'s `GenericImapResult.responseCodeCopyUid` already parses this (`/data/home/.pub-cache/hosted/pub.dev/enough_mail-2.1.7/lib/src/imap/response.dart:53`). Stalwart and every modern IMAP server advertise UIDPLUS. We currently throw this away — `client.uidMove(...)` is called for its side effect and the result is ignored (`lib/data/repositories/email_repository_impl.dart:2320,2335,2354`). +- **Fallback — `UID SEARCH HEADER Message-ID …`**: bounded, deterministic, works on every IMAP server. Needed only when `responseCodeCopyUid` is missing. + +The undo path already attempts a Message-ID recovery (`lib/core/services/undo_service.dart:78-84`), and `findEmailByMessageId` exists (`lib/data/repositories/email_repository_impl.dart:1899`), so the building blocks are in place; we just need to remap on the write path instead of guessing on the read path. + +There is also a latent race: the row is optimistically moved (`mailbox_path = dest`) before the server flush, but the row still carries the source UID. If `_reconcileDeletedImap` runs against the **destination** mailbox between the optimistic write and the flush, it will not find that UID on the server and will hard-delete the row (`lib/data/repositories/email_repository_impl.dart:758-761`). + +## Goal + +After an IMAP move, the local email row keeps tracking the same physical message under its new UID, without orphaning, without re-fetching, and without breaking UndoLog references. + +## Concrete changes + +### 1. Capture the new UID inside the flush +`lib/data/repositories/email_repository_impl.dart` — `_applyPendingChangeImap`, cases `move`, `snooze`, `unsnooze`: + +- Capture the `GenericImapResult` returned by `client.uidMove(...)`. +- Read `result.responseCodeCopyUid`. If non-null, zip `sourceSequence` ↔ `targetSequence` to map `oldUid → newUid`. +- If null (no UIDPLUS), look up the row's `messageId`. If present: `client.selectMailboxByPath(dest)` then `client.uidSearchMessages(searchCriteria: 'HEADER Message-ID ""')` and take the highest UID. If `messageId` is also missing, log a warning and leave the row to be reconciled out on the next pass (worst case: cache miss, body refetch). +- For each old/new pair, call a new helper `_remapEmailAfterMove(row, newUid, destPath)`. + +### 2. New helper `_remapEmailAfterMove` (same file) + +Composes `newId = "accountId:destPath:newUid"`, then in one Drift transaction: +- `UPDATE email_bodies SET email_id = newId WHERE email_id = oldId` +- `UPDATE emails SET id = newId, uid = newUid, mailbox_path = destPath WHERE id = oldId` +- Patch any `threads.email_ids_json` containing `oldId` (reuse the JSON helper from the v41 migration path, or just rebuild affected threads via `_updateThread`). +- `UPDATE pending_changes SET resource_id = newId WHERE resource_id = oldId` (in case multiple mutations are queued). +- Call `UndoRepository.remapEmailId(oldId, newId)` (new method, see §3). + +`PRAGMA defer_foreign_keys = ON` for the transaction, mirroring the v41 migration pattern. + +### 3. UndoLog remap + +- `lib/data/repositories/undo_repository_impl.dart`: add `Future remapEmailId(String oldId, String newId)` that loads each `undo_actions` row, rewrites `emailIds[]` entries equal to `oldId` and `originalEmails[].id` entries equal to `oldId`, and re-saves. +- `lib/core/repositories/undo_repository.dart`: add the interface method. +- Inject `UndoRepository` into `EmailRepositoryImpl` via DI (already a singleton in `lib/di.dart`) so `_remapEmailAfterMove` can call it. +- The fallback path in `UndoService.undo` (the Message-ID lookup) becomes a defensive backstop and can stay as-is. + +### 4. Reconciliation guard + +`_reconcileDeletedImap` (`lib/data/repositories/email_repository_impl.dart:731`): before deleting a row whose UID is absent from the server, query `pending_changes` for a `move`/`snooze`/`unsnooze` row with `resource_id = row.id`. If one exists, skip the delete — the row is mid-flight. Since flush always runs before sync (`SYNC.md:199-200`) the race is narrow but real on retries when flush failed mid-cycle. + +### 5. Tests + +- `test/unit/fake_imap.dart`: make `uidMove` return a `GenericImapResult` whose `responseCode` is a configurable `"COPYUID "` string; add a UIDPLUS-off mode that returns no response code and serves `uidSearchMessages` with a HEADER predicate. +- New `test/unit/repositories/email_repository_move_test.dart`: + - Enqueue a move → flush → assert `emails`, `email_bodies`, `threads`, `pending_changes`, `undo_actions` all carry the new id. + - Same with UIDPLUS-off + HEADER-search fallback. + - Move with no Message-ID and no COPYUID → row is unchanged, warning logged, next reconciliation does not delete it because pending change is still queued. +- New test in the same file: `_reconcileDeletedImap` skips a row that has a queued `move`. +- Extend `test/integration/concurrent_sync_test.dart` (or sibling file) with a real Stalwart move-then-sync-then-undo cycle: assert the row keeps its body, the new UID matches what `UID SEARCH` returns, and undo restores it to the source mailbox. + +### 6. Docs + +- `SYNC.md` §3: note that IMAP moves remap the local id from the `COPYUID` response (UIDPLUS) or `UID SEARCH HEADER Message-ID` fallback, so `email_bodies` and `undo_actions` stay valid across moves. +- `DB-SYNC.md` IMAP bullets: add "IMAP move remap: local row keeps its body/undo references via `COPYUID` (RFC 4315) or Message-ID header search." + +## Non-goals + +- No new "predicted mail" table — `COPYUID` + header search is reliable and stateless. +- No change to JMAP moves (JMAP IDs are mailbox-independent). +- No change to `uidValidity` handling — orthogonal. +- No CONDSTORE/QRESYNC — already listed as future work in `DB-SYNC.md`. + +## Open question for review + +Should the fallback header-search be best-effort-quiet (current plan) or should it surface a `FailedMutation` when both `COPYUID` and `Message-ID` are missing? I lean quiet because the next full sync will still pull the message under its new UID — we'd just lose the cached body and any queued mutations on top of it. Happy to escalate to `FailedMutation` if you'd rather make the data loss visible. -- 2.52.0