plan: refresh plan for issue #539

.pre-commit-config.yaml

@@ -53,9 +53,3 @@ repos:
         entry: bash -c 'cd "$(git rev-parse --show-toplevel)" && nix develop --command task check-ci-images'
         pass_filenames: false
         files: ^(ci/main\.go|\.fvmrc)$
-      - id: dagger-versions-aligned
-        name: verify Dagger version is consistent across dagger.json, flake.nix, Dockerfile and DAGGER.md
-        language: system
-        entry: bash -c 'cd "$(git rev-parse --show-toplevel)" && scripts/check_dagger_versions.sh'
-        pass_filenames: false
-        files: ^(ci/dagger\.json|flake\.nix|\.forgejo/Dockerfile|DAGGER\.md)$

ISSUE-484.md

@@ -1,100 +0,0 @@
-## Root cause analysis
-
-The "Load remote images" button is rendered in two places: `lib/ui/screens/email_detail_screen.dart:228-262` (single mail view) and `lib/ui/screens/thread_detail_screen.dart:203-237` (thread view). Both call the same pattern:
-
-```dart
-onPressed: () {
-  setState(() => _loadRemoteImages = true);   // 1. schedule rebuild
-  if (senderEmail != null) {
-    unawaited(...addTrustedImageSender(senderEmail));  // 2. fire-and-forget DB write
-    ScaffoldMessenger.of(ctx).showSnackBar(SnackBar(   // 3. queue snack bar
-      duration: const Duration(seconds: 3),
-      ...
-    ));
-  }
-}
-```
-
-Although `duration: 3s` is already set, the snack bar fails to auto-dismiss. This mirrors the bug fixed in PR #401 (issue #399): there, a snack bar fired during a navigation transition and the duration timer "didn't start correctly" because the snack bar was queued on an unstable scaffold.
-
-Here, the analogous instability comes from three rebuilds that all land between `showSnackBar` and the moment the SnackBar's enter-animation would normally complete and start its dismiss timer:
-
-1. The synchronous `setState` flips `_loadRemoteImages` → `true`, which immediately removes the "Load remote images" button (the very widget whose `onPressed` was running) and swaps the `SecureEmailWebView` into the rebuilt subtree with `loadRemoteImages: true`. The WebView's `didUpdateWidget` then triggers an async `loadHtmlString` reload (see `lib/ui/widgets/secure_email_webview.dart:100-106`), which subsequently calls `setState(() => _height = h)` inside `_measureHeight`.
-2. The fire-and-forget `addTrustedImageSender` write resolves a moment later, the `trustedImageSendersProvider` stream emits, and `ref.watch(trustedImageSendersProvider)` in `email_detail_screen.dart:197` causes another rebuild of the whole screen body — including the `Scaffold`'s body subtree that hosts the snack bar overlay's host context.
-3. These rebuilds happen during the SnackBar's enter animation, so the `_SnackBarState` ends up holding stale animation state and the per-snack-bar timer that schedules `hideCurrentSnackBar` after `duration` never fires.
-
-## Plan
-
-### Fix
-
-Queue the snack bar **before** mutating state, so it reaches `ScaffoldMessenger` while the Scaffold subtree is still stable, and defer the state change to a post-frame callback so the snack bar's enter-animation can finish before the WebView reload and the provider-driven rebuild run.
-
-In `lib/ui/screens/email_detail_screen.dart`, replace the body of `OutlinedButton.icon.onPressed` at lines 231-261 with:
-
-```dart
-onPressed: () {
-  if (senderEmail != null) {
-    unawaited(
-      ref
-          .read(userPreferencesRepositoryProvider)
-          .addTrustedImageSender(senderEmail),
-    );
-    ScaffoldMessenger.of(ctx).showSnackBar(
-      SnackBar(
-        duration: const Duration(seconds: 3),
-        content: const Text(
-          'Images will be loaded automatically for this sender.',
-        ),
-        action: SnackBarAction(
-          label: 'View',
-          onPressed: () {
-            if (mounted) {
-              unawaited(
-                context.push(
-                  '/accounts/trusted-senders',
-                  extra: senderEmail,
-                ),
-              );
-            }
-          },
-        ),
-      ),
-    );
-  }
-  WidgetsBinding.instance.addPostFrameCallback((_) {
-    if (mounted) setState(() => _loadRemoteImages = true);
-  });
-},
-```
-
-Apply the same reordering to `lib/ui/screens/thread_detail_screen.dart:206-236`.
-
-The key changes:
-- `showSnackBar` runs first, on the still-stable scaffold subtree.
-- `setState` (which triggers WebView swap-in and subsequent rebuilds) is deferred to a post-frame callback.
-- When `senderEmail == null` (no trusted-sender to register, so no snack bar), the post-frame callback still flips `_loadRemoteImages` to true — preserving existing behavior of the button working even for unknown senders.
-
-### Tests
-
-Add a widget test in `test/widget/email_detail_screen_test.dart` that:
-
-1. Pumps an `EmailDetailScreen` with an HTML body and a non-empty `From` header.
-2. Taps the "Load remote images" button.
-3. Verifies the snack bar with text "Images will be loaded automatically for this sender." appears.
-4. Calls `tester.pump(const Duration(seconds: 4))` (or uses `tester.pumpAndSettle` after a 3.5s pump).
-5. Verifies the snack bar is gone (`expect(find.byType(SnackBar), findsNothing)`).
-6. Verifies `_loadRemoteImages` did flip, by checking that the "Load remote images" button is no longer present.
-
-Add an analogous test in `test/widget/thread_detail_screen_test.dart` (or wherever thread tests live; create the file if it does not exist yet — use the email_detail test as a template).
-
-### Out of scope
-
-- The "First update agent loop, fix search bug" line in the issue body is two unrelated todo notes the reporter jotted down (the search bug is tracked separately). This plan does not address them.
-- Other `showSnackBar` call sites in `email_detail_screen.dart` (download success/failure, copy-to-clipboard, raw-email errors, etc.) are not affected by the same rebuild pattern and stay unchanged.
-
-### Verification checklist
-
-- [ ] `dart test` (or the project's `task test` equivalent) passes, including the two new widget tests.
-- [ ] Manual: open a single mail in `EmailDetailScreen` with HTML body from a sender not yet trusted; tap "Load remote images"; verify snack bar appears, images load, and snack bar disappears after ~3 seconds.
-- [ ] Manual: tap "View" on the snack bar before it dismisses; verify it navigates to `/accounts/trusted-senders` and that the snack bar is dismissed by the navigation as expected.
-- [ ] Manual: repeat in `ThreadDetailScreen`.

ISSUE-539.md

@@ -0,0 +1,72 @@
+## Background
+
+The issue is real but the proposed mechanism (predicted-mail table + Message-ID match at next sync) is more complex than needed. Two IMAP facilities cover this directly:
+
+- **RFC 4315 / 6851 — `UIDPLUS` + `MOVE` response code `COPYUID`**: the server returns the destination UID(s) inline with the `MOVE` (or `COPY`+`EXPUNGE`) response. `enough_mail`'s `GenericImapResult.responseCodeCopyUid` already parses this (`/data/home/.pub-cache/hosted/pub.dev/enough_mail-2.1.7/lib/src/imap/response.dart:53`). Stalwart and every modern IMAP server advertise UIDPLUS. We currently throw this away — `client.uidMove(...)` is called for its side effect and the result is ignored (`lib/data/repositories/email_repository_impl.dart:2320,2335,2354`).
+- **Fallback — `UID SEARCH HEADER Message-ID …`**: bounded, deterministic, works on every IMAP server. Needed only when `responseCodeCopyUid` is missing.
+
+The undo path already attempts a Message-ID recovery (`lib/core/services/undo_service.dart:78-84`), and `findEmailByMessageId` exists (`lib/data/repositories/email_repository_impl.dart:1899`), so the building blocks are in place; we just need to remap on the write path instead of guessing on the read path.
+
+There is also a latent race: the row is optimistically moved (`mailbox_path = dest`) before the server flush, but the row still carries the source UID. If `_reconcileDeletedImap` runs against the **destination** mailbox between the optimistic write and the flush, it will not find that UID on the server and will hard-delete the row (`lib/data/repositories/email_repository_impl.dart:758-761`).
+
+## Goal
+
+After an IMAP move, the local email row keeps tracking the same physical message under its new UID, without orphaning, without re-fetching, and without breaking UndoLog references.
+
+## Concrete changes
+
+### 1. Capture the new UID inside the flush
+`lib/data/repositories/email_repository_impl.dart` — `_applyPendingChangeImap`, cases `move`, `snooze`, `unsnooze`:
+
+- Capture the `GenericImapResult` returned by `client.uidMove(...)`.
+- Read `result.responseCodeCopyUid`. If non-null, zip `sourceSequence` ↔ `targetSequence` to map `oldUid → newUid`.
+- If null (no UIDPLUS), look up the row's `messageId`. If present: `client.selectMailboxByPath(dest)` then `client.uidSearchMessages(searchCriteria: 'HEADER Message-ID "<id>"')` and take the highest UID. If `messageId` is also missing, log a warning and leave the row to be reconciled out on the next pass (worst case: cache miss, body refetch).
+- For each old/new pair, call a new helper `_remapEmailAfterMove(row, newUid, destPath)`.
+
+### 2. New helper `_remapEmailAfterMove` (same file)
+
+Composes `newId = "accountId:destPath:newUid"`, then in one Drift transaction:
+- `UPDATE email_bodies SET email_id = newId WHERE email_id = oldId`
+- `UPDATE emails SET id = newId, uid = newUid, mailbox_path = destPath WHERE id = oldId`
+- Patch any `threads.email_ids_json` containing `oldId` (reuse the JSON helper from the v41 migration path, or just rebuild affected threads via `_updateThread`).
+- `UPDATE pending_changes SET resource_id = newId WHERE resource_id = oldId` (in case multiple mutations are queued).
+- Call `UndoRepository.remapEmailId(oldId, newId)` (new method, see §3).
+
+`PRAGMA defer_foreign_keys = ON` for the transaction, mirroring the v41 migration pattern.
+
+### 3. UndoLog remap
+
+- `lib/data/repositories/undo_repository_impl.dart`: add `Future<void> remapEmailId(String oldId, String newId)` that loads each `undo_actions` row, rewrites `emailIds[]` entries equal to `oldId` and `originalEmails[].id` entries equal to `oldId`, and re-saves.
+- `lib/core/repositories/undo_repository.dart`: add the interface method.
+- Inject `UndoRepository` into `EmailRepositoryImpl` via DI (already a singleton in `lib/di.dart`) so `_remapEmailAfterMove` can call it.
+- The fallback path in `UndoService.undo` (the Message-ID lookup) becomes a defensive backstop and can stay as-is.
+
+### 4. Reconciliation guard
+
+`_reconcileDeletedImap` (`lib/data/repositories/email_repository_impl.dart:731`): before deleting a row whose UID is absent from the server, query `pending_changes` for a `move`/`snooze`/`unsnooze` row with `resource_id = row.id`. If one exists, skip the delete — the row is mid-flight. Since flush always runs before sync (`SYNC.md:199-200`) the race is narrow but real on retries when flush failed mid-cycle.
+
+### 5. Tests
+
+- `test/unit/fake_imap.dart`: make `uidMove` return a `GenericImapResult` whose `responseCode` is a configurable `"COPYUID <validity> <src> <dest>"` string; add a UIDPLUS-off mode that returns no response code and serves `uidSearchMessages` with a HEADER predicate.
+- New `test/unit/repositories/email_repository_move_test.dart`:
+  - Enqueue a move → flush → assert `emails`, `email_bodies`, `threads`, `pending_changes`, `undo_actions` all carry the new id.
+  - Same with UIDPLUS-off + HEADER-search fallback.
+  - Move with no Message-ID and no COPYUID → row is unchanged, warning logged, next reconciliation does not delete it because pending change is still queued.
+- New test in the same file: `_reconcileDeletedImap` skips a row that has a queued `move`.
+- Extend `test/integration/concurrent_sync_test.dart` (or sibling file) with a real Stalwart move-then-sync-then-undo cycle: assert the row keeps its body, the new UID matches what `UID SEARCH` returns, and undo restores it to the source mailbox.
+
+### 6. Docs
+
+- `SYNC.md` §3: note that IMAP moves remap the local id from the `COPYUID` response (UIDPLUS) or `UID SEARCH HEADER Message-ID` fallback, so `email_bodies` and `undo_actions` stay valid across moves.
+- `DB-SYNC.md` IMAP bullets: add "IMAP move remap: local row keeps its body/undo references via `COPYUID` (RFC 4315) or Message-ID header search."
+
+## Non-goals
+
+- No new "predicted mail" table — `COPYUID` + header search is reliable and stateless.
+- No change to JMAP moves (JMAP IDs are mailbox-independent).
+- No change to `uidValidity` handling — orthogonal.
+- No CONDSTORE/QRESYNC — already listed as future work in `DB-SYNC.md`.
+
+## Open question for review
+
+Should the fallback header-search be best-effort-quiet (current plan) or should it surface a `FailedMutation` when both `COPYUID` and `Message-ID` are missing? I lean quiet because the next full sync will still pull the message under its new UID — we'd just lose the cached body and any queued mutations on top of it. Happy to escalate to `FailedMutation` if you'd rather make the data loss visible.

Taskfile.yml

@@ -712,11 +712,6 @@ tasks:
     cmds:
       - scripts/check_ci_images.sh
 
-  check-dagger-versions:
-    desc: Verify ci/dagger.json, flake.nix, .forgejo/Dockerfile and DAGGER.md pin the same Dagger version
-    cmds:
-      - scripts/check_dagger_versions.sh
-
   _integrations:
     internal: true
     run: once

flake.nix

@@ -49,16 +49,14 @@
           '';
         };
 
-        # The dagger/nix flake's Nix wrapper is a broken self-exec loop, so we
-        # fetch the CLI binary directly.  Keep this version in lockstep with
-        # ci/dagger.json (engineVersion) and .forgejo/Dockerfile (DAGGER_VERSION) —
-        # scripts/check_dagger_versions.sh enforces this.
-        daggerCli = pkgs.stdenv.mkDerivation {
+        # The dagger/nix flake pins 0.20.8, whose Nix wrapper is a broken self-exec
+        # loop.  Fetch 0.21.4 directly so the pre-commit dart-check hook can run.
+        dagger021 = pkgs.stdenv.mkDerivation {
           pname = "dagger";
-          version = "0.20.8";
+          version = "0.21.4";
           src = pkgs.fetchurl {
-            url = "https://dl.dagger.io/dagger/releases/0.20.8/dagger_v0.20.8_linux_amd64.tar.gz";
-            sha256 = "1ns6wq2z1skd2fq9lbrcali0s8kn24p3haamnjjgchg6zlv6b960";
+            url = "https://dl.dagger.io/dagger/releases/0.21.4/dagger_v0.21.4_linux_amd64.tar.gz";
+            sha256 = "0wlnbr4g5069755131yjp2a6alacn64f1c8b27xn0cbynq3zicjd";
           };
           sourceRoot = ".";
           installPhase = ''
@@ -71,7 +69,7 @@
         devShells.default = pkgs.mkShell {
           buildInputs = with pkgs; [
             # Dagger CLI
-            daggerCli
+            dagger021
 
             # Go compiler — for Dagger development
             go

scripts/check_dagger_versions.sh

@@ -1,49 +0,0 @@
-#!/usr/bin/env bash
-# Verify that the Dagger version is consistent across the project.
-#
-# The Dagger CLI must speak the same protocol as the engine it talks to.  We
-# pin the version in four places (engine image in DAGGER.md, the CLI in
-# flake.nix, the CLI in the Forgejo runner Dockerfile, and the module
-# engineVersion in ci/dagger.json).  This script fails if any of them drift.
-set -euo pipefail
-
-ROOT=$(git rev-parse --show-toplevel)
-
-# ci/dagger.json — strip leading "v" for comparison.
-dagger_json=$(grep -oE '"engineVersion"[[:space:]]*:[[:space:]]*"[^"]+"' "$ROOT/ci/dagger.json" \
-  | sed -E 's/.*"v?([^"]+)"$/\1/')
-
-# flake.nix — the dagger021 derivation's CLI download URL.
-flake_nix=$(grep -oE 'dagger_v[0-9]+\.[0-9]+\.[0-9]+_linux' "$ROOT/flake.nix" \
-  | head -n1 \
-  | sed -E 's/dagger_v([0-9.]+)_linux/\1/')
-
-# .forgejo/Dockerfile — DAGGER_VERSION env on the install line.
-dockerfile=$(grep -oE 'DAGGER_VERSION=[0-9]+\.[0-9]+\.[0-9]+' "$ROOT/.forgejo/Dockerfile" \
-  | head -n1 \
-  | cut -d= -f2)
-
-# DAGGER.md — engine image tag in the example systemd unit.
-dagger_md=$(grep -oE 'dagger/nix/v[0-9]+\.[0-9]+\.[0-9]+' "$ROOT/DAGGER.md" \
-  | head -n1 \
-  | sed -E 's@.*/v@@')
-
-printf 'ci/dagger.json    engineVersion = v%s\n' "$dagger_json"
-printf 'flake.nix         dagger021     = %s\n'  "$flake_nix"
-printf '.forgejo/Dockerf. DAGGER_VERSION= %s\n'  "$dockerfile"
-printf 'DAGGER.md         engine tag    = v%s\n' "$dagger_md"
-
-for v in "$flake_nix" "$dockerfile" "$dagger_md"; do
-  if [ -z "$v" ]; then
-    echo "ERROR: failed to parse a Dagger version reference." >&2
-    exit 1
-  fi
-  if [ "$v" != "$dagger_json" ]; then
-    echo "" >&2
-    echo "ERROR: Dagger versions are out of sync." >&2
-    echo "  Align ci/dagger.json, flake.nix, .forgejo/Dockerfile and DAGGER.md to the same version." >&2
-    exit 1
-  fi
-done
-
-echo "Dagger versions aligned (v$dagger_json)."