guettli/sharedinbox
1
0
Fork 0
Code Issues 21 Pull Requests 6 Actions Packages Projects Releases Wiki Activity

Pre fetch email bodies #373

New Issue
Closed
opened 2026-06-03 19:27:08 +00:00 by guettli · 2 comments
guettli commented 2026-06-03 19:27:08 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Email bodies should be available offline.

But the server could contain huge mailboxes with gigabyte of mails.

Create a plan how to keep mails in local cache without downloading all.

Aspects: large attachments don't need to be in local cache.

Check network: on mobile network download less.

User should be able to configure the local cache size.

Email bodies should be available offline. But the server could contain huge mailboxes with gigabyte of mails. Create a plan how to keep mails in local cache without downloading all. Aspects: large attachments don't need to be in local cache. Check network: on mobile network download less. User should be able to configure the local cache size.
guettlibot commented 2026-06-03 21:35:02 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Here is the implementation plan to post as a comment on issue #373:

Implementation Plan: Pre-fetch Email Bodies

Overview

Email bodies are currently fetched on-demand with a 7-day TTL. This plan adds a background pre-fetch mechanism that respects network type, skips large messages, and enforces a user-configured cache size limit. The existing EmailBodies.cachedAt column and background sync infrastructure in background_sync.dart are the natural hooks.

Step 1 — Store message size during sync (schema v37)

Add sizeBytes IntColumn (nullable) to the Emails table in lib/data/db/database.dart. Populate it from RFC822.SIZE during the existing IMAP sync so we have a size signal without an extra round-trip at prefetch time.

Run dart run build_runner build --delete-conflicting-outputs after the schema change.

Step 2 — Add prefetch settings to UserPreferences

Extend lib/core/models/user_preferences.dart with two new fields:

  • prefetchMode (enum: never / wifiOnly / always, default wifiOnly)
  • bodyCacheLimitMb (int, default 100)

Persist them in the existing UserPreferences repository. No new DB table needed if prefs use a key-value store; otherwise add columns to whatever table backs them.

Step 3 — Add connectivity_plus dependency

Add connectivity_plus to pubspec.yaml. Create lib/core/services/network_info.dart with a single function:

Future<bool> isOnWifi(); // true when ConnectivityResult.wifi

Used by the prefetch service and background sync to gate downloads.

Step 4 — Body prefetch service

New file: lib/core/services/body_prefetch_service.dart

Algorithm:

  1. Load UserPreferences. If prefetchMode == never, return immediately.
  2. If prefetchMode == wifiOnly and not on Wi-Fi, return immediately.
  3. Compute current cache size: SELECT SUM(length(textBody) + length(htmlBody)) FROM email_bodies. If already ≥ bodyCacheLimitMb × 1024 × 1024, evict before prefetching (Step 5).
  4. Query candidate emails: Emails rows that have no corresponding EmailBodies row (or cachedAt is null/expired), ordered by receivedAt DESC, where sizeBytes IS NULL OR sizeBytes < 512000 (skip messages over ~500 KB to avoid large attachments dragging down performance).
  5. Process up to a configurable batch (e.g., 20 emails per run). For each, call the existing getEmailBody() logic — no new IMAP code needed, just reuse the existing on-demand fetch path.
  6. Stop early if the cache size limit is reached mid-batch.

The 512 KB per-message threshold should be exposed as a constant (not user-configurable) for now.

Step 5 — Cache eviction

When cache size exceeds the limit, delete EmailBodies rows oldest-first (ORDER BY cachedAt ASC) until the size is back under the limit. Implement as a private helper in the prefetch service, called at the start of each prefetch run.

Step 6 — Integrate with background sync

At the end of _doBackgroundSync() in lib/core/sync/background_sync.dart, instantiate and call BodyPrefetchService.run(). The existing Constraints(networkType: NetworkType.connected) on the WorkManager task already gates this on connectivity; the prefetch service adds the WiFi-vs-mobile check on top.

Step 7 — Settings UI

Add two controls to lib/ui/screens/user_preferences_screen.dart:

  • Prefetch mode: RadioGroup (Never / Wi-Fi only / Wi-Fi + mobile) — consistent with the existing radio groups for menu position.
  • Cache limit: A DropdownButton or segmented control with options 50 / 100 / 200 / 500 MB, plus a read-only display of current cache size (computed from the DB query in Step 4).

What is NOT downloaded during prefetch

  • Binary attachments are never downloaded. The fetch call (BODY.PEEK[]) retrieves the full MIME message, but only textBody and htmlBody are stored; binary parts are discarded. The sizeBytes guard in Step 4 additionally skips entire messages that are too large to be worth fetching.
  • On-demand attachment viewing remains unchanged.

File summary

File Change
lib/data/db/database.dart Add sizeBytes to Emails (schema v37)
lib/core/db_schema_version.dart Bump to 37
lib/core/models/user_preferences.dart Add prefetchMode, bodyCacheLimitMb
lib/data/repositories/user_preferences_repository_impl.dart Persist new fields
pubspec.yaml Add connectivity_plus
lib/core/services/network_info.dart New — Wi-Fi detection
lib/core/services/body_prefetch_service.dart New — prefetch + eviction logic
lib/core/sync/background_sync.dart Call prefetch service at end of sync
lib/ui/screens/user_preferences_screen.dart Add prefetch mode + cache limit controls
Here is the implementation plan to post as a comment on issue #373: --- ## Implementation Plan: Pre-fetch Email Bodies ### Overview Email bodies are currently fetched on-demand with a 7-day TTL. This plan adds a background pre-fetch mechanism that respects network type, skips large messages, and enforces a user-configured cache size limit. The existing `EmailBodies.cachedAt` column and background sync infrastructure in `background_sync.dart` are the natural hooks. --- ### Step 1 — Store message size during sync (schema v37) Add `sizeBytes IntColumn` (nullable) to the `Emails` table in `lib/data/db/database.dart`. Populate it from `RFC822.SIZE` during the existing IMAP sync so we have a size signal without an extra round-trip at prefetch time. Run `dart run build_runner build --delete-conflicting-outputs` after the schema change. --- ### Step 2 — Add prefetch settings to UserPreferences Extend `lib/core/models/user_preferences.dart` with two new fields: - `prefetchMode` (enum: `never` / `wifiOnly` / `always`, default `wifiOnly`) - `bodyCacheLimitMb` (int, default `100`) Persist them in the existing UserPreferences repository. No new DB table needed if prefs use a key-value store; otherwise add columns to whatever table backs them. --- ### Step 3 — Add `connectivity_plus` dependency Add `connectivity_plus` to `pubspec.yaml`. Create `lib/core/services/network_info.dart` with a single function: ```dart Future<bool> isOnWifi(); // true when ConnectivityResult.wifi ``` Used by the prefetch service and background sync to gate downloads. --- ### Step 4 — Body prefetch service New file: `lib/core/services/body_prefetch_service.dart` Algorithm: 1. Load `UserPreferences`. If `prefetchMode == never`, return immediately. 2. If `prefetchMode == wifiOnly` and not on Wi-Fi, return immediately. 3. Compute current cache size: `SELECT SUM(length(textBody) + length(htmlBody)) FROM email_bodies`. If already ≥ `bodyCacheLimitMb × 1024 × 1024`, evict before prefetching (Step 5). 4. Query candidate emails: `Emails` rows that have **no** corresponding `EmailBodies` row (or `cachedAt` is null/expired), ordered by `receivedAt DESC`, where `sizeBytes IS NULL OR sizeBytes < 512000` (skip messages over ~500 KB to avoid large attachments dragging down performance). 5. Process up to a configurable batch (e.g., 20 emails per run). For each, call the existing `getEmailBody()` logic — no new IMAP code needed, just reuse the existing on-demand fetch path. 6. Stop early if the cache size limit is reached mid-batch. The 512 KB per-message threshold should be exposed as a constant (not user-configurable) for now. --- ### Step 5 — Cache eviction When cache size exceeds the limit, delete `EmailBodies` rows oldest-first (`ORDER BY cachedAt ASC`) until the size is back under the limit. Implement as a private helper in the prefetch service, called at the start of each prefetch run. --- ### Step 6 — Integrate with background sync At the end of `_doBackgroundSync()` in `lib/core/sync/background_sync.dart`, instantiate and call `BodyPrefetchService.run()`. The existing `Constraints(networkType: NetworkType.connected)` on the WorkManager task already gates this on connectivity; the prefetch service adds the WiFi-vs-mobile check on top. --- ### Step 7 — Settings UI Add two controls to `lib/ui/screens/user_preferences_screen.dart`: - **Prefetch mode**: `RadioGroup` (Never / Wi-Fi only / Wi-Fi + mobile) — consistent with the existing radio groups for menu position. - **Cache limit**: A `DropdownButton` or segmented control with options 50 / 100 / 200 / 500 MB, plus a read-only display of current cache size (computed from the DB query in Step 4). --- ### What is NOT downloaded during prefetch - Binary attachments are never downloaded. The fetch call (`BODY.PEEK[]`) retrieves the full MIME message, but only `textBody` and `htmlBody` are stored; binary parts are discarded. The `sizeBytes` guard in Step 4 additionally skips entire messages that are too large to be worth fetching. - On-demand attachment viewing remains unchanged. --- ### File summary | File | Change | |---|---| | `lib/data/db/database.dart` | Add `sizeBytes` to `Emails` (schema v37) | | `lib/core/db_schema_version.dart` | Bump to 37 | | `lib/core/models/user_preferences.dart` | Add `prefetchMode`, `bodyCacheLimitMb` | | `lib/data/repositories/user_preferences_repository_impl.dart` | Persist new fields | | `pubspec.yaml` | Add `connectivity_plus` | | `lib/core/services/network_info.dart` | New — Wi-Fi detection | | `lib/core/services/body_prefetch_service.dart` | New — prefetch + eviction logic | | `lib/core/sync/background_sync.dart` | Call prefetch service at end of sync | | `lib/ui/screens/user_preferences_screen.dart` | Add prefetch mode + cache limit controls |
guettlibot commented 2026-06-04 01:30:03 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

agentloop: agent exceeded its 30m0s runtime budget and was stopped.

agent stderr tail:

[acpx] session agentloop-sharedinbox-issue-373 (0483c0a3-f6a8-4d5c-bfb9-5c3c6085a894) · /home/si/agentloop/loop-data/sharedinbox/issues/373 · agent needs reconnect
agentloop: agent exceeded its 30m0s runtime budget and was stopped. agent stderr tail: ``` [acpx] session agentloop-sharedinbox-issue-373 (0483c0a3-f6a8-4d5c-bfb9-5c3c6085a894) · /home/si/agentloop/loop-data/sharedinbox/issues/373 · agent needs reconnect ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
NeedSupervisor
Issue escalated to a human supervisor; agentloop will skip it until cleared.
 State/InProgress
State/Later
State/Planned
automerge
Eligible for automatic merge by CI
 ci-failure
Issue opened by agentloop to track a failing CI workflow; used for deduplication.
 do-not-merge
Plan PR — review only, do not merge.
 loop/code
Add to run the built-in "code" prompt; override at prompts/code.md.
 loop/code-ci-pending
Prompt "code" finished; waiting for the PR's CI to pass before advancing.
 loop/code-done
Prompt "code" finished successfully.
 loop/code-in-process
Agent for the "code" prompt is currently running on this issue.
 loop/merge
Managed by agentloop
 loop/merge-done
Managed by agentloop
 loop/merge-in-process
Managed by agentloop
 loop/plan
Add to run the built-in "plan" prompt; override at prompts/plan.md.
 loop/plan-done
Prompt "plan" finished successfully.
 loop/plan-in-process
Agent for the "plan" prompt is currently running on this issue.
No labels NeedSupervisor
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
guettlibot guettli
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: guettli/sharedinbox#373