guettli/sharedinbox
1
0
Fork 0
Code Issues 21 Pull Requests 6 Actions Packages Projects Releases Wiki Activity

Structured Search #466

New Issue
Closed
opened 2026-06-06 06:59:53 +00:00 by guettli · 2 comments
guettli commented 2026-06-06 06:59:53 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

I want a structured search.

There should be a ui for it:

User can add several rows:

Field Comparison Value

Example:

From Contains foo

...

Combining with AND OR should be possible. With grouping.

Like: (expression1 and (exp2 or exp3))
...

The same is needed for local Sieve.

This ui is needed three times:

Search, local Sieve, remote Sieve.

...

Research code base. What exists and could be used/extended?

...

Finally it must be possible to create valid Sieve.

Parsing Sieve and show it in ui (so that user can update it) most be possible.

...

Not the whole Sieve spec needs to be implemented. A subset is enough.

I want a structured search. There should be a ui for it: User can add several rows: Field Comparison Value Example: From Contains foo ... Combining with AND OR should be possible. With grouping. Like: (expression1 and (exp2 or exp3)) ... The same is needed for local Sieve. This ui is needed three times: Search, local Sieve, remote Sieve. ... Research code base. What exists and could be used/extended? ... Finally it must be possible to create valid Sieve. Parsing Sieve and show it in ui (so that user can update it) most be possible. ... Not the whole Sieve spec needs to be implemented. A subset is enough.
guettlibot commented 2026-06-06 07:05:13 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

I now have everything I need. Here's the implementation plan:

Implementation Plan: Structured Search & Filter UI

What exists (reuse/extend)

  • SieveCondition, SieveRule, SieveParser, SieveInterpreter in lib/core/sieve/ — parse a Sieve text into a flat rule list. The parser can recurse into nested allof/anyof but flattens the result into SieveRule.conditions. No serialiser (rules → text) exists yet.
  • SieveScriptEditScreen — raw text editor, shared for local and remote Sieve via the isLocal flag.
  • SearchScreen — simple text search over a FTS5 virtual table (email_fts, covering subject/preview/from_json) plus IMAP server search.
  • EmailRepository.searchEmailsGlobal — executes FTS5 SQL directly.

Step 1 — Add FilterExpression tree model

New file: lib/core/filter/filter_expression.dart

enum FilterField { from_, to, cc, subject, size, hasAttachment }
enum FilterComparison { contains, is_, matches, over, under, exists }
enum FilterOperator { and_, or_ }

sealed class FilterNode {}
final class FilterLeaf extends FilterNode { field, comparison, value }
final class FilterGroup extends FilterNode { operator, List<FilterNode> children }

This is the canonical in-memory representation shared by all three surfaces (Search, local Sieve, remote Sieve). It is intentionally separate from SieveRule (which stays as the interpreter's flat runtime model).

Supported subset:

  • Fields: from, to, cc, subject (→ header/address Sieve test), size, hasAttachment (→ exists)
  • Comparisons: contains, is, matches, over, under, exists
  • Nesting: one level of sub-groups is enough to express (A AND (B OR C)). The model supports arbitrary depth; the UI renders two levels.

Step 2 — Add Sieve serialiser

New file: lib/core/sieve/sieve_serializer.dart

String serialise(FilterGroup filter, List<SieveAction> actions) → emits a complete Sieve script:

require ["fileinto", "imap4flags"];
if allof(
    address :contains "from" "foo",
    anyof(header :contains "subject" "bar", size :over 100K)
) {
    fileinto "Archive";
}

Maps FilterGroup(and_) → allof(…), FilterGroup(or_) → anyof(…), FilterLeaf → the appropriate Sieve test. Nested FilterGroup nodes become nested allof/anyof calls, which is valid RFC 5228.

Step 3 — Add Sieve → FilterExpression converter

New file: lib/core/filter/filter_sieve_converter.dart

FilterGroup? parseFilterGroup(String script) reuses SieveParser's _Scanner but builds a FilterGroup tree instead of a flat SieveRule list. It reads only the first if block's test expression; actions are extracted separately as List<SieveAction>.

If the script contains features outside the supported subset (e.g., envelope, body, not), return null so the caller can fall back to the raw text editor.

Step 4 — Add structured SQL search

New method on EmailRepository (abstract + impl):

Future<List<Email>> searchEmailsStructured(
    String? accountId, FilterGroup filter);

New file: lib/core/filter/filter_sql_builder.dart

Builds a Drift Expression<bool> from a FilterGroup by walking the tree. Maps:

  • from / to / cc → JSON field checks on from_json / to_json / cc_json
  • subject → LIKE '%value%' (or = for is)
  • size → not in local cache (omit or return empty; note email size is not stored locally)
  • hasAttachment → has_attachment = 1

No DB schema changes needed.

Step 5 — Build FilterBuilderWidget

New file: lib/ui/widgets/filter_builder.dart

FilterBuilderWidget({required FilterGroup value, required void Function(FilterGroup) onChanged})

Renders:

  • A SegmentedButton<FilterOperator> (AND / OR) at the top of each group
  • One row per FilterLeaf: [FieldDropdown] [ComparisonDropdown] [TextField] [IconButton(delete)]
  • Nested FilterGroup children rendered recursively with a Card/indent + their own AND/OR toggle
  • "Add condition" TextButton appends a new FilterLeaf
  • "Add group" TextButton appends a new nested FilterGroup
  • Available FilterComparison values update based on the selected FilterField (text fields show contains/is/matches; size shows over/under; hasAttachment shows only exists)

Step 6 — Integrate into Search screen

lib/ui/screens/search_screen.dart

  • Add an IconButton(Icons.tune) in the AppBar that toggles an "advanced" panel
  • Advanced panel: FilterBuilderWidget + a "Search" button; the free-text field is hidden in advanced mode
  • On search: call emailRepo.searchEmailsStructured(accountId, filter) and display results in the same _SearchResults widget (mailbox/address sections are omitted in structured mode)
  • No persistence of the filter between sessions; state lives in _SearchScreenState

Step 7 — Integrate into Sieve edit screen

lib/ui/screens/sieve_script_edit_screen.dart

  • Add a TabBar with two tabs: "Visual" and "Script"
  • Script tab: unchanged raw text editor
  • Visual tab:
    • On load: call FilterSieveConverter.parseFilterGroup(script); if successful populate FilterBuilderWidget + a simple action editor (see below); if it returns null, show a banner "Script uses unsupported features — edit in Script tab" and disable the Visual tab
    • ActionEditor: a simple DropdownButton + optional text field for the one action of the script's if block (File Into [folder] / Mark as Read / Discard / Keep). Only one action per script is supported in the Visual editor; multi-action scripts load as null.
    • On switch from Visual → Script: serialise FilterGroup + action via SieveSerialiser and update the script text field
  • The same SieveScriptEditScreen is used for both isLocal = true and isLocal = false, so the Visual editor works for all three Sieve surfaces (local, ManageSieve, JMAP)

File summary

File Status
lib/core/filter/filter_expression.dart New
lib/core/filter/filter_sieve_converter.dart New
lib/core/filter/filter_sql_builder.dart New
lib/core/sieve/sieve_serializer.dart New
lib/ui/widgets/filter_builder.dart New
lib/core/repositories/email_repository.dart Add searchEmailsStructured
lib/data/repositories/email_repository_impl.dart Implement searchEmailsStructured
lib/ui/screens/search_screen.dart Add advanced toggle + structured search
lib/ui/screens/sieve_script_edit_screen.dart Add Visual/Script tabs

No DB schema changes → no build_runner run required.

Out of scope

  • Nesting deeper than two levels in the UI (the model supports it; the widget renders two levels)
  • Parsing Sieve with not, envelope, body, reject, or multiple if blocks in the Visual editor (graceful fallback to Script tab)
  • IMAP server-side structured search (IMAP SEARCH command); structured search stays local-DB-only for now
  • Persisting the last-used structured search query
I now have everything I need. Here's the implementation plan: --- ## Implementation Plan: Structured Search & Filter UI ### What exists (reuse/extend) - `SieveCondition`, `SieveRule`, `SieveParser`, `SieveInterpreter` in `lib/core/sieve/` — parse a Sieve text into a flat rule list. The parser **can** recurse into nested `allof`/`anyof` but flattens the result into `SieveRule.conditions`. No serialiser (rules → text) exists yet. - `SieveScriptEditScreen` — raw text editor, shared for local and remote Sieve via the `isLocal` flag. - `SearchScreen` — simple text search over a FTS5 virtual table (`email_fts`, covering subject/preview/from_json) plus IMAP server search. - `EmailRepository.searchEmailsGlobal` — executes FTS5 SQL directly. --- ### Step 1 — Add `FilterExpression` tree model New file: `lib/core/filter/filter_expression.dart` ``` enum FilterField { from_, to, cc, subject, size, hasAttachment } enum FilterComparison { contains, is_, matches, over, under, exists } enum FilterOperator { and_, or_ } sealed class FilterNode {} final class FilterLeaf extends FilterNode { field, comparison, value } final class FilterGroup extends FilterNode { operator, List<FilterNode> children } ``` This is the canonical in-memory representation shared by all three surfaces (Search, local Sieve, remote Sieve). It is intentionally separate from `SieveRule` (which stays as the interpreter's flat runtime model). Supported subset: - Fields: `from`, `to`, `cc`, `subject` (→ `header`/`address` Sieve test), `size`, `hasAttachment` (→ `exists`) - Comparisons: `contains`, `is`, `matches`, `over`, `under`, `exists` - Nesting: one level of sub-groups is enough to express `(A AND (B OR C))`. The model supports arbitrary depth; the UI renders two levels. --- ### Step 2 — Add Sieve serialiser New file: `lib/core/sieve/sieve_serializer.dart` `String serialise(FilterGroup filter, List<SieveAction> actions)` → emits a complete Sieve script: ```sieve require ["fileinto", "imap4flags"]; if allof( address :contains "from" "foo", anyof(header :contains "subject" "bar", size :over 100K) ) { fileinto "Archive"; } ``` Maps `FilterGroup(and_)` → `allof(…)`, `FilterGroup(or_)` → `anyof(…)`, `FilterLeaf` → the appropriate Sieve test. Nested `FilterGroup` nodes become nested `allof`/`anyof` calls, which is valid RFC 5228. --- ### Step 3 — Add Sieve → `FilterExpression` converter New file: `lib/core/filter/filter_sieve_converter.dart` `FilterGroup? parseFilterGroup(String script)` reuses `SieveParser`'s `_Scanner` but builds a `FilterGroup` tree instead of a flat `SieveRule` list. It reads only the first `if` block's test expression; actions are extracted separately as `List<SieveAction>`. If the script contains features outside the supported subset (e.g., `envelope`, `body`, `not`), return `null` so the caller can fall back to the raw text editor. --- ### Step 4 — Add structured SQL search New method on `EmailRepository` (abstract + impl): ```dart Future<List<Email>> searchEmailsStructured( String? accountId, FilterGroup filter); ``` New file: `lib/core/filter/filter_sql_builder.dart` Builds a Drift `Expression<bool>` from a `FilterGroup` by walking the tree. Maps: - `from` / `to` / `cc` → JSON field checks on `from_json` / `to_json` / `cc_json` - `subject` → `LIKE '%value%'` (or `=` for `is`) - `size` → not in local cache (omit or return empty; note email size is not stored locally) - `hasAttachment` → `has_attachment = 1` No DB schema changes needed. --- ### Step 5 — Build `FilterBuilderWidget` New file: `lib/ui/widgets/filter_builder.dart` `FilterBuilderWidget({required FilterGroup value, required void Function(FilterGroup) onChanged})` Renders: - A `SegmentedButton<FilterOperator>` (AND / OR) at the top of each group - One row per `FilterLeaf`: `[FieldDropdown] [ComparisonDropdown] [TextField] [IconButton(delete)]` - Nested `FilterGroup` children rendered recursively with a Card/indent + their own AND/OR toggle - "Add condition" `TextButton` appends a new `FilterLeaf` - "Add group" `TextButton` appends a new nested `FilterGroup` - Available `FilterComparison` values update based on the selected `FilterField` (text fields show contains/is/matches; size shows over/under; hasAttachment shows only exists) --- ### Step 6 — Integrate into Search screen `lib/ui/screens/search_screen.dart` - Add an `IconButton(Icons.tune)` in the AppBar that toggles an "advanced" panel - Advanced panel: `FilterBuilderWidget` + a "Search" button; the free-text field is hidden in advanced mode - On search: call `emailRepo.searchEmailsStructured(accountId, filter)` and display results in the same `_SearchResults` widget (mailbox/address sections are omitted in structured mode) - No persistence of the filter between sessions; state lives in `_SearchScreenState` --- ### Step 7 — Integrate into Sieve edit screen `lib/ui/screens/sieve_script_edit_screen.dart` - Add a `TabBar` with two tabs: "Visual" and "Script" - **Script tab**: unchanged raw text editor - **Visual tab**: - On load: call `FilterSieveConverter.parseFilterGroup(script)`; if successful populate `FilterBuilderWidget` + a simple action editor (see below); if it returns null, show a banner "Script uses unsupported features — edit in Script tab" and disable the Visual tab - `ActionEditor`: a simple `DropdownButton` + optional text field for the one action of the script's `if` block (File Into `[folder]` / Mark as Read / Discard / Keep). Only one action per script is supported in the Visual editor; multi-action scripts load as null. - On switch from Visual → Script: serialise `FilterGroup + action` via `SieveSerialiser` and update the script text field - The same `SieveScriptEditScreen` is used for both `isLocal = true` and `isLocal = false`, so the Visual editor works for all three Sieve surfaces (local, ManageSieve, JMAP) --- ### File summary | File | Status | |---|---| | `lib/core/filter/filter_expression.dart` | New | | `lib/core/filter/filter_sieve_converter.dart` | New | | `lib/core/filter/filter_sql_builder.dart` | New | | `lib/core/sieve/sieve_serializer.dart` | New | | `lib/ui/widgets/filter_builder.dart` | New | | `lib/core/repositories/email_repository.dart` | Add `searchEmailsStructured` | | `lib/data/repositories/email_repository_impl.dart` | Implement `searchEmailsStructured` | | `lib/ui/screens/search_screen.dart` | Add advanced toggle + structured search | | `lib/ui/screens/sieve_script_edit_screen.dart` | Add Visual/Script tabs | No DB schema changes → no `build_runner` run required. --- ### Out of scope - Nesting deeper than two levels in the UI (the model supports it; the widget renders two levels) - Parsing Sieve with `not`, `envelope`, `body`, `reject`, or multiple `if` blocks in the Visual editor (graceful fallback to Script tab) - IMAP server-side structured search (IMAP SEARCH command); structured search stays local-DB-only for now - Persisting the last-used structured search query
guettlibot commented 2026-06-06 08:15:04 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

agentloop: agent exceeded its 30m0s runtime budget and was stopped.

agent stderr tail:

[acpx] session agentloop-sharedinbox-issue-466 (4fcf0d78-2287-4183-bd34-65ed2d98512e) · /home/si/agentloop/loop-data/sharedinbox/issues/466 · agent needs reconnect
agentloop: agent exceeded its 30m0s runtime budget and was stopped. agent stderr tail: ``` [acpx] session agentloop-sharedinbox-issue-466 (4fcf0d78-2287-4183-bd34-65ed2d98512e) · /home/si/agentloop/loop-data/sharedinbox/issues/466 · agent needs reconnect ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
NeedSupervisor
Issue escalated to a human supervisor; agentloop will skip it until cleared.
 State/InProgress
State/Later
State/Planned
automerge
Eligible for automatic merge by CI
 ci-failure
Issue opened by agentloop to track a failing CI workflow; used for deduplication.
 do-not-merge
Plan PR — review only, do not merge.
 loop/code
Add to run the built-in "code" prompt; override at prompts/code.md.
 loop/code-ci-pending
Prompt "code" finished; waiting for the PR's CI to pass before advancing.
 loop/code-done
Prompt "code" finished successfully.
 loop/code-in-process
Agent for the "code" prompt is currently running on this issue.
 loop/merge
Managed by agentloop
 loop/merge-done
Managed by agentloop
 loop/merge-in-process
Managed by agentloop
 loop/plan
Add to run the built-in "plan" prompt; override at prompts/plan.md.
 loop/plan-done
Prompt "plan" finished successfully.
 loop/plan-in-process
Agent for the "plan" prompt is currently running on this issue.
No labels NeedSupervisor
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
guettlibot guettli
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: guettli/sharedinbox#466