guettli/sharedinbox
1
0
Fork 0
Code Issues 21 Pull Requests 6 Actions Packages Projects Releases Wiki Activity

flutter test --concurrency=<number_of_cores> #328

New Issue
Open
opened 2026-05-28 06:39:40 +00:00 by guettli · 2 comments
guettli commented 2026-05-28 06:39:40 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Create plan to switch to:

flutter test --concurrency=<number_of_cores>

to speed up tests.

Create plan to switch to: ``` flutter test --concurrency=<number_of_cores> ``` to speed up tests.
guettlibot commented 2026-05-28 06:46:11 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Implementation Plan: flutter test --concurrency=<number_of_cores>

Context

After exploring the codebase, there are two distinct situations that must be handled differently:

Test suite Current concurrency Can be parallelised?
Unit tests (test/unit/) Default (unset) Yes
Widget tests (test/widget/) Default (unset) Yes
Backend tests (test/backend/) --concurrency=1 (explicit) No – uses a live Stalwart instance
Sync reliability tests --concurrency=1 (explicit) No – same reason

The backend and sync-reliability tests must stay at --concurrency=1 because they talk to a shared Stalwart mail server; parallel execution would cause race conditions on mailbox state. Only unit and widget tests are safe to parallelise.

Files to change

1. scripts/run_unit_tests.sh

Current invocation:

fvm flutter test test/unit/ test/widget/ --coverage --no-pub --reporter expanded

Change to:

CORES=$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4)
fvm flutter test test/unit/ test/widget/ --concurrency="$CORES" --coverage --no-pub --reporter expanded

2. scripts/run_widget_tests.sh

Current invocation:

fvm flutter test test/widget/ --no-pub --reporter expanded

Change to:

CORES=$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4)
fvm flutter test test/widget/ --concurrency="$CORES" --no-pub --reporter expanded

3. ci/main.go – Coverage() function (unit test step)

Add runtime.NumCPU() import and pass concurrency to the flutter test invocation for the coverage/unit-test run. The TestBackend() and TestSyncReliability() functions must keep --concurrency=1 unchanged.

import (
    "runtime"
    "strconv"
)

// inside Coverage():
cores := strconv.Itoa(runtime.NumCPU())
// add "--concurrency=" + cores to the flutter test args for unit tests

4. Taskfile.yml – test-flutter task

Current:

fvm flutter test

Change to (using Taskfile's sh interpolation):

fvm flutter test --concurrency={{"sh": "nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4"}}

Approach summary

  • Use nproc (Linux) / sysctl -n hw.logicalcpu (macOS) with a fallback of 4 in shell scripts.
  • Use runtime.NumCPU() in Go (Dagger CI pipeline, ci/main.go).
  • Leave all --concurrency=1 invocations (backend, sync-reliability) untouched.

Risks and open questions

  1. Test isolation – The biggest risk. If any unit or widget test writes to a shared global, singleton, or static field (common with GetIt, SharedPreferences fakes, or hive), concurrent execution will cause flaky failures. All tests should be reviewed for shared mutable state before enabling higher concurrency.

  2. CI runner resources – The self-hosted runner's core count determines the effective concurrency. If the runner has many cores but limited RAM, high concurrency could cause OOM during widget tests (each isolate spawns a Flutter engine). Consider capping concurrency (e.g., min(nproc, 8)) if this becomes an issue.

  3. Coverage file races – When --merge-coverage is used alongside high concurrency, multiple test isolates may write to coverage/lcov.info simultaneously. This is not an issue for the unit-test run (no --merge-coverage), but worth noting if coverage merging is added later.

  4. Reporter readability – The --reporter expanded output becomes interleaved and harder to read at high concurrency. Switching to --reporter json (and post-processing) or --reporter compact might be worth considering for CI.

  5. stalwart-dev/test.sh – This local development script already uses --concurrency=1 for backend tests; no change needed there.

## Implementation Plan: `flutter test --concurrency=<number_of_cores>` ### Context After exploring the codebase, there are **two distinct situations** that must be handled differently: | Test suite | Current concurrency | Can be parallelised? | |---|---|---| | Unit tests (`test/unit/`) | Default (unset) | Yes | | Widget tests (`test/widget/`) | Default (unset) | Yes | | Backend tests (`test/backend/`) | `--concurrency=1` (explicit) | **No** – uses a live Stalwart instance | | Sync reliability tests | `--concurrency=1` (explicit) | **No** – same reason | The backend and sync-reliability tests **must stay at `--concurrency=1`** because they talk to a shared Stalwart mail server; parallel execution would cause race conditions on mailbox state. Only unit and widget tests are safe to parallelise. --- ### Files to change #### 1. `scripts/run_unit_tests.sh` Current invocation: ```bash fvm flutter test test/unit/ test/widget/ --coverage --no-pub --reporter expanded ``` Change to: ```bash CORES=$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4) fvm flutter test test/unit/ test/widget/ --concurrency="$CORES" --coverage --no-pub --reporter expanded ``` #### 2. `scripts/run_widget_tests.sh` Current invocation: ```bash fvm flutter test test/widget/ --no-pub --reporter expanded ``` Change to: ```bash CORES=$(nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4) fvm flutter test test/widget/ --concurrency="$CORES" --no-pub --reporter expanded ``` #### 3. `ci/main.go` – `Coverage()` function (unit test step) Add `runtime.NumCPU()` import and pass concurrency to the flutter test invocation for the coverage/unit-test run. The `TestBackend()` and `TestSyncReliability()` functions must keep `--concurrency=1` unchanged. ```go import ( "runtime" "strconv" ) // inside Coverage(): cores := strconv.Itoa(runtime.NumCPU()) // add "--concurrency=" + cores to the flutter test args for unit tests ``` #### 4. `Taskfile.yml` – `test-flutter` task Current: ```yaml fvm flutter test ``` Change to (using Taskfile's `sh` interpolation): ```yaml fvm flutter test --concurrency={{"sh": "nproc 2>/dev/null || sysctl -n hw.logicalcpu 2>/dev/null || echo 4"}} ``` --- ### Approach summary - Use `nproc` (Linux) / `sysctl -n hw.logicalcpu` (macOS) with a fallback of `4` in shell scripts. - Use `runtime.NumCPU()` in Go (Dagger CI pipeline, `ci/main.go`). - Leave all `--concurrency=1` invocations (backend, sync-reliability) untouched. --- ### Risks and open questions 1. **Test isolation** – The biggest risk. If any unit or widget test writes to a shared global, singleton, or static field (common with `GetIt`, `SharedPreferences` fakes, or `hive`), concurrent execution will cause flaky failures. All tests should be reviewed for shared mutable state before enabling higher concurrency. 2. **CI runner resources** – The self-hosted runner's core count determines the effective concurrency. If the runner has many cores but limited RAM, high concurrency could cause OOM during widget tests (each isolate spawns a Flutter engine). Consider capping concurrency (e.g., `min(nproc, 8)`) if this becomes an issue. 3. **Coverage file races** – When `--merge-coverage` is used alongside high concurrency, multiple test isolates may write to `coverage/lcov.info` simultaneously. This is not an issue for the unit-test run (no `--merge-coverage`), but worth noting if coverage merging is added later. 4. **Reporter readability** – The `--reporter expanded` output becomes interleaved and harder to read at high concurrency. Switching to `--reporter json` (and post-processing) or `--reporter compact` might be worth considering for CI. 5. **`stalwart-dev/test.sh`** – This local development script already uses `--concurrency=1` for backend tests; no change needed there.
guettlibot commented 2026-05-28 06:47:01 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Planning complete. To resume this session:

claude --resume 1b1dff33-696c-479f-88f0-19714006ccb0
Planning complete. To resume this session: ``` claude --resume 1b1dff33-696c-479f-88f0-19714006ccb0 ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
NeedSupervisor
Issue escalated to a human supervisor; agentloop will skip it until cleared.
 State/InProgress
State/Later
State/Planned
automerge
Eligible for automatic merge by CI
 ci-failure
Issue opened by agentloop to track a failing CI workflow; used for deduplication.
 do-not-merge
Plan PR — review only, do not merge.
 loop/code
Add to run the built-in "code" prompt; override at prompts/code.md.
 loop/code-ci-pending
Prompt "code" finished; waiting for the PR's CI to pass before advancing.
 loop/code-done
Prompt "code" finished successfully.
 loop/code-in-process
Agent for the "code" prompt is currently running on this issue.
 loop/merge
Managed by agentloop
 loop/merge-done
Managed by agentloop
 loop/merge-in-process
Managed by agentloop
 loop/plan
Add to run the built-in "plan" prompt; override at prompts/plan.md.
 loop/plan-done
Prompt "plan" finished successfully.
 loop/plan-in-process
Agent for the "plan" prompt is currently running on this issue.
No labels State/Planned
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
guettlibot guettli
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: guettli/sharedinbox#328