guettli/sharedinbox
1
0
Fork 0
Code Issues 21 Pull Requests 6 Actions Packages Projects Releases Wiki Activity

CI: parallelize Format/Analyze/CheckGenerated/Coverage in Check() to cut wall-clock time ~50% #491

New Issue
Closed
opened 2026-06-06 15:35:52 +00:00 by guettlibot · 1 comment
guettlibot commented 2026-06-06 15:35:52 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Problem

The Check() function in ci/main.go runs four checks sequentially even though they have no data dependencies on each other's output:

Phase 1 [parallel]  : CheckHygiene + CheckLayers      (trivially fast)
Phase 2 [sequential]: Format                           (dart format check)
Phase 3 [sequential]: Analyze                          (dart analyze)
Phase 4 [sequential]: CheckGenerated                   (build_runner + git diff)
Phase 5 [sequential]: Coverage                         (unit + widget tests)
Phase 6 [parallel]  : TestBackend + TestIntegration

All four of phases 2–5 share the same setup(checkSrc()) Dagger base. None of them depends on the output of another. Running them sequentially means waiting ~8–12 min for four steps that could complete in ~3 min (the slowest one wins).

Note: CheckFast() already does this correctly — it runs hygiene, layers, format, analyze, mocks, and coverage in parallel. Check() should match that structure.

Fix

Replace the sequential block in Check() with an errgroup.Group, similar to what TestBackend + TestIntegration already use:

var eg errgroup.Group
eg.Go(func() error { _, err := m.Format(ctx); return err })
eg.Go(func() error { _, err := m.Analyze(ctx); return err })
eg.Go(func() error { _, err := m.CheckGenerated(ctx); return err })
eg.Go(func() error { _, err := m.Coverage(ctx); return err })
if err := eg.Wait(); err != nil {
    return "", err
}

Then run TestBackend + TestIntegration in parallel after (as already done).

Expected impact

  • Wall-clock time for the CI check job drops by roughly half.
  • No correctness change — all the same checks run, just concurrently.
  • Reduces the time a developer waits for PR feedback significantly.

Files to change

  • ci/main.go — Check() function (~line 579)
## Problem The `Check()` function in `ci/main.go` runs four checks **sequentially** even though they have no data dependencies on each other's output: ``` Phase 1 [parallel] : CheckHygiene + CheckLayers (trivially fast) Phase 2 [sequential]: Format (dart format check) Phase 3 [sequential]: Analyze (dart analyze) Phase 4 [sequential]: CheckGenerated (build_runner + git diff) Phase 5 [sequential]: Coverage (unit + widget tests) Phase 6 [parallel] : TestBackend + TestIntegration ``` All four of phases 2–5 share the same `setup(checkSrc())` Dagger base. None of them depends on the output of another. Running them sequentially means waiting ~8–12 min for four steps that could complete in ~3 min (the slowest one wins). Note: `CheckFast()` already does this correctly — it runs hygiene, layers, format, analyze, mocks, and coverage **in parallel**. `Check()` should match that structure. ## Fix Replace the sequential block in `Check()` with an `errgroup.Group`, similar to what `TestBackend + TestIntegration` already use: ```go var eg errgroup.Group eg.Go(func() error { _, err := m.Format(ctx); return err }) eg.Go(func() error { _, err := m.Analyze(ctx); return err }) eg.Go(func() error { _, err := m.CheckGenerated(ctx); return err }) eg.Go(func() error { _, err := m.Coverage(ctx); return err }) if err := eg.Wait(); err != nil { return "", err } ``` Then run TestBackend + TestIntegration in parallel after (as already done). ## Expected impact - Wall-clock time for the CI check job drops by roughly half. - No correctness change — all the same checks run, just concurrently. - Reduces the time a developer waits for PR feedback significantly. ## Files to change - `ci/main.go` — `Check()` function (~line 579)
guettlibot commented 2026-06-07 00:21:19 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

The PR is open at https://codeberg.org/guettli/sharedinbox/pulls/513.

Here's what I did:

Change: In ci/main.go (Check(), around line 597), replaced the four sequential calls — dart format, dart analyze, CheckGenerated, and Coverage — with a single errgroup.Group (checkEg) that runs all four concurrently. Each goroutine captures its own result variable. The downstream TestBackend/TestIntegration parallel block and the final fmt.Sprintf return are unchanged.

Why no data races: Each goroutine creates its own m.setup(m.checkSrc()) Dagger pipeline handle. The only shared mutable state is analyze, mocks, and coverage strings, which are each written by exactly one goroutine and only read after checkEg.Wait().

PR: https://codeberg.org/guettli/sharedinbox/pulls/513 (Closes #491)

The PR is open at **https://codeberg.org/guettli/sharedinbox/pulls/513**. --- Here's what I did: **Change:** In `ci/main.go` (`Check()`, around line 597), replaced the four sequential calls — `dart format`, `dart analyze`, `CheckGenerated`, and `Coverage` — with a single `errgroup.Group` (`checkEg`) that runs all four concurrently. Each goroutine captures its own result variable. The downstream `TestBackend`/`TestIntegration` parallel block and the final `fmt.Sprintf` return are unchanged. **Why no data races:** Each goroutine creates its own `m.setup(m.checkSrc())` Dagger pipeline handle. The only shared mutable state is `analyze`, `mocks`, and `coverage` strings, which are each written by exactly one goroutine and only read after `checkEg.Wait()`. **PR:** https://codeberg.org/guettli/sharedinbox/pulls/513 (Closes #491)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
NeedSupervisor
Issue escalated to a human supervisor; agentloop will skip it until cleared.
 State/InProgress
State/Later
State/Planned
automerge
Eligible for automatic merge by CI
 ci-failure
Issue opened by agentloop to track a failing CI workflow; used for deduplication.
 do-not-merge
Plan PR — review only, do not merge.
 loop/code
Add to run the built-in "code" prompt; override at prompts/code.md.
 loop/code-ci-pending
Prompt "code" finished; waiting for the PR's CI to pass before advancing.
 loop/code-done
Prompt "code" finished successfully.
 loop/code-in-process
Agent for the "code" prompt is currently running on this issue.
 loop/merge
Managed by agentloop
 loop/merge-done
Managed by agentloop
 loop/merge-in-process
Managed by agentloop
 loop/plan
Add to run the built-in "plan" prompt; override at prompts/plan.md.
 loop/plan-done
Prompt "plan" finished successfully.
 loop/plan-in-process
Agent for the "plan" prompt is currently running on this issue.
No labels loop/code-done
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
guettlibot guettli
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: guettli/sharedinbox#491