guettli/sharedinbox
1
0
Fork 0
Code Issues 21 Pull Requests 6 Actions Packages Projects Releases Wiki Activity

CI: CheckGenerated runs build_runner twice per pipeline — eliminate duplicate work #492

New Issue
Closed
opened 2026-06-06 15:36:12 +00:00 by guettlibot · 1 comment
guettlibot commented 2026-06-06 15:36:12 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Problem

Every task check-dagger run executes build_runner build twice:

  1. In codegenBase() — shared by Format, Analyze, and Coverage via setup(checkSrc())
  2. In CheckGenerated() — a second, independent run starting from pubGetLayer()

The second run exists because CheckGenerated uses a git-diff approach: it snapshots the committed source (including stale .g.dart / .mocks.dart) with git commit, runs build_runner fresh, then diffs the generated files. This is necessary for detection correctness but expensive — build_runner takes ~2–5 min.

Since codegenBase() and CheckGenerated() diverge at pubGetLayer() (different source overlays), Dagger cannot share their execution caches.

Alternative approach — compare against codegenBase() output directly

Instead of re-running build_runner, CheckGenerated could:

  1. Take the set of committed generated files (from the source directory)
  2. Take the freshly generated files from codegenBase().Directory("/src")
  3. Diff them with a simple file comparison

This eliminates the second build_runner run entirely and saves ~3 min per CI run.

Sketch:

func (m *Ci) CheckGenerated(ctx context.Context) (string, error) {
    fresh := m.codegenBase().Directory("/src")
    // compare *.g.dart and *.mocks.dart between m.checkSrc() and fresh
    return m.pubGetLayer().
        WithDirectory("/committed", m.checkSrc(), ...).
        WithDirectory("/generated", fresh, ...).
        WithExec([]string{"bash", "-c",
            `diff -rq --include='*.g.dart' --include='*.mocks.dart' /committed /generated`}).
        Stdout(ctx)
}

Expected impact

  • Removes ~3 min from every CI check run.
  • Reduces Dagger engine compute on the remote host.

Files to change

  • ci/main.go — CheckGenerated() function (~line 509)
## Problem Every `task check-dagger` run executes `build_runner build` **twice**: 1. In `codegenBase()` — shared by Format, Analyze, and Coverage via `setup(checkSrc())` 2. In `CheckGenerated()` — a second, independent run starting from `pubGetLayer()` The second run exists because `CheckGenerated` uses a git-diff approach: it snapshots the committed source (including stale `.g.dart` / `.mocks.dart`) with `git commit`, runs `build_runner` fresh, then diffs the generated files. This is necessary for detection correctness but expensive — `build_runner` takes ~2–5 min. Since `codegenBase()` and `CheckGenerated()` diverge at `pubGetLayer()` (different source overlays), Dagger cannot share their execution caches. ## Alternative approach — compare against codegenBase() output directly Instead of re-running `build_runner`, `CheckGenerated` could: 1. Take the set of committed generated files (from the source directory) 2. Take the freshly generated files from `codegenBase().Directory("/src")` 3. Diff them with a simple file comparison This eliminates the second `build_runner` run entirely and saves ~3 min per CI run. Sketch: ```go func (m *Ci) CheckGenerated(ctx context.Context) (string, error) { fresh := m.codegenBase().Directory("/src") // compare *.g.dart and *.mocks.dart between m.checkSrc() and fresh return m.pubGetLayer(). WithDirectory("/committed", m.checkSrc(), ...). WithDirectory("/generated", fresh, ...). WithExec([]string{"bash", "-c", `diff -rq --include='*.g.dart' --include='*.mocks.dart' /committed /generated`}). Stdout(ctx) } ``` ## Expected impact - Removes ~3 min from every CI check run. - Reduces Dagger engine compute on the remote host. ## Files to change - `ci/main.go` — `CheckGenerated()` function (~line 509)
guettlibot commented 2026-06-07 00:26:32 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

The PR was created successfully at https://codeberg.org/guettli/sharedinbox/pulls/514.

Here's what I did to resolve issue #492:

Change made: ci/main.go — CheckGenerated() function (line 505)

The old implementation ran build_runner a second time using a git-snapshot approach (init repo, commit baseline, run build_runner, git diff). Since this started from pubGetLayer() with a different source overlay than codegenBase(), Dagger could not share the execution cache between them.

The new implementation:

  1. Grabs m.codegenBase().Directory("/src") — the freshly-generated files that codegenBase() already produced for Format/Analyze/Coverage (reusing the existing Dagger cache layer)
  2. Mounts committed source at /committed and the fresh output at /generated
  3. Runs diff -rq --include='*.g.dart' --include='*.mocks.dart' /committed /generated to detect any staleness

This eliminates the second build_runner invocation entirely, saving ~3 min per CI run. PR: https://codeberg.org/guettli/sharedinbox/pulls/514

The PR was created successfully at **https://codeberg.org/guettli/sharedinbox/pulls/514**. Here's what I did to resolve issue #492: **Change made:** `ci/main.go` — `CheckGenerated()` function (line 505) The old implementation ran `build_runner` a second time using a git-snapshot approach (init repo, commit baseline, run build_runner, git diff). Since this started from `pubGetLayer()` with a different source overlay than `codegenBase()`, Dagger could not share the execution cache between them. The new implementation: 1. Grabs `m.codegenBase().Directory("/src")` — the freshly-generated files that `codegenBase()` already produced for Format/Analyze/Coverage (reusing the existing Dagger cache layer) 2. Mounts committed source at `/committed` and the fresh output at `/generated` 3. Runs `diff -rq --include='*.g.dart' --include='*.mocks.dart' /committed /generated` to detect any staleness This eliminates the second `build_runner` invocation entirely, saving ~3 min per CI run. PR: https://codeberg.org/guettli/sharedinbox/pulls/514
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
NeedSupervisor
Issue escalated to a human supervisor; agentloop will skip it until cleared.
 State/InProgress
State/Later
State/Planned
automerge
Eligible for automatic merge by CI
 ci-failure
Issue opened by agentloop to track a failing CI workflow; used for deduplication.
 do-not-merge
Plan PR — review only, do not merge.
 loop/code
Add to run the built-in "code" prompt; override at prompts/code.md.
 loop/code-ci-pending
Prompt "code" finished; waiting for the PR's CI to pass before advancing.
 loop/code-done
Prompt "code" finished successfully.
 loop/code-in-process
Agent for the "code" prompt is currently running on this issue.
 loop/merge
Managed by agentloop
 loop/merge-done
Managed by agentloop
 loop/merge-in-process
Managed by agentloop
 loop/plan
Add to run the built-in "plan" prompt; override at prompts/plan.md.
 loop/plan-done
Prompt "plan" finished successfully.
 loop/plan-in-process
Agent for the "plan" prompt is currently running on this issue.
No labels loop/code-done
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
guettlibot guettli
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: guettli/sharedinbox#492