fix: limit dagger-data volume growth by pruning named caches (#193) #197

Merged
guettlibot merged 1 commits from issue-193-fix into main 2026-05-24 04:00:18 +00:00
guettlibot commented 2026-05-24 03:52:08 +00:00 (Migrated from codeberg.org)

Why the volume grows to 200 GB

The dagger-data Docker volume accumulates because Dagger maintains two kinds of cache:

  1. Execution-cache snapshots — results of WithExec() steps (pub get, build_runner, flutter build, etc.). These are already pruned by the existing prune calls because they become unreferenced after a build completes.

  2. Named cache volumesgradle-cache, go-build-cache, go-mod-cache, pip-cache created via dag.CacheVolume(). The plain prune (no arguments) only removes unreferenced entries and never touches named volumes, so they grow without bound across CI runs.

Fix

The Dagger localCache.prune API accepts maxUsedSpace and targetSpace arguments (discovered in the generated dagger.gen.go). When maxUsedSpace is set, Dagger will also reclaim named cache volumes if the total exceeds the threshold.

  • CI prune steps (before and after check): prune(maxUsedSpace: "75gb", targetSpace: "50gb") — only reclaims named volumes when the total exceeds 75 GB, leaving frequently-used caches intact otherwise.
  • Disk-space emergency recovery in check-dagger: prune(targetSpace: "20gb") — aggressively prunes everything including named volumes when the runner is out of disk space.
  • New dagger-prune task for manual operator use.

Test plan

  • CI passes on this branch
  • task dagger-prune runs without error against a live Dagger engine
  • After several CI runs, docker volume inspect dagger-data shows size well below 200 GB

🤖 Generated with Claude Code

## Why the volume grows to 200 GB The `dagger-data` Docker volume accumulates because Dagger maintains two kinds of cache: 1. **Execution-cache snapshots** — results of `WithExec()` steps (pub get, build_runner, flutter build, etc.). These are already pruned by the existing `prune` calls because they become unreferenced after a build completes. 2. **Named cache volumes** — `gradle-cache`, `go-build-cache`, `go-mod-cache`, `pip-cache` created via `dag.CacheVolume()`. The plain `prune` (no arguments) only removes *unreferenced* entries and never touches named volumes, so they grow without bound across CI runs. ## Fix The Dagger `localCache.prune` API accepts `maxUsedSpace` and `targetSpace` arguments (discovered in the generated `dagger.gen.go`). When `maxUsedSpace` is set, Dagger will also reclaim named cache volumes if the total exceeds the threshold. - **CI prune steps** (before and after check): `prune(maxUsedSpace: "75gb", targetSpace: "50gb")` — only reclaims named volumes when the total exceeds 75 GB, leaving frequently-used caches intact otherwise. - **Disk-space emergency recovery** in `check-dagger`: `prune(targetSpace: "20gb")` — aggressively prunes everything including named volumes when the runner is out of disk space. - **New `dagger-prune` task** for manual operator use. ## Test plan - [ ] CI passes on this branch - [ ] `task dagger-prune` runs without error against a live Dagger engine - [ ] After several CI runs, `docker volume inspect dagger-data` shows size well below 200 GB 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.