## Summary
Fixes the persistent `Load key "/root/.ssh/id_ed25519": error in libcrypto` failures in the `deploy-apk` and `deploy-linux` CI jobs (and the `website` workflow SSH steps) that have been occurring on every deploy run since the jobs first started running after #369.
Closes#404
### Root cause (diagnosed from run #1516 log)
Two compounding problems were found:
1. **Stale Dagger cache** — The `tr -d \x27\r\x27` normalisation step added in #369 was shown as `CACHED` by Dagger on every subsequent run. Dagger caches by input-content hash; if the very first execution produced a corrupted key file, that broken cached layer is replayed forever.
2. **`.ssh/` directory permissions** — Dagger creates parent directories for secret mounts with 755 permissions. Mounting the raw key directly inside `/root/.ssh/` may cause Dagger to (re-)create that directory with 755 instead of the 700 that OpenSSH requires.
### Changes (`ci/main.go` — `Deployer` function only)
- **Explicit `.ssh` setup**: `mkdir -p /root/.ssh && chmod 700 /root/.ssh` runs before any Dagger secret mount.
- **Move raw-key mount out of `.ssh/`**: Secret mounted at `/tmp/id_ed25519.raw`.
- **Python3 normalisation instead of `tr`**: Handles CRLF, bare-CR, and missing trailing newline. Changing the command changes the Dagger cache key, forcing a fresh read of the current live secret.
## Test plan
- [ ] `deploy-apk` job completes without `error in libcrypto`
- [ ] `deploy-linux` job completes without `error in libcrypto`
- [ ] `publish-android` (Play Store) job continues to succeed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Thomas SharedInbox <sharedinbox@thomas-guettler.de>
Reviewed-on: https://codeberg.org/guettli/sharedinbox/pulls/406
## What
PR #356 (Renovate) was blocked with `renovate/artifacts` — \"Artifact file update failure\" — because `ci/go.sum` could not be updated automatically.
**Root cause**: `ci/main.go` imports `dagger/ci/internal/dagger` (generated by `dagger develop`, not committed to the repo). Without that generated package present, `go mod tidy` cannot resolve the full dependency graph, so Renovate's artifact update step always fails.
The actual OpenTelemetry version bump from PR #356 was already applied manually in PR #363.
## Fix
Adds a `packageRule` to `renovate.json` to disable the `gomod` manager for `ci/**`. Renovate will no longer open failing PRs for Go dependencies in the Dagger CI module; updates to `ci/go.mod` and `ci/go.sum` must be done manually (using `dagger develop && go mod tidy` inside `ci/`).
## Verification
- `renovate.json` validates against the Renovate schema.
- No Go or Drift schema changes; `task check` is unaffected.
Closes#368
Co-authored-by: Thomas SharedInbox <sharedinbox@thomas-guettler.de>
Co-authored-by: guettli <guettli@noreply.codeberg.org>
Reviewed-on: https://codeberg.org/guettli/sharedinbox/pulls/370
## Summary
Fixes three distinct failures from CI deploy run #1424 and concurrent website update failures.
- **Play Store job**: `pip install google-auth requests` fails on Ubuntu 24.04 with PEP 668. Fixed by using `python3 -m venv` for an isolated install.
- **SSH key error (APK, Linux, website jobs)**: All SSH/rsync steps fail with `Load key "/root/.ssh/id_ed25519": error in libcrypto` inside the Dagger Alpine 3.21 container. This is the first time these jobs actually ran (all previous deploy runs had every job skipped). Two fixes:
- `setup_dagger_remote.sh`: `export_secret` was appending an extra trailing newline to values (like SSH private keys) that already end with `\n`. Now only adds one when needed.
- `ci/main.go` `Deployer`: mounts the key at a `.raw` path, strips Windows-style CRLF endings with `tr -d '\r'`, then writes the normalised key to `id_ed25519`. CRLF bytes cause "error in libcrypto" in Alpine's LibreSSL-backed openssh.
## Test plan
- [ ] Deploy run triggers after merge; all three deploy jobs complete
- [ ] Play Store verification step passes
- [ ] SSH commands in Alpine load the key without `error in libcrypto`
Closes#366
Co-authored-by: Thomas SharedInbox <sharedinbox@thomas-guettler.de>
Reviewed-on: https://codeberg.org/guettli/sharedinbox/pulls/369
## What
PR #356 (Renovate) was blocked with "Artifact file update failure" because `ci/go.sum` was out of sync with `ci/go.mod`.
**Root cause**: The `require` section listed otel log packages at v0.17.0 while `replace` directives pinned them to v0.19.0, but `go.sum` only had hashes for v0.16.0. Renovate couldn't auto-update go.sum because the Dagger module's `internal/dagger` generated package isn't in version control, so standard `go mod tidy` couldn't resolve the full dependency graph.
## Changes
- Bumps `go.opentelemetry.io/otel` + `otel/trace` + `otel/sdk` v1.43.0 → v1.44.0 (implementing PR #356's intent)
- Updates all related otel exporters and sub-packages to v1.44.0 / v0.20.0
- Aligns `replace` directives from v0.19.0 → v0.20.0 (consistent with require section)
- Also picks up `grpc` v1.79.3→v1.80.0 and `proto/otlp` v1.9.0→v1.10.0 (from `go mod tidy`)
- Adds all missing `h1:` and `/go.mod` hashes to `go.sum`
## Verification
- `go mod verify` passes
- Hashes fetched directly via `go mod download -json` from the official Go module proxy
Closes#359
Co-authored-by: Thomas SharedInbox <sharedinbox@thomas-guettler.de>
Reviewed-on: https://codeberg.org/guettli/sharedinbox/pulls/363
## Root cause
`BuildWebsite` and `PublishWebsite` in `ci/main.go` ran `hugo --minify` without setting the `HUGO_PARAMS_GITVERSION` environment variable. Hugo maps that env var to `site.Params.gitversion`, which the `website/layouts/_partials/extend_head.html` template uses to render `<meta name="x-version" content="...">` in the page `<head>`.
Without that meta tag, `website-verify.sh` (which greps for `x-version.*${VERSION}` in the live HTML) always timed out and reported failure — even though the site itself was deployed successfully.
## Fix
- Added an optional `commitHash` parameter to `BuildWebsite` and `PublishWebsite` in `ci/main.go`. When provided, it is passed to the Hugo container via `WithEnvVariable("HUGO_PARAMS_GITVERSION", commitHash)` — consistent with how `BuildLinuxRelease` and friends already inject `GIT_HASH`.
- Updated `task publish-website` in `Taskfile.yml` to compute `HASH=$(git rev-parse --short HEAD)` and forward it as `--commit-hash "$HASH"` — matching the pattern used by `task deploy-linux`.
## Verification
- `gofmt` passes on the modified `ci/main.go`.
- The logic mirrors the existing `BuildLinuxRelease` pattern that already works in CI.
Closes#360
Co-authored-by: Thomas SharedInbox <sharedinbox@thomas-guettler.de>
Reviewed-on: https://codeberg.org/guettli/sharedinbox/pulls/362
## Summary
Closes#349
Two bugs prevented `check-dagger` from failing fast when checks failed:
- **Hygiene + Layers checked sequentially** — they are cheap structural checks with no dependency on each other. Running them in parallel (`errgroup.Group`) means failures are reported sooner.
- **Spurious retries from `errgroup.WithContext`** — the backend and integration tests previously shared a derived context via `errgroup.WithContext`. When one test failed, the context was cancelled, causing the sibling test to emit `"context canceled"` in Dagger's `--progress=plain` output. The `retry_dagger` function in `Taskfile.yml` matched that string as a transient network error and re-ran the entire pipeline up to 3 times — a real test failure could take 30+ minutes to be reported instead of ~10.
**Fix in `ci/main.go`:**
- Hygiene + layers now run in parallel with `errgroup.Group`
- Backend + integration tests now use `errgroup.Group` (no shared cancel context), so a failure in one does not emit `"context canceled"` for the other
**Fix in `Taskfile.yml`:**
- Removed `context canceled` from the `retry_dagger` grep pattern; the remaining patterns (`connection reset`, `context deadline exceeded`, `connection refused`, `invalid return status code`) still cover genuine network/engine transients
## Test plan
- [ ] Confirm the Forgejo CI run completes and, when a check fails, it fails fast (no 3× retry loop in logs)
- [ ] Verify `task check-dagger` still retries on actual connection errors
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Thomas SharedInbox <sharedinbox@thomas-guettler.de>
Co-authored-by: guettli <guettli@noreply.codeberg.org>
Reviewed-on: https://codeberg.org/guettli/sharedinbox/pulls/350
The /usr/local/renovate/dist directory is owned by root.
Temporarily switch to root for the sed patch, then back to ubuntu.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Files are under dist/ not lib/, and we need to patch both
forgejo and gitea platform caches since platform=forgejo is set.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Codeberg's API times out (504) on GET /pulls?state=all&limit=100
but completes in ~9s at limit=10. Patch the compiled pr-cache.js
in the renovate:43 image before running to replace the hardcoded
20/100 page sizes with 10.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Codeberg's API times out (504) when fetching 100 closed PRs
(GET /pulls?state=all&limit=100), but succeeds with limit=20.
Renovate uses limit=100 on the first run and limit=20 on incremental
syncs. Pre-seeding the repository cache with one dummy entry tricks
Renovate into using the limit=20 incremental path from the start.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
renovate/renovate:39 did not support "forgejo" as a platform name;
v43 does. Upgrade the image and restore the correct platform name.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
renovate/renovate:39 does not recognise "forgejo" as a platform name;
the correct value is "gitea", which covers Forgejo/Gitea instances
including Codeberg.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a Renovate() Dagger function using the forgejo platform and a
.forgejo/workflows/renovate.yml workflow triggered at 06:00 UTC daily.
Uses RENOVATE_FORGEJO_TOKEN secret; no dedicated Renovate service account needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduce androidBase() and firebaseBase() helpers that wrap setup() with
the Gradle named-cache volume, mirroring the pattern already used in
BuildAndroidDebugApks(). Use these in BuildAndroidRelease(), setupKeystore(),
and BuildAndroidDebugApks() so Gradle dependencies survive Dagger
execution-cache misses instead of being re-downloaded on every source change.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
httplib2 treats 308 Resume Incomplete responses (used by Google's
resumable upload API) as redirects and raises RedirectMissingLocation
when the response lacks a Location header. Switch to
google.auth.transport.requests.AuthorizedSession + direct HTTP calls
so the upload uses the requests library, which handles 308 correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Dagger mounts the secret file with 0600 but the parent directory may
get created with world-readable permissions, causing SSH to refuse
the key with exit 255.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- agent_loop.py: create log dir with mode 0700 and enforce it on
existing dirs; open log files with mode 0600; chmod state file
to 0600 after every write. Prevents other local processes from
reading agent output (which may contain credential paths) or
tampering with the state file's pid field.
- ci/main.go (TestAndroidFirebase): replace
echo "$FIREBASE_SA_KEY" > /tmp/key.json
with bash process substitution
--key-file=<(echo "$FIREBASE_SA_KEY")
The key is now passed via a file descriptor — it never touches
disk, so it cannot be stranded by a failed gcloud auth call or
snapshotted into the Dagger layer cache.
- ci.yml / deploy.yml: add "Cleanup TLS credentials" step
(if: always()) at the end of every job that calls
setup_dagger_remote.sh. Removes /tmp/dagger-tls,
/tmp/stunnel-dagger.conf, /tmp/stunnel.pid from the self-hosted
runner after each job, so client certs do not accumulate between
job runs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
flutter pub get is pure Dart — it never invokes Gradle. The mutable
gradle-cache volume mount caused the same execution-cache instability
we just fixed for the pub cache: Dagger sees a changed volume and
cache-misses pubGetLayer() on every run.
The Gradle cache stays in Base(), which is only used for steps that
actually build Android code.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The mutable flutter-pub-cache volume made the execution cache key unstable —
pub get cache-missed every run because the volume's mutable layer changed the
snapshot hash. Removing the volume lets Dagger snapshot packages inside the
execution-cache layer, which is stable and reclaimable via dagger prune.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Only pick up issues created by guettli, guettlibot, or guettlibot2
to prevent the loop from acting on external/bot issues.
- Post an explanatory comment on the issue whenever the loop sets
State/Question (agent killed, no CI run, no push detected), so the
reason is visible without digging through cron logs. Closes#158.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Firebase CLI emits "A non-retryable error occurred." even for passing runs.
The grep -qwi 'error' triggered on this message despite gcloud exiting 0
and the result table showing Passed. The gcloud exit code, device-count,
and Passed checks are sufficient to detect real failures.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Capture gcloud auth stderr separately and fail on unexpected output;
ignore the two known informational lines ("Activated service account
credentials for: [...]" and "Updated property [core/project].") while
keeping a strict "fail if unknown stderr" check for anything else.
- Replace the narrow pattern grep (non-retryable error|infrastructure_failure|
test execution failed) with a broad whole-word case-insensitive grep for
'error', so any infrastructure or Firebase error in the output causes CI
failure.
- Verify that the number of device result rows in the result table matches
the expected device count (1), so a silent test-run failure cannot slip
through.
- Add scripts/test_firebase_check.sh with 18 unit tests for the three new
bash patterns (auth stderr filter, error-word detection, device count).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The gradle-cache volume was mounted without an owner, so the root-owned
volume caused "Permission denied" when the ci user tried to create
gradle-8.14-all.zip.lck during bundleRelease. Add Owner: "ci" to all
three WithMountedCache calls so the ci user can write to the caches.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add -qq to apt-get update/install in Dagger toolchain to suppress
verbose package-list output (hundreds of lines on cold cache)
- Wrap sdkmanager in silent-on-success pattern — only shows output
on failure, like the build_runner and flutter pub get steps
- Set debug = warning in stunnel config to suppress LOG5 (info/notice)
startup lines while keeping LOG4 (warning) and above
- Add org.gradle.welcome=never to android/gradle.properties to
suppress the "Welcome to Gradle N.NN!" banner
- Filter SKIPPED Gradle tasks, Gradle Daemon startup messages, and
gcloud support-page promo lines in run_firebase_test.sh
Errors and warnings are preserved in all cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The cirruslabs/flutter:3.41.6 image already has UID 1000 assigned to
another user, so `useradd -u 1000` exits with code 4 ("UID not unique")
and the ci user is never created. Dagger then fails to resolve `owner:
"ci"` on subsequent WithDirectory calls. Removing the explicit UID lets
useradd pick the next available one.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Create a non-root user 'ci' (UID 1000) in the Dagger toolchain container,
transfer ownership of the Flutter SDK and Android SDK to that user, and
switch to it with WithUser("ci"). Update all cache mount paths from /root/
to /home/ci/ and set Owner: "ci" on every WithDirectory call so Flutter
can write build output. Flutter emits a strong warning when run as root;
this change eliminates that warning by running the tool as a regular user.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously build_runner compiled separately for each setup() variant
(checkSrc, backendSrc, integrationSrc, etc.) since their differing
source inputs produced distinct Dagger cache keys. CheckMocks also ran
build_runner twice: once inside setup() and again explicitly — and the
second run always compared two freshly-generated outputs, so stale mocks
in the repo were never detected.
Introduce codegenBase() that runs build_runner on the minimal common
source (lib/, test/, assets/, pubspec.*) excluding committed generated
files. All setup() calls now share this single Dagger cache entry, so
build_runner compiles only once per pipeline run instead of once per
source variant.
Fix CheckMocks to start from pubGetLayer() + committed source (including
any stale *.mocks.dart), commit that state as the git baseline, then run
build_runner once. The subsequent git diff now correctly detects stale
mocks in the repository, matching the behaviour of check_mocks_fresh.sh.
Also update Graph() to reflect the new codegenBase node.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously, `gcloud firebase test android run` could exit 0 while printing
"A non-retryable error occurred." in its output. The old check
`&& echo "$out" || { exit 1; }` only caught non-zero exit codes, and the
success grep `'Passed|passed|test cases'` was too broad — "test cases" can
appear in Firebase output before the error, giving a false positive.
The fix captures gcloud's exit code explicitly via `rc=$?`, adds an explicit
error-string check for known Firebase failure phrases (non-retryable error,
infrastructure_failure, test execution failed), and tightens the success
pattern to `'Passed|passed'` only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
gcloud exits 0 even when no tests ran. Add a post-check that greps
the output for 'Passed/passed/test cases' and fails explicitly if
none are found, so 'no test case results' turns the CI red.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The default Firebase Test Lab bucket is in a Google-managed project so
project-level IAM grants have no effect on it. Use sharedinbox-ftl-results
which is in sharedinbox-496103 where the service account has storage.admin.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
WithEnvVariable(CACHE_BUSTER, time.Now()) ensures gcloud firebase test
always runs fresh rather than returning a cached result from a prior run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
'Pixel6' is not a valid Firebase Test Lab model ID.
'oriole' is the correct internal codename for Pixel 6.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
