guettli/sharedinbox
1
0
Fork 0
Code Issues 21 Pull Requests 6 Actions Packages Projects Releases Wiki Activity

Plan: Exclude chaos_monkey_test.dart from regular CI (TestBackend) #505

New Issue
Closed
opened 2026-06-06 19:34:19 +00:00 by guettlibot · 1 comment
guettlibot commented 2026-06-06 19:34:19 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Problem

TestBackend in ci/main.go:542 runs flutter test --concurrency=1 test/backend which includes all files in test/backend/, including chaos_monkey_test.dart. This means the chaos monkey test runs on every push and pull_request via the regular check-dagger step — even though it is slow (30 random IMAP/SMTP rounds) and inherently flaky.

A dedicated chaos-monkey.yml workflow already runs the ChaosMonkeyBackend Dagger function on a daily schedule (0 3 * * *). The issue is that TestBackend unintentionally also picks it up.

Confirmed by CI run #1861 failing in the chaos monkey step during a normal PR check.

Plan

  1. Add @Tags(['nightly']) at the top of test/backend/chaos_monkey_test.dart (inside main(), or as a file-level tag).
  2. In ci/main.go TestBackend (line 542), add --exclude-tags=nightly to the flutter test command.
  3. In ci/main.go ChaosMonkeyBackend (line 573), add --tags=nightly to ensure it still exercises the test.
  4. Run task check-dagger locally to confirm chaos monkey no longer runs in regular CI.
  5. Run task chaos-monkey-backend to confirm the nightly path still works.

Outcome

Chaos monkey runs only via the nightly chaos-monkey.yml schedule, not on every PR push.

## Problem `TestBackend` in `ci/main.go:542` runs `flutter test --concurrency=1 test/backend` which includes **all** files in `test/backend/`, including `chaos_monkey_test.dart`. This means the chaos monkey test runs on every push and pull_request via the regular `check-dagger` step — even though it is slow (30 random IMAP/SMTP rounds) and inherently flaky. A dedicated `chaos-monkey.yml` workflow already runs the `ChaosMonkeyBackend` Dagger function on a daily schedule (`0 3 * * *`). The issue is that `TestBackend` unintentionally also picks it up. Confirmed by CI run [#1861](https://codeberg.org/guettli/sharedinbox/actions/runs/1861) failing in the chaos monkey step during a normal PR check. ## Plan 1. Add `@Tags(['nightly'])` at the top of `test/backend/chaos_monkey_test.dart` (inside `main()`, or as a file-level tag). 2. In `ci/main.go` `TestBackend` (line 542), add `--exclude-tags=nightly` to the `flutter test` command. 3. In `ci/main.go` `ChaosMonkeyBackend` (line 573), add `--tags=nightly` to ensure it still exercises the test. 4. Run `task check-dagger` locally to confirm chaos monkey no longer runs in regular CI. 5. Run `task chaos-monkey-backend` to confirm the nightly path still works. ## Outcome Chaos monkey runs only via the nightly `chaos-monkey.yml` schedule, not on every PR push.
guettlibot commented 2026-06-07 00:47:28 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

The PR has been created at https://codeberg.org/guettli/sharedinbox/pulls/518.

Here's what I did to fix issue #505:

Changes made:

  1. test/backend/chaos_monkey_test.dart — Added a @Tags(['nightly']) library-level annotation. This marks every test in the file with the nightly tag, which Flutter's test runner uses for tag-based filtering.

  2. ci/main.go — TestBackend — Added --exclude-tags=nightly to the flutter test command. This ensures that when TestBackend runs all files in test/backend/, it skips the chaos monkey test.

  3. ci/main.go — ChaosMonkeyBackend — Added --tags=nightly to its flutter test command. This ensures the nightly chaos-monkey.yml workflow still runs the chaos monkey test as intended.

The result: chaos monkey only runs via the dedicated nightly schedule, not on every PR push.

The PR has been created at https://codeberg.org/guettli/sharedinbox/pulls/518. Here's what I did to fix issue #505: **Changes made:** 1. **`test/backend/chaos_monkey_test.dart`** — Added a `@Tags(['nightly'])` library-level annotation. This marks every test in the file with the `nightly` tag, which Flutter's test runner uses for tag-based filtering. 2. **`ci/main.go` — `TestBackend`** — Added `--exclude-tags=nightly` to the `flutter test` command. This ensures that when `TestBackend` runs all files in `test/backend/`, it skips the chaos monkey test. 3. **`ci/main.go` — `ChaosMonkeyBackend`** — Added `--tags=nightly` to its `flutter test` command. This ensures the nightly `chaos-monkey.yml` workflow still runs the chaos monkey test as intended. The result: chaos monkey only runs via the dedicated nightly schedule, not on every PR push.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
NeedSupervisor
Issue escalated to a human supervisor; agentloop will skip it until cleared.
 State/InProgress
State/Later
State/Planned
automerge
Eligible for automatic merge by CI
 ci-failure
Issue opened by agentloop to track a failing CI workflow; used for deduplication.
 do-not-merge
Plan PR — review only, do not merge.
 loop/code
Add to run the built-in "code" prompt; override at prompts/code.md.
 loop/code-ci-pending
Prompt "code" finished; waiting for the PR's CI to pass before advancing.
 loop/code-done
Prompt "code" finished successfully.
 loop/code-in-process
Agent for the "code" prompt is currently running on this issue.
 loop/merge
Managed by agentloop
 loop/merge-done
Managed by agentloop
 loop/merge-in-process
Managed by agentloop
 loop/plan
Add to run the built-in "plan" prompt; override at prompts/plan.md.
 loop/plan-done
Prompt "plan" finished successfully.
 loop/plan-in-process
Agent for the "plan" prompt is currently running on this issue.
No labels loop/code-done
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
guettlibot guettli
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: guettli/sharedinbox#505