guettli/sharedinbox
1
0
Fork 0
Code Issues 21 Pull Requests 6 Actions Packages Projects Releases Wiki Activity

Sops in ci, CI redaction.... #434

New Issue
Closed
opened 2026-06-05 04:45:23 +00:00 by guettli · 2 comments
guettli commented 2026-06-05 04:45:23 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

CI does not know that these strings are secrets. Redaction does not work.

CI does not know that these strings are secrets. Redaction does not work.
guettlibot commented 2026-06-05 19:25:03 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

I now have everything I need. Here is the implementation plan:

Implementation Plan: Fix CI secret redaction in setup_dagger_remote.sh

Problem

scripts/setup_dagger_remote.sh decrypts secrets.enc.yaml via SOPS and writes the resulting values into $GITHUB_ENV so subsequent workflow steps can use them. However, the Forgejo/GitHub Actions runner only redacts values it has been explicitly told to treat as secrets — either via ${{ secrets.* }} substitutions, or via the ::add-mask:: workflow command. Because the script never issues ::add-mask:: commands, any step that echoes or logs one of the exported values (e.g., a dagger call progress line, a stray set -x, or a debug printout) will expose the secret in plain text in the CI log.

Root cause

The export_secret() function in scripts/setup_dagger_remote.sh writes values to $GITHUB_ENV but never emits ::add-mask::. The runner therefore has no record of which strings to redact.

Fix — one file: scripts/setup_dagger_remote.sh

1. Add per-line masking inside export_secret()

Immediately after the value=$(jq -r ...) line, emit one ::add-mask:: per non-empty line of the value. Multiline secrets such as SSH private keys must be masked line-by-line because the ::add-mask:: command only covers what is on that one output line.

export_secret() {
    local name="$1"
    local value
    value=$(jq -r --arg k "$name" '.[$k] // empty' "$SECRETS_JSON")
    # Register each non-empty line for log redaction in the Actions runner.
    if [ -n "$value" ] && [ -n "${GITHUB_ENV:-}" ]; then
        while IFS= read -r line; do
            [ -n "$line" ] && printf '::add-mask::%s\n' "$line"
        done <<< "$value"
    fi
    if [ -n "${GITHUB_ENV:-}" ]; then
        {
            printf '%s<<__EOF__\n' "$name"
            printf '%s' "$value"
            [ "${value%$'\n'}" = "$value" ] && printf '\n'
            printf '__EOF__\n'
        } >> "$GITHUB_ENV"
    fi
    printf '[secrets] exported %s (%d chars)\n' "$name" "${#value}"
}

2. Mask the two inline variables DAGGER_SSH_KEY and DAGGER_ENGINE_HOST

These are extracted from the JSON but never passed through export_secret(). Add masking directly after they are assigned:

DAGGER_SSH_KEY=$(jq -r '.DAGGER_SSH_KEY' "$SECRETS_JSON")
DAGGER_ENGINE_HOST=$(jq -r '.DAGGER_ENGINE_HOST' "$SECRETS_JSON")

# Mask inline secrets so the runner redacts them from logs.
printf '::add-mask::%s\n' "$DAGGER_ENGINE_HOST"
while IFS= read -r line; do
    [ -n "$line" ] && printf '::add-mask::%s\n' "$line"
done <<< "$DAGGER_SSH_KEY"

What does NOT change

  • No Dart/Flutter code, no Drift schema, no pubspec.yaml.
  • No new files, no Taskfile changes, no CI workflow YAML changes.
  • The existing [secrets] exported NAME (N chars) log line is safe and stays.
  • The ::add-mask:: output is consumed by the runner and does not appear in the visible log.

Testing guidance

After the change, a CI run that encounters any logged value (e.g., from set -x in a sub-script, or a Dagger progress line that echoes an env-var) should show *** instead of the raw secret. Verify by triggering the deploy.yml workflow manually (workflow_dispatch) and checking that none of the exported secret values appear in plain text in the log output for the steps that follow "Setup Dagger Remote Engine".

I now have everything I need. Here is the implementation plan: --- ## Implementation Plan: Fix CI secret redaction in `setup_dagger_remote.sh` ### Problem `scripts/setup_dagger_remote.sh` decrypts `secrets.enc.yaml` via SOPS and writes the resulting values into `$GITHUB_ENV` so subsequent workflow steps can use them. However, the Forgejo/GitHub Actions runner only redacts values it has been explicitly told to treat as secrets — either via `${{ secrets.* }}` substitutions, or via the `::add-mask::` workflow command. Because the script never issues `::add-mask::` commands, any step that echoes or logs one of the exported values (e.g., a `dagger call` progress line, a stray `set -x`, or a debug printout) will expose the secret in plain text in the CI log. ### Root cause The `export_secret()` function in `scripts/setup_dagger_remote.sh` writes values to `$GITHUB_ENV` but never emits `::add-mask::`. The runner therefore has no record of which strings to redact. ### Fix — one file: `scripts/setup_dagger_remote.sh` **1. Add per-line masking inside `export_secret()`** Immediately after the `value=$(jq -r ...)` line, emit one `::add-mask::` per non-empty line of the value. Multiline secrets such as SSH private keys must be masked line-by-line because the `::add-mask::` command only covers what is on that one output line. ```bash export_secret() { local name="$1" local value value=$(jq -r --arg k "$name" '.[$k] // empty' "$SECRETS_JSON") # Register each non-empty line for log redaction in the Actions runner. if [ -n "$value" ] && [ -n "${GITHUB_ENV:-}" ]; then while IFS= read -r line; do [ -n "$line" ] && printf '::add-mask::%s\n' "$line" done <<< "$value" fi if [ -n "${GITHUB_ENV:-}" ]; then { printf '%s<<__EOF__\n' "$name" printf '%s' "$value" [ "${value%$'\n'}" = "$value" ] && printf '\n' printf '__EOF__\n' } >> "$GITHUB_ENV" fi printf '[secrets] exported %s (%d chars)\n' "$name" "${#value}" } ``` **2. Mask the two inline variables `DAGGER_SSH_KEY` and `DAGGER_ENGINE_HOST`** These are extracted from the JSON but never passed through `export_secret()`. Add masking directly after they are assigned: ```bash DAGGER_SSH_KEY=$(jq -r '.DAGGER_SSH_KEY' "$SECRETS_JSON") DAGGER_ENGINE_HOST=$(jq -r '.DAGGER_ENGINE_HOST' "$SECRETS_JSON") # Mask inline secrets so the runner redacts them from logs. printf '::add-mask::%s\n' "$DAGGER_ENGINE_HOST" while IFS= read -r line; do [ -n "$line" ] && printf '::add-mask::%s\n' "$line" done <<< "$DAGGER_SSH_KEY" ``` ### What does NOT change - No Dart/Flutter code, no Drift schema, no `pubspec.yaml`. - No new files, no Taskfile changes, no CI workflow YAML changes. - The existing `[secrets] exported NAME (N chars)` log line is safe and stays. - The `::add-mask::` output is consumed by the runner and does not appear in the visible log. ### Testing guidance After the change, a CI run that encounters any logged value (e.g., from `set -x` in a sub-script, or a Dagger progress line that echoes an env-var) should show `***` instead of the raw secret. Verify by triggering the `deploy.yml` workflow manually (`workflow_dispatch`) and checking that none of the exported secret values appear in plain text in the log output for the steps that follow "Setup Dagger Remote Engine".
guettlibot commented 2026-06-05 21:45:03 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Summary

Fixed CI secret redaction in scripts/setup_dagger_remote.sh by adding ::add-mask:: workflow commands.

Root cause: The Forgejo/GitHub Actions runner only redacts values it's been explicitly told about. Secrets decrypted from secrets.enc.yaml and written to $GITHUB_ENV were never registered for masking, so they could appear in plain text in CI log output.

Fix: Added ::add-mask:: calls in two places:

  1. Inside export_secret() — each non-empty line of every exported secret is registered before being written to $GITHUB_ENV. Multiline values (SSH keys, JSON service account files) are masked line-by-line since ::add-mask:: covers one line at a time.
  2. For the inline variables DAGGER_ENGINE_HOST and DAGGER_SSH_KEY — these were extracted directly from the JSON and used without going through export_secret(), so they needed their own masking calls.

Pull request: https://codeberg.org/guettli/sharedinbox/pulls/460

## Summary Fixed CI secret redaction in `scripts/setup_dagger_remote.sh` by adding `::add-mask::` workflow commands. **Root cause:** The Forgejo/GitHub Actions runner only redacts values it's been explicitly told about. Secrets decrypted from `secrets.enc.yaml` and written to `$GITHUB_ENV` were never registered for masking, so they could appear in plain text in CI log output. **Fix:** Added `::add-mask::` calls in two places: 1. **Inside `export_secret()`** — each non-empty line of every exported secret is registered before being written to `$GITHUB_ENV`. Multiline values (SSH keys, JSON service account files) are masked line-by-line since `::add-mask::` covers one line at a time. 2. **For the inline variables** `DAGGER_ENGINE_HOST` and `DAGGER_SSH_KEY` — these were extracted directly from the JSON and used without going through `export_secret()`, so they needed their own masking calls. Pull request: https://codeberg.org/guettli/sharedinbox/pulls/460
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
NeedSupervisor
Issue escalated to a human supervisor; agentloop will skip it until cleared.
 State/InProgress
State/Later
State/Planned
automerge
Eligible for automatic merge by CI
 ci-failure
Issue opened by agentloop to track a failing CI workflow; used for deduplication.
 do-not-merge
Plan PR — review only, do not merge.
 loop/code
Add to run the built-in "code" prompt; override at prompts/code.md.
 loop/code-ci-pending
Prompt "code" finished; waiting for the PR's CI to pass before advancing.
 loop/code-done
Prompt "code" finished successfully.
 loop/code-in-process
Agent for the "code" prompt is currently running on this issue.
 loop/merge
Managed by agentloop
 loop/merge-done
Managed by agentloop
 loop/merge-in-process
Managed by agentloop
 loop/plan
Add to run the built-in "plan" prompt; override at prompts/plan.md.
 loop/plan-done
Prompt "plan" finished successfully.
 loop/plan-in-process
Agent for the "plan" prompt is currently running on this issue.
No labels loop/code-done
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
guettlibot guettli
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: guettli/sharedinbox#434