guettli/sharedinbox
1
0
Fork 0
Code Issues 21 Pull Requests 6 Actions Packages Projects Releases Wiki Activity

Visualize CI run in SVG/HTML/Mermaid/Dot #126

New Issue
Closed
opened 2026-05-19 06:36:46 +00:00 by guettli · 1 comment
guettli commented 2026-05-19 06:36:46 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

do plan

do plan
guettlibot commented 2026-05-21 05:10:14 +00:00 (Migrated from codeberg.org)
Copy Link
Copy Source

Plan: CI Run Timeline Visualization

Summary

The OTEL receiver (ci/otelrecv.py) already captures start_ns and end_ns for every span. We can use this data to render a self-contained HTML Gantt chart and upload it to the website.

Data available from the existing OTEL receiver

otelrecv.py already decodes start_time_unix_nano (field 7) and end_time_unix_nano (field 8) from each span. The _span() function computes dur from them but throws away the absolute timestamps. We need to keep them.

Changes needed

1. Extend otelrecv.py — keep absolute timestamps

Change _span() to return start_s and end_s alongside dur:

return {
    "name": name,
    "start_s": start_ns / 1e9,   # ← add
    "end_s":   end_ns / 1e9,     # ← add
    "dur":     max(0.0, (end_ns - start_ns) / 1e9),
    "cached":  cached,
}

2. Add scripts/ci_timeline.py — generate the HTML

Reads the span list, assigns parallel lanes via a greedy overlap algorithm (same idea as interval scheduling), and writes a self-contained HTML file with inline CSS. No JavaScript, no external resources.

Gantt layout algorithm:

  1. Sort spans by start_s.
  2. Assign each span to the first lane whose last span has already ended.
  3. The final width = (end_s - start_s) / total_duration * 100%.

HTML structure (top-to-bottom timeline, left-aligned bars):

|--- 0s ----------------------------------------- Ns ---|
[lane 0] ██████ dart format    [0.3s]
         █████████████████ dart analyze  [12s LIVE]
[lane 1] ███████████ flutter pub get    [1.3s CACHED]

Each span is a <div> with position: absolute; left: X%; width: W% inside a fixed-height row. Colors: green = CACHED, steelblue = LIVE.

3. Extend otelrecv.py — write JSON on shutdown

After _report(), write /tmp/otel-spans.json (or a path passed via --json-file):

with open(args.json_file, "w") as f:
    json.dump(rows, f)

4. Extend check-dagger task in Taskfile.yml

After dagger completes and the report is printed, run ci_timeline.py:

HASH=$(git rev-parse --short HEAD)
python3 scripts/ci_timeline.py \
    --spans-json /tmp/otel-spans.json \
    --output /tmp/ci-timeline-${HASH}.html

if [ -n "${SSH_PRIVATE_KEY:-}" ] && [ -n "${SSH_HOST:-}" ]; then
    REMOTE_PATH="public_html/ci-timeline/${HASH}.html"
    # write key file, scp, remove key
    echo "${SSH_PRIVATE_KEY}" > /tmp/ci-key; chmod 600 /tmp/ci-key
    scp -o StrictHostKeyChecking=no -i /tmp/ci-key \
        /tmp/ci-timeline-${HASH}.html \
        ${SSH_USER}@${SSH_HOST}:${REMOTE_PATH}
    rm -f /tmp/ci-key
    echo "CI Timeline: ${WEBSITE_URL}ci-timeline/${HASH}.html"
fi

The WEBSITE_URL env var (already set to https://sharedinbox.de/) is used to print the final URL.

Environment variables

Variable Source Usage
SSH_PRIVATE_KEY existing CI secret Upload key
SSH_USER existing CI secret Upload target user
SSH_HOST existing CI secret Upload target host
WEBSITE_URL new (already created) URL prefix for printed link

No new secrets are needed — the same credentials used for Linux/APK deploy.

Questions / decisions for you

  1. Upload on every run or only on main? The check-dagger task runs on both PRs and main. Should the upload be gated by github.ref == 'refs/heads/main' (already done in the CI YAML), or should PR runs also upload (with a per-commit path)?

--> Upload for every commit. Use structure: short-git-commit-hash/ci-run-id/

  1. Retention policy? The files accumulate at ci-timeline/*.html. Should they be cleaned up after N days, or left forever (they are tiny ~50 KB files)?

--> Different topic. Do not clean up. Just create.

  1. Confirm WEBSITE_SSH_* = existing SSH_USER / SSH_HOST / SSH_PRIVATE_KEY? The issue mentions WEBSITE_SSH_... vars; I'll use the existing ones unless you've set up separate credentials.

--> Yes, same credentials

## Plan: CI Run Timeline Visualization ### Summary The OTEL receiver (`ci/otelrecv.py`) already captures `start_ns` and `end_ns` for every span. We can use this data to render a self-contained HTML Gantt chart and upload it to the website. ### Data available from the existing OTEL receiver `otelrecv.py` already decodes `start_time_unix_nano` (field 7) and `end_time_unix_nano` (field 8) from each span. The `_span()` function computes `dur` from them but throws away the absolute timestamps. We need to keep them. ### Changes needed #### 1. Extend `otelrecv.py` — keep absolute timestamps Change `_span()` to return `start_s` and `end_s` alongside `dur`: ```python return { "name": name, "start_s": start_ns / 1e9, # ← add "end_s": end_ns / 1e9, # ← add "dur": max(0.0, (end_ns - start_ns) / 1e9), "cached": cached, } ``` #### 2. Add `scripts/ci_timeline.py` — generate the HTML Reads the span list, assigns parallel lanes via a greedy overlap algorithm (same idea as interval scheduling), and writes a self-contained HTML file with inline CSS. No JavaScript, no external resources. **Gantt layout algorithm:** 1. Sort spans by `start_s`. 2. Assign each span to the first lane whose last span has already ended. 3. The final width = `(end_s - start_s) / total_duration * 100%`. **HTML structure (top-to-bottom timeline, left-aligned bars):** ``` |--- 0s ----------------------------------------- Ns ---| [lane 0] ██████ dart format [0.3s] █████████████████ dart analyze [12s LIVE] [lane 1] ███████████ flutter pub get [1.3s CACHED] ``` Each span is a `<div>` with `position: absolute; left: X%; width: W%` inside a fixed-height row. Colors: green = CACHED, steelblue = LIVE. #### 3. Extend `otelrecv.py` — write JSON on shutdown After `_report()`, write `/tmp/otel-spans.json` (or a path passed via `--json-file`): ```python with open(args.json_file, "w") as f: json.dump(rows, f) ``` #### 4. Extend `check-dagger` task in `Taskfile.yml` After dagger completes and the report is printed, run `ci_timeline.py`: ```bash HASH=$(git rev-parse --short HEAD) python3 scripts/ci_timeline.py \ --spans-json /tmp/otel-spans.json \ --output /tmp/ci-timeline-${HASH}.html if [ -n "${SSH_PRIVATE_KEY:-}" ] && [ -n "${SSH_HOST:-}" ]; then REMOTE_PATH="public_html/ci-timeline/${HASH}.html" # write key file, scp, remove key echo "${SSH_PRIVATE_KEY}" > /tmp/ci-key; chmod 600 /tmp/ci-key scp -o StrictHostKeyChecking=no -i /tmp/ci-key \ /tmp/ci-timeline-${HASH}.html \ ${SSH_USER}@${SSH_HOST}:${REMOTE_PATH} rm -f /tmp/ci-key echo "CI Timeline: ${WEBSITE_URL}ci-timeline/${HASH}.html" fi ``` The `WEBSITE_URL` env var (already set to `https://sharedinbox.de/`) is used to print the final URL. --- ### Environment variables | Variable | Source | Usage | |----------|--------|-------| | `SSH_PRIVATE_KEY` | existing CI secret | Upload key | | `SSH_USER` | existing CI secret | Upload target user | | `SSH_HOST` | existing CI secret | Upload target host | | `WEBSITE_URL` | new (already created) | URL prefix for printed link | No new secrets are needed — the same credentials used for Linux/APK deploy. --- ### Questions / decisions for you 1. **Upload on every run or only on `main`?** The `check-dagger` task runs on both PRs and `main`. Should the upload be gated by `github.ref == 'refs/heads/main'` (already done in the CI YAML), or should PR runs also upload (with a per-commit path)? --> Upload for every commit. Use structure: short-git-commit-hash/ci-run-id/ 2. **Retention policy?** The files accumulate at `ci-timeline/*.html`. Should they be cleaned up after N days, or left forever (they are tiny ~50 KB files)? --> Different topic. Do not clean up. Just create. 3. **Confirm `WEBSITE_SSH_*` = existing `SSH_USER` / `SSH_HOST` / `SSH_PRIVATE_KEY`?** The issue mentions `WEBSITE_SSH_...` vars; I'll use the existing ones unless you've set up separate credentials. --> Yes, same credentials
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
NeedSupervisor
Issue escalated to a human supervisor; agentloop will skip it until cleared.
 State/InProgress
State/Later
State/Planned
automerge
Eligible for automatic merge by CI
 ci-failure
Issue opened by agentloop to track a failing CI workflow; used for deduplication.
 do-not-merge
Plan PR — review only, do not merge.
 loop/code
Add to run the built-in "code" prompt; override at prompts/code.md.
 loop/code-ci-pending
Prompt "code" finished; waiting for the PR's CI to pass before advancing.
 loop/code-done
Prompt "code" finished successfully.
 loop/code-in-process
Agent for the "code" prompt is currently running on this issue.
 loop/merge
Managed by agentloop
 loop/merge-done
Managed by agentloop
 loop/merge-in-process
Managed by agentloop
 loop/plan
Add to run the built-in "plan" prompt; override at prompts/plan.md.
 loop/plan-done
Prompt "plan" finished successfully.
 loop/plan-in-process
Agent for the "plan" prompt is currently running on this issue.
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
guettlibot guettli
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: guettli/sharedinbox#126