fix: run ci/build.sh container as invoking user, not root #13

Closed

Breadway wants to merge 1 commit from fix/build-sh-user-perms into main

Breadway commented

2026-09-05 00:13:02 +08:00

Owner

Root cause

docker run in ci/build.sh had no --user flag, so cargo (and the
final cp -a step) ran as root inside the container. Because the
container's /cargo-target is a bind-adjacent copy source and
${REPO_ROOT} is bind-mounted straight from the host, the final step:

cp -a /cargo-target/. /workspace/target/

wrote onto the host checkout's target/ directory as root, leaving it
root-owned after every local run of ci/build.sh. Any later host-side
cargo command against that checkout then fails with a permission error
until someone manually chowns target/ back. This has already hit
breadmon, breadshot, breadclip, and breadpad.

Why a bare `--user` flag isn't sufficient on its own

The obvious fix — add --user "$(id -u):$(id -g)" — isn't enough by
itself, because the cargo registry/git/target caches were named docker
volumes mounted at /root/.cargo/registry, /root/.cargo/git, and
/cargo-target:

/root is 0750 root:root in the base Arch image, so a non-root,
non-root-group user can't even traverse into it — /root/.cargo/registry
is unreachable regardless of that subdirectory's own permissions.
dockerd's local volume driver initializes a brand-new named volume as
root:root 0755 and does not chown it to match --user, so even
the top-level /cargo-target mount (outside /root) has the same
problem for a fresh volume.

I verified both failure modes and the fix locally with podman (using
--userns=keep-id, which is needed under rootless podman to reproduce
plain rootful docker's 1:1 host/container uid mapping — this sandbox's
podman is rootless with subuid remapping by default, so without that flag
the test isn't representative of the real self-hosted docker runner):

Old layout + --user: touch /root/.cargo/registry/testfile →
Permission denied.
New layout + --user: writes to the registry/git/target caches and the
final cp -a into /workspace/target all succeed, and every resulting
file lands owned by the invoking host user, not root.

Fix

Move the cargo registry/git/target caches from named docker volumes
under /root/... to real host directories under
${BREAD_CI_CACHE_ROOT:-$HOME/.cache/bread-ci}, bind-mounted in
(inherits the invoking user's ownership since mkdir -p runs on the
host before the container starts).
Point CARGO_HOME at the new registry/git mount and keep
CARGO_TARGET_DIR pointed at the new target mount.
Add --user "$(id -u):$(id -g)" and -e HOME=/tmp to docker run.

No chown-based cleanup step was added — this prevents the root-owned
files from being written in the first place rather than fixing them up
after the fact.

Note for whoever runs this on the actual CI/dev hosts: this switches the
cache location away from the old bread-ci-cargo-registry /
bread-ci-cargo-git / bread-ci-<product>-target named docker volumes
entirely, so there's no stale root-owned volume content to worry about —
the new host cache dirs start empty and get populated fresh on first run
under the new user. The old named volumes are simply orphaned and can be
docker volume rm'd whenever convenient; nothing depends on them anymore.

## Root cause `docker run` in `ci/build.sh` had no `--user` flag, so `cargo` (and the final `cp -a` step) ran as root inside the container. Because the container's `/cargo-target` is a bind-adjacent copy source and `${REPO_ROOT}` is bind-mounted straight from the host, the final step: ```sh cp -a /cargo-target/. /workspace/target/ ``` wrote onto the host checkout's `target/` directory *as root*, leaving it root-owned after every local run of `ci/build.sh`. Any later host-side `cargo` command against that checkout then fails with a permission error until someone manually `chown`s `target/` back. This has already hit breadmon, breadshot, breadclip, and breadpad. ## Why a bare `--user` flag isn't sufficient on its own The obvious fix — add `--user "$(id -u):$(id -g)"` — isn't enough by itself, because the cargo registry/git/target caches were named docker volumes mounted at `/root/.cargo/registry`, `/root/.cargo/git`, and `/cargo-target`: - `/root` is `0750 root:root` in the base Arch image, so a non-root, non-root-group user can't even traverse into it — `/root/.cargo/registry` is unreachable regardless of that subdirectory's own permissions. - dockerd's `local` volume driver initializes a brand-new named volume as `root:root 0755` and does **not** chown it to match `--user`, so even the top-level `/cargo-target` mount (outside `/root`) has the same problem for a fresh volume. I verified both failure modes and the fix locally with `podman` (using `--userns=keep-id`, which is needed under rootless podman to reproduce plain rootful docker's 1:1 host/container uid mapping — this sandbox's podman is rootless with subuid remapping by default, so without that flag the test isn't representative of the real self-hosted docker runner): - Old layout + `--user`: `touch /root/.cargo/registry/testfile` → `Permission denied`. - New layout + `--user`: writes to the registry/git/target caches and the final `cp -a` into `/workspace/target` all succeed, and every resulting file lands owned by the invoking host user, not root. ## Fix - Move the cargo registry/git/target caches from named docker volumes under `/root/...` to real host directories under `${BREAD_CI_CACHE_ROOT:-$HOME/.cache/bread-ci}`, bind-mounted in (inherits the invoking user's ownership since `mkdir -p` runs on the host before the container starts). - Point `CARGO_HOME` at the new registry/git mount and keep `CARGO_TARGET_DIR` pointed at the new target mount. - Add `--user "$(id -u):$(id -g)"` and `-e HOME=/tmp` to `docker run`. No chown-based cleanup step was added — this prevents the root-owned files from being written in the first place rather than fixing them up after the fact. Note for whoever runs this on the actual CI/dev hosts: this switches the cache location away from the old `bread-ci-cargo-registry` / `bread-ci-cargo-git` / `bread-ci-<product>-target` named docker volumes entirely, so there's no stale root-owned volume content to worry about — the new host cache dirs start empty and get populated fresh on first run under the new user. The old named volumes are simply orphaned and can be `docker volume rm`'d whenever convenient; nothing depends on them anymore.

Breadway added 1 commit

2026-09-05 00:13:07 +08:00

fix: run ci/build.sh container as invoking user, not root

check / check (push) Successful in 3m56s

Details

3ad3836657

docker run had no --user flag, so cargo built as root inside the
container. The final `cp -a /cargo-target/. /workspace/target/` step
then wrote onto the host bind-mounted repo as root, leaving
target/ root-owned on the host after every local run — breaking any
later host-side cargo command until someone manually chowned it back.
This has bitten breadmon, breadshot, breadclip, and breadpad before.

Adding a bare --user flag isn't enough on its own: the cargo
registry/git/target caches were named docker volumes mounted under
/root/.cargo/... and /cargo-target. /root is 0750 root:root in the
base image, so a non-root user can't even traverse into it, and
dockerd initializes fresh named volumes as root:root 0755 regardless
of --user, so the top-level /cargo-target mount has the same problem.

Switch those caches to host directories under
${BREAD_CI_CACHE_ROOT:-$HOME/.cache/bread-ci} (real bind mounts, not
named volumes), point CARGO_HOME/CARGO_TARGET_DIR at them, and add
--user "$(id -u):$(id -g)". Verified the failure mode and the fix
with podman --userns=keep-id locally (rootless podman needs that flag
to reproduce plain rootful docker's 1:1 uid mapping): the old
/root/.cargo mount rejects writes even with --user set, while the new
bind-mount layout lets the container write everywhere it needs to and
leaves every resulting file owned by the invoking user on the host.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013r8iu6nnSMfZLirp82jaNn

Breadway closed this pull request

2026-09-05 02:28:23 +08:00

Breadway referenced this pull request

2026-09-28 21:57:59 +08:00

Dev track builds against dependency main; releases stay on locked tag pins #38

Breadway referenced this pull request

2026-09-29 08:34:38 +08:00

Dev track builds against dependency main; releases stay on locked tag pins #38

Breadway referenced this pull request

2026-10-02 17:43:23 +08:00

Track remaining RC workflow wildcard cleanup copies #56

Breadway referenced this pull request