Merge pull request 'ISO: flip [breadway] to the signed dl.breadway.dev/arch repo' (#5) from fix/breadway-repo-signing into main

Reviewed-on: #5
This commit is contained in:
Breadway 2026-08-31 19:18:50 +08:00
commit 87887f09af
9 changed files with 137 additions and 67 deletions

View file

@ -16,7 +16,7 @@ taken as current:
| Work on `dev`; origin = GitHub | Single-trunk `main`; `stable` is a CI marker. `origin` = Forgejo, `github` = GitHub. |
| `[breadway]` provides bakery/breadbar/bos-settings | `[breadway]` is breadlock + AUR republishes. Desktop apps are bakery. **Not shipped:** breadcast, breadarr. |
| NVIDIA / A/B / Secure Boot / LUKS2 | NVIDIA proprietary is **unsupported**. A/B root swapping is **not implemented**. Secure Boot is **Setup Mode only** (self-signed `sbctl`). Disk encryption is **LUKS1** because GRUB cannot unlock LUKS2+Argon2id. |
| `SigLevel = Required` on `[breadway]` | **No.** Forgejo's Arch registry has no pacman-compatible db signatures. `SigLevel = Never` is TLS only; flipping Required without a signed db breaks installs. `KEYS.asc` signs ISO SHA256SUMS, not the pacman repo. |
| `SigLevel = Required` on `[breadway]` | **Yes, as of the signed repo.** `[breadway]` points at `https://dl.breadway.dev/arch` where `scripts/ci-publish-signed-repo.sh` detach-signs every `.pkg.tar.zst` and the db with the BOS release key (`56203B86…`, `KEYS.asc`). That key is trusted in the pacman keyring at build time, on the live medium, and on the installed target. |
---

View file

@ -174,10 +174,10 @@ dedicated release-signing key (not reused from anything else):
5620 3B86 A110 695A E7F3 1093 4AF3 323D 678E B5E2
```
The public half is committed at [`KEYS.asc`](KEYS.asc). That key signs
**ISO checksums only** — it does not sign the `[breadway]` pacman repo
(Forgejo's Arch registry has no pacman-compatible db signatures; that
section stays `SigLevel = Never` until a signed repo exists — see
The public half is committed at [`KEYS.asc`](KEYS.asc). The same key signs
the ISO checksums **and** the `[breadway]` pacman repo — every package and
the db at `https://dl.breadway.dev/arch` carry a `.sig` from it, and that
section is `SigLevel = Required` (see
[docs/signed-repo.md](docs/signed-repo.md)). To verify a download:
```sh
@ -381,8 +381,9 @@ until `dl.breadway.dev/arch` exists).
- **Snapshots assume btrfs**: the snapper/grub-btrfs tooling expects the default
btrfs subvolume layout the installer creates. Recovery is the GRUB
snapshots submenu, not `snapper rollback` — [docs/hardware.md](docs/hardware.md).
- **`[breadway]` signatures**: `SigLevel = Never` until a signed repo is
stood up at `dl.breadway.dev/arch`. See [docs/signed-repo.md](docs/signed-repo.md).
- **`[breadway]` signatures**: `SigLevel = Required` — the signed repo at
`dl.breadway.dev/arch` is live (db + every package `.sig`ned with the BOS
release key). See [docs/signed-repo.md](docs/signed-repo.md).
## Recovery

View file

@ -25,14 +25,19 @@ OUT="${OUT:-$REPO/out}"
STAGE=/tmp/bos-iso-stage
rm -rf "$STAGE" && cp -a "$REPO/iso" "$STAGE"
# Rewrite the [breadway] pacman repo URL to the fastest reachable address.
# CI_BUILD=1 — container runs on hestia with --network=host; localhost:3002 is direct
# default — building on hermes; git.breadway.dev is flaky from there, use Tailscale
# Only ever rewrites the staged copy, never the committed pacman.conf.
if [ "${CI_BUILD:-0}" = "1" ]; then
sed -i 's#https://git.breadway.dev/api/packages/Breadway/arch/os#http://localhost:3002/api/packages/Breadway/arch/os#' "$STAGE/pacman.conf"
else
sed -i 's#https://git.breadway.dev/api/packages/Breadway/arch/os#http://100.66.238.26:3002/api/packages/Breadway/arch/os#' "$STAGE/pacman.conf"
# [breadway] now points at the signed public repo https://dl.breadway.dev/arch
# (SigLevel = Required) — no Forgejo-registry URL rewrite needed anymore.
#
# Trust the [breadway] repo key in *this* build host's pacman keyring so
# `pacstrap` can verify [breadway] packages while assembling the airootfs.
# The same key is baked into the image at etc/pacman.d/breadway-repo.asc and
# re-trusted on the live medium / installed target (calamares/post-install.sh).
BREADWAY_KEY_FPR="56203B86A110695AE7F310934AF3323D678EB5E2"
BREADWAY_KEY_SRC="$REPO/iso/airootfs/etc/pacman.d/breadway-repo.asc"
if ! pacman-key --list-keys "$BREADWAY_KEY_FPR" &>/dev/null; then
echo "=== trusting [breadway] repo key ($BREADWAY_KEY_FPR) in the host pacman keyring ==="
pacman-key --add "$BREADWAY_KEY_SRC"
pacman-key --lsign-key "$BREADWAY_KEY_FPR"
fi
if [ "${FAST_BUILD:-0}" = "1" ]; then

View file

@ -1,18 +1,20 @@
# Signed `[breadway]` repo
Today the ISO's `[Breadway.os.git.breadway.dev]` section is
`SigLevel = Never`. That is TLS-only integrity: packages come from Forgejo's
Arch registry, which does **not** serve pacman-compatible database
signatures. `KEYS.asc` signs **ISO `SHA256SUMS`** and, once published, the
`dl.breadway.dev/arch` database. It is **not** imported as a pacman repo key
on the ISO yet. Do not flip `SigLevel` to `Required` on that section until
a signed repo exists and has been verified; Required without signatures
breaks the ISO and every installed system.
**Status: live.** The ISO's `[breadway]` section is `SigLevel = Required`
and points at `https://dl.breadway.dev/arch/$arch`, where every
`.pkg.tar.zst` and the db carry a detached `.sig` from the BOS release key
(`56203B86…`, `KEYS.asc`, `releases@breadway.dev`). That key is trusted in
the pacman keyring at build time (`build-local.sh`), on the live medium
(`iso/airootfs/root/customize_airootfs.sh`), and on the installed target
(`iso/airootfs/etc/calamares/post-install.sh`).
The signed repo belongs at `https://dl.breadway.dev/arch`, not on Forgejo's
registry. Forgejo publishing stays as it is (`package.yml` / packaging
workflows PUT unsigned `.pkg.tar.zst` so existing Never installs keep
working).
Forgejo publishing is unchanged: `package.yml` / packaging workflows still
PUT unsigned `.pkg.tar.zst` to Forgejo's Arch registry. The signed tree at
`dl.breadway.dev/arch` is rebuilt from that registry by
`.forgejo/workflows/signed-repo.yml` + `scripts/ci-publish-signed-repo.sh`.
The rest of this doc is the original stand-up / verification procedure,
kept for reference and for re-verifying after key rotation.
## Stand up `dl.breadway.dev/arch`
@ -139,15 +141,29 @@ stays; Never installs keep working. The signed tree is rebuilt by the bos
workflow above (registry fetch + sign + `repo-add -s`), not by writing
`/srv` from breadlock's container.
## After the signed repo exists
## The ISO flip (done)
Only after `https://dl.breadway.dev/arch/x86_64/breadway.db.sig` HEADs 200
and the verify commands above succeed:
All three steps have landed:
1. Import `KEYS.asc` into the ISO keyring (`pacman-key --add` + `--lsign-key`).
2. Point `[breadway]` `Server` at `https://dl.breadway.dev/arch/$arch`.
3. Only then flip that section to `SigLevel = Required`.
1. **Key trusted.** The public key is committed at
`iso/airootfs/etc/pacman.d/breadway-repo.asc`. `build-local.sh`
`pacman-key --add` + `--lsign-key`s it into the build host keyring;
`customize_airootfs.sh` does the same in the airootfs;
`calamares/post-install.sh` re-does it in the target chroot.
2. **`Server`** in `iso/pacman.conf` and `iso/airootfs/etc/pacman.conf`
points at `https://dl.breadway.dev/arch/$arch`, section renamed to
`[breadway]` (matching `breadway.db`).
3. **`SigLevel = Required`** on that section.
Do not do those three steps against Forgejo's registry. See
`iso/pacman.conf` and `iso/airootfs/etc/pacman.conf`. This tree does
**not** change either file.
### Re-verify after any build
In a VM booted from a fresh ISO:
```sh
sudo pacman -Sy # must fetch breadway.db + .sig, no signature error
sudo pacman -Si breadlock # lists the [breadway] section
sudo pacman -S --noconfirm yay-bin # installs with no key prompt
```
Then run the installer and, on the installed system, `sudo pacman -Sy`
again — the target keyring must already trust `56203B86…`.

View file

@ -73,15 +73,23 @@ passwd -l root || true
# over to the target (unpackfs may skip it / perms differ), leaving the installed
# system unable to verify package signatures — the first `pacman -Syu` then dies
# with "keyring is not writable / required key missing". Initialise it here so a
# fresh install can update out of the box. archlinux-keyring is already present
# and is the only keyring populated — it verifies official Arch packages.
# [breadway] stays SigLevel=Never (Forgejo does not serve pacman-compatible
# db signatures). Do not import KEYS.asc here: that key signs ISO SHA256SUMS,
# not the pacman repo; treating it as a repo key would be a lie.
# fresh install can update out of the box. archlinux-keyring verifies official
# Arch packages; the BOS release key (56203B86…, shipped at
# /etc/pacman.d/breadway-repo.asc) verifies the signed [breadway] repo at
# dl.breadway.dev/arch — SigLevel = Required there, every package and the db
# carry a .sig from it.
# ---------------------------------------------------------------------------
BREADWAY_KEY_FPR="56203B86A110695AE7F310934AF3323D678EB5E2"
if command -v pacman-key &>/dev/null; then
pacman-key --init || echo "WARN: pacman-key --init failed"
pacman-key --populate archlinux || echo "WARN: pacman-key --populate failed"
if [[ -f /etc/pacman.d/breadway-repo.asc ]]; then
pacman-key --add /etc/pacman.d/breadway-repo.asc \
&& pacman-key --lsign-key "$BREADWAY_KEY_FPR" \
|| echo "WARN: could not trust the [breadway] repo key — pacman -Sy will fail on [breadway]"
else
echo "WARN: /etc/pacman.d/breadway-repo.asc missing — [breadway] (SigLevel=Required) will not verify"
fi
fi
# ---------------------------------------------------------------------------

View file

@ -32,18 +32,17 @@ Include = /etc/pacman.d/mirrorlist
# are NOT here; they are bakery-baked into /usr/local at ISO build time.
#
# Packages are published to the Forgejo Arch registry (group "os") by the
# .forgejo/workflows/*.yml workflows in this repo (and breadlock's).
# .forgejo/workflows/*.yml workflows; scripts/ci-publish-signed-repo.sh then
# collects them, detach-signs each .pkg.tar.zst with the BOS release key
# (releases@breadway.dev), runs `repo-add -s`, and publishes the signed db
# at https://dl.breadway.dev/arch/$arch (signed-repo.yml).
#
# Forgejo's Arch package registry does not serve pacman-compatible db
# signatures. SigLevel = Never is TLS-only integrity: the connection is
# HTTPS (or rewritten to hestia's localhost:3002 in CI). breadlock (PAM)
# rides this repo. Do NOT flip to SigLevel = Required unless a signed db
# has been verified to work — Required without signatures breaks the ISO
# and every install that uses [breadway]. KEYS.asc is the ISO SHA256SUMS
# signing key, not a pacman repo key.
# SigLevel = Required: every package AND the db carry a .sig from key
# 56203B86A110695AE7F310934AF3323D678EB5E2 — the same key committed as
# KEYS.asc / etc/pacman.d/breadway-repo.asc, imported into the pacman
# keyring at build time (build-local.sh), on the live medium, and on the
# installed target (calamares/post-install.sh).
# -----------------------------------------------------------------------
# The section name must match Forgejo's served db filename
# ({owner}.{group}.{domain}.db) — pacman fetches "<section>.db" from Server.
[Breadway.os.git.breadway.dev]
SigLevel = Never
Server = https://git.breadway.dev/api/packages/Breadway/arch/os/$arch
[breadway]
SigLevel = Required
Server = https://dl.breadway.dev/arch/$arch

View file

@ -0,0 +1,15 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mDMEakhwGhYJKwYBBAHaRw8BAQdA/sZ/GYec5M2MD+w20mVF5tMUhGji210Dg7zL
TAhNsg60WUJPUyBSZWxlYXNlIFNpZ25pbmcgKGdpdC5icmVhZHdheS5kZXYvQnJl
YWR3YXkvYm9zIHJlbGVhc2VzIG9ubHkpIDxyZWxlYXNlc0BicmVhZHdheS5kZXY+
iJYEExYKAD4WIQRWIDuGoRBpWufzEJNK8zI9Z4614gUCakhwGgIbIwUJA8JnAAUL
CQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRBK8zI9Z4614ggYAQDP8FTZ14i9YPKD
ARvZuP5QaYOUFhQ8uyG0CowXKy9O0AEAqYfjnvyJI3N651pVFSNUXyP16w1kMPSs
K0g3CLsztQ+4OARqSHAaEgorBgEEAZdVAQUBAQdAuJFuy2GHz5m9wXTm/PdSpLE9
gERwHOLyM1OFuttrJW4DAQgHiH4EGBYKACYWIQRWIDuGoRBpWufzEJNK8zI9Z461
4gUCakhwGgIbDAUJA8JnAAAKCRBK8zI9Z4614nzLAP9grcIFsAAeCyVKhziHmpXq
E0Hm6FfIr4sdEf63HZkyfwD/XeKeWfb3EWvVsloJrZZ9tDmR67iK52Hwl82wfFAU
cAo=
=Mrh1
-----END PGP PUBLIC KEY BLOCK-----

View file

@ -0,0 +1,27 @@
#!/usr/bin/env bash
# Run by mkarchiso inside the airootfs chroot, after packages are installed
# and before the squashfs is built. (archiso prints a deprecation warning for
# this hook, but there is no non-deprecated replacement for "trust an extra
# pacman repo key in the image keyring", and BOS ships no pacman-init.service.)
#
# Purpose: trust the BOS release key (56203B86…) in the image's pacman
# keyring so the signed [breadway] repo (SigLevel = Required,
# https://dl.breadway.dev/arch) verifies both on the live medium and — via
# calamares' unpackfs, which copies this squashfs to the target — on the
# installed system. calamares/post-install.sh re-does this in the target
# chroot as a fallback (unpackfs can skip /etc/pacman.d/gnupg).
set -euo pipefail
BREADWAY_KEY_FPR="56203B86A110695AE7F310934AF3323D678EB5E2"
KEY_FILE="/etc/pacman.d/breadway-repo.asc"
pacman-key --init
pacman-key --populate archlinux
if [[ -f "$KEY_FILE" ]]; then
pacman-key --add "$KEY_FILE"
pacman-key --lsign-key "$BREADWAY_KEY_FPR"
echo "customize_airootfs: trusted [breadway] repo key $BREADWAY_KEY_FPR"
else
echo "customize_airootfs: WARNING $KEY_FILE missing; [breadway] will not verify" >&2
fi

View file

@ -49,18 +49,17 @@ Include = /etc/pacman.d/mirrorlist
# are NOT here; they are bakery-baked into /usr/local at ISO build time.
#
# Packages are published to the Forgejo Arch registry (group "os") by the
# .forgejo/workflows/*.yml workflows in this repo (and breadlock's).
# .forgejo/workflows/*.yml workflows; scripts/ci-publish-signed-repo.sh then
# collects them, detach-signs each .pkg.tar.zst with the BOS release key
# (releases@breadway.dev), runs `repo-add -s`, and publishes the signed db
# at https://dl.breadway.dev/arch/$arch (signed-repo.yml).
#
# Forgejo's Arch package registry does not serve pacman-compatible db
# signatures. SigLevel = Never is TLS-only integrity: the connection is
# HTTPS (or rewritten to hestia's localhost:3002 in CI). breadlock (PAM)
# rides this repo. Do NOT flip to SigLevel = Required unless a signed db
# has been verified to work — Required without signatures breaks the ISO
# and every install that uses [breadway]. KEYS.asc is the ISO SHA256SUMS
# signing key, not a pacman repo key.
# SigLevel = Required: every package AND the db carry a .sig from key
# 56203B86A110695AE7F310934AF3323D678EB5E2 — the same key committed as
# KEYS.asc / airootfs/etc/pacman.d/breadway-repo.asc, imported into the
# pacman keyring at build time (build-local.sh), on the live medium, and
# on the installed target (calamares/post-install.sh).
# -----------------------------------------------------------------------
# The section name must match Forgejo's served db filename
# ({owner}.{group}.{domain}.db) — pacman fetches "<section>.db" from Server.
[Breadway.os.git.breadway.dev]
SigLevel = Never
Server = https://git.breadway.dev/api/packages/Breadway/arch/os/$arch
[breadway]
SigLevel = Required
Server = https://dl.breadway.dev/arch/$arch