Support iGPU GTT memory and inference servers on iGPUs

Breadway commented

2026-10-11 23:35:51 +08:00

Owner

Requested by the operator. breadram currently reports device memory as unknown and only accounts host RSS and cgroup memory, so a model loaded on an integrated GPU (shared system memory) is invisible or misattributed, and an iGPU inference server cannot be reasoned about.

How it was found

Direct operator request: "breadram needs to support iGPU GTT memory and inference servers using iGPUs".

Scope

Read integrated GPU shared memory (GTT): Intel i915/xe mem_info_gtt_used / mem_info_gtt_total, and amdgpu mem_info_gtt_used where present, from sysfs/drm, bounded and read-only.
Report it distinctly in the memory breakdown (GTT used and total), never folding it into host available memory, since GTT is shared host RAM rather than dedicated VRAM.
Account an iGPU-backed model (for example Ollama or llama.cpp with GPU layers on an iGPU) as device memory that is really shared host memory, so relief decisions are not made on a false zero.
Degrade honestly: no iGPU, no driver, or an unreadable counter reports unknown with a reason, never a fabricated zero.

Acceptance

A fixture reads GTT used/total from a fake drm tree and reports them separately from host RAM and from dedicated VRAM.
An absent or unreadable iGPU reports unknown with a reason.
The memory breakdown, diagnostics, and CLI show the GTT figures.
Inference-server adapters can attribute a model's iGPU footprint, and a workload that is otherwise idle is not released while it holds a loaded iGPU model.

Notes

Distinct from host RAM and from discrete VRAM: on an iGPU the "device" memory is a carve-out of system RAM, so it must be counted once, not twice.

Requested by the operator. breadram currently reports device memory as unknown and only accounts host RSS and cgroup memory, so a model loaded on an integrated GPU (shared system memory) is invisible or misattributed, and an iGPU inference server cannot be reasoned about. ## How it was found Direct operator request: "breadram needs to support iGPU GTT memory and inference servers using iGPUs". ## Scope - Read integrated GPU shared memory (GTT): Intel i915/xe `mem_info_gtt_used` / `mem_info_gtt_total`, and amdgpu `mem_info_gtt_used` where present, from sysfs/drm, bounded and read-only. - Report it distinctly in the memory breakdown (GTT used and total), never folding it into host available memory, since GTT is shared host RAM rather than dedicated VRAM. - Account an iGPU-backed model (for example Ollama or llama.cpp with GPU layers on an iGPU) as device memory that is really shared host memory, so relief decisions are not made on a false zero. - Degrade honestly: no iGPU, no driver, or an unreadable counter reports unknown with a reason, never a fabricated zero. ## Acceptance - A fixture reads GTT used/total from a fake drm tree and reports them separately from host RAM and from dedicated VRAM. - An absent or unreadable iGPU reports unknown with a reason. - The memory breakdown, diagnostics, and CLI show the GTT figures. - Inference-server adapters can attribute a model's iGPU footprint, and a workload that is otherwise idle is not released while it holds a loaded iGPU model. ## Notes Distinct from host RAM and from discrete VRAM: on an iGPU the "device" memory is a carve-out of system RAM, so it must be counted once, not twice.

Breadway commented

2026-10-11 23:52:14 +08:00

Author

Owner

Started. First piece published on feature/ram-67-igpu-gtt.

PR: #35

What landed: reads and reports integrated GPU shared memory. GTT used and total are read from the drm class, bounded and read-only, summed across cards, carried in the memory breakdown as gtt_used_bytes and gtt_total_bytes (protocol minor 7, defaulted), shown by breadram diagnostics, and exposed as the gpu_gtt capability with honest degradation. Missing is never zero.

Still to come: attributing an iGPU-backed model's footprint through the inference adapters so an otherwise idle workload is not released while it holds a loaded iGPU model, counting the carve-out once when it overlaps cgroup accounting, and exercising it against a real iGPU host. The PR body says Part of RAM-67.

Started. First piece published on `feature/ram-67-igpu-gtt`. PR: https://git.breadway.dev/Breadway/breadram/pulls/35 What landed: reads and reports integrated GPU shared memory. GTT used and total are read from the drm class, bounded and read-only, summed across cards, carried in the memory breakdown as gtt_used_bytes and gtt_total_bytes (protocol minor 7, defaulted), shown by breadram diagnostics, and exposed as the gpu_gtt capability with honest degradation. Missing is never zero. Still to come: attributing an iGPU-backed model's footprint through the inference adapters so an otherwise idle workload is not released while it holds a loaded iGPU model, counting the carve-out once when it overlaps cgroup accounting, and exercising it against a real iGPU host. The PR body says Part of RAM-67.

Breadway self-assigned this

2026-10-11 23:52:14 +08:00

Breadway commented

2026-10-11 23:57:28 +08:00

Author

Owner

Live evidence for the telemetry piece, on a real iGPU host.

This host has an AMD integrated GPU (card1, vendor 0x1002, amdgpu). Running breadramd --mode monitor on the branch and then breadram diagnostics:

iGPU GTT:   used 7.5 GiB, total 30.6 GiB
  gpu_gtt          available
  integrated GPU shared memory (GTT) is measured: 8074977280 bytes in use, separate from host available memory and from dedicated VRAM; a model on an iGPU is holding host RAM

The JSON report carries gtt_used_bytes 8074977280 and gtt_total_bytes 32892583936. Recorded in docs/IGPU.md. Still untested on i915 or xe.

Live evidence for the telemetry piece, on a real iGPU host. This host has an AMD integrated GPU (card1, vendor 0x1002, amdgpu). Running `breadramd --mode monitor` on the branch and then `breadram diagnostics`: ``` iGPU GTT: used 7.5 GiB, total 30.6 GiB gpu_gtt available integrated GPU shared memory (GTT) is measured: 8074977280 bytes in use, separate from host available memory and from dedicated VRAM; a model on an iGPU is holding host RAM ``` The JSON report carries gtt_used_bytes 8074977280 and gtt_total_bytes 32892583936. Recorded in docs/IGPU.md. Still untested on i915 or xe.

Breadway commented

2026-10-12 00:08:36 +08:00

Author

Owner

Inference attribution landed on the same branch (PR #35).

A provider that runs on an integrated GPU is now marked in the configuration with igpu = true (a new ProviderEndpoint field, default false). With it set, the Ollama adapter attributes a loaded model's reported device memory to host RAM, because on an iGPU it is a carve-out of system memory rather than dedicated VRAM, and it reports no separate device figure. The registry passes the flag through and records a note saying which way the endpoint was attributed. A test asserts the attribution against the two-model fixture. docs/IGPU.md documents the flag and why it matters: counting the carve-out as device memory would hide where it really lives and could let a relief decision treat the host as holding less than it does.

Remaining, honestly:

Count the GTT carve-out once when it overlaps a workload's cgroup accounting. Today the GTT figure and the cgroup figure are reported separately and neither is derived from the other, so nothing double counts, but that is an invariant to state and test rather than a mechanism that exists.
Exercise the reading on a non-amdgpu host (i915 or xe). The amdgpu path is verified live on this host.

Verification: cargo fmt --all --check, cargo clippy --workspace --all-targets --locked -- -D warnings, cargo build --workspace --locked, cargo test --workspace --locked all pass.

Inference attribution landed on the same branch (PR #35). A provider that runs on an integrated GPU is now marked in the configuration with `igpu = true` (a new ProviderEndpoint field, default false). With it set, the Ollama adapter attributes a loaded model's reported device memory to host RAM, because on an iGPU it is a carve-out of system memory rather than dedicated VRAM, and it reports no separate device figure. The registry passes the flag through and records a note saying which way the endpoint was attributed. A test asserts the attribution against the two-model fixture. docs/IGPU.md documents the flag and why it matters: counting the carve-out as device memory would hide where it really lives and could let a relief decision treat the host as holding less than it does. Remaining, honestly: - Count the GTT carve-out once when it overlaps a workload's cgroup accounting. Today the GTT figure and the cgroup figure are reported separately and neither is derived from the other, so nothing double counts, but that is an invariant to state and test rather than a mechanism that exists. - Exercise the reading on a non-amdgpu host (i915 or xe). The amdgpu path is verified live on this host. Verification: cargo fmt --all --check, cargo clippy --workspace --all-targets --locked -- -D warnings, cargo build --workspace --locked, cargo test --workspace --locked all pass.

Breadway commented

2026-10-12 00:14:23 +08:00

Author

Owner

The count-once invariant landed on the same branch (PR #35 head a9de8f9): GTT is reported in its own field and never folded into host available memory, because on an iGPU it is a carve-out of the same RAM. A daemon test asserts the breakdown copies the GTT figure and leaves ram_available_bytes exactly as the kernel reported it, and the notes say the two are separate.

That leaves one item on this card: exercise the reading on a non-amdgpu driver host (i915 or xe). This host is amdgpu and verified live; i915/xe expose the same mem_info_gtt_* attribute but that is reference material, not a reading taken here. That is a hardware-gated item, so the card is otherwise complete for the hardware available. Moving it to review.

The count-once invariant landed on the same branch (PR #35 head a9de8f9): GTT is reported in its own field and never folded into host available memory, because on an iGPU it is a carve-out of the same RAM. A daemon test asserts the breakdown copies the GTT figure and leaves ram_available_bytes exactly as the kernel reported it, and the notes say the two are separate. That leaves one item on this card: exercise the reading on a non-amdgpu driver host (i915 or xe). This host is amdgpu and verified live; i915/xe expose the same mem_info_gtt_* attribute but that is reference material, not a reading taken here. That is a hardware-gated item, so the card is otherwise complete for the hardware available. Moving it to review.

Breadway commented

2026-10-12 00:20:57 +08:00

Author

Owner

Blocked: PR #35 conflicts with main, needs a rebase. Not merged.

CI is green on head a9de8f9 (CPU checks / check). The blocker is a merge conflict, not the change. The branch is based on 2f4e2f3, one commit behind main, which now carries baebd12 "Support containerized and system-service workloads". A trial merge of origin/main into feature/ram-67-igpu-gtt fails on a single file, crates/breadram-core/src/ipc.rs, at two hunks (both the protocol minor bump):

PROTOCOL_MINOR at ipc.rs:37: main sets it to 6 ("per workload effective memory limit"), the branch sets it to 7 ("integrated GPU shared memory"). The branch's doc comment already carries main's minor-6 wording, so the right resolution keeps 7.
The comment block and assert_eq!(PROTOCOL_MINOR, ...) in the test a_diagnostics_payload_from_the_previous_minor_version_still_decodes (ipc.rs around 985-1000), same 6-vs-7 clash.

Everything else auto-merges. Main's other unrelated ipc.rs addition, WorkloadSummary.limit_bytes, is in a separate hunk and survives the rebase untouched.

To unblock: on feature/ram-67-igpu-gtt, git fetch origin && git rebase origin/main, resolve ipc.rs to minor 7 (keeping main's limit_bytes), and push to the same branch. PR #35 updates in place; do not open a second PR for this key.

Nothing else was flagged in the diff. Against RAM-67 the change reads as complete for the hardware present: the GTT counter read with the per-card sum and honest unknown/absent degradation, the gpu_gtt capability, the breakdown plus human and JSON diagnostics, the igpu provider flag that attributes an iGPU model's device memory to host RAM, and the count-once invariant test. The remaining item (a live reading on an i915 or xe host) stays hardware gated.

Leaving this card in review pending the rebase.

**Blocked: PR #35 conflicts with main, needs a rebase. Not merged.** CI is green on head a9de8f9 (CPU checks / check). The blocker is a merge conflict, not the change. The branch is based on 2f4e2f3, one commit behind main, which now carries baebd12 "Support containerized and system-service workloads". A trial merge of origin/main into feature/ram-67-igpu-gtt fails on a single file, crates/breadram-core/src/ipc.rs, at two hunks (both the protocol minor bump): - `PROTOCOL_MINOR` at ipc.rs:37: main sets it to 6 ("per workload effective memory limit"), the branch sets it to 7 ("integrated GPU shared memory"). The branch's doc comment already carries main's minor-6 wording, so the right resolution keeps 7. - The comment block and `assert_eq!(PROTOCOL_MINOR, ...)` in the test `a_diagnostics_payload_from_the_previous_minor_version_still_decodes` (ipc.rs around 985-1000), same 6-vs-7 clash. Everything else auto-merges. Main's other unrelated ipc.rs addition, `WorkloadSummary.limit_bytes`, is in a separate hunk and survives the rebase untouched. To unblock: on `feature/ram-67-igpu-gtt`, `git fetch origin && git rebase origin/main`, resolve ipc.rs to minor 7 (keeping main's `limit_bytes`), and push to the same branch. PR #35 updates in place; do not open a second PR for this key. Nothing else was flagged in the diff. Against RAM-67 the change reads as complete for the hardware present: the GTT counter read with the per-card sum and honest unknown/absent degradation, the `gpu_gtt` capability, the breakdown plus human and JSON diagnostics, the `igpu` provider flag that attributes an iGPU model's device memory to host RAM, and the count-once invariant test. The remaining item (a live reading on an i915 or xe host) stays hardware gated. Leaving this card in review pending the rebase.

Breadway commented

2026-10-12 00:27:19 +08:00

Author

Owner

Rebased onto main (baebd12) as the review asked. The only conflict was the protocol minor bump in ipc.rs; resolved to keep 7 (integrated GPU shared memory) with 6 (per workload effective memory limit) already above it, and the older-payload test asserts 7. Head is now 2ff1108, full check set green. Ready to merge.

Rebased onto main (baebd12) as the review asked. The only conflict was the protocol minor bump in ipc.rs; resolved to keep 7 (integrated GPU shared memory) with 6 (per workload effective memory limit) already above it, and the older-payload test asserts 7. Head is now 2ff1108, full check set green. Ready to merge.

Breadway commented

2026-10-12 00:39:20 +08:00

Author

Owner

Reviewed and merged. PR #35 squashed onto main as b38f03b, branch deleted. CI was green on the rebased head 2ff1108 (CPU checks / check), the PR was mergeable, and each RAM-67 acceptance point is met:

GTT used/total are read from the drm class (telemetry/igpu.rs), bounded to 16 cards and summed, carried as gtt_used_bytes/gtt_total_bytes (protocol minor 7, both defaulted), and surfaced as the gpu_gtt capability.
Reported separately from host RAM and from dedicated VRAM: the breakdown copies the figures and ram_available_bytes is left exactly as the kernel reported it (daemon test integrated_gpu_memory_is_reported_separately_and_never_folded_into_host_ram).
An absent or unreadable iGPU reports unknown with a reason, never zero; the capability reads available, degraded, or unavailable to match.
Diagnostics (human and JSON) and the CLI show the GTT figures.
A provider marked igpu = true attributes a loaded model's whole size to host RAM and reports no separate device figure, so an otherwise idle workload holding an iGPU model keeps holding host RAM rather than being released on a false zero.

Minor, non-blocking notes, suggested as a follow-up card rather than rework:

The unload projection in ollama.rs (operation()) still splits host/device using size_vram_bytes even when the endpoint is igpu = true, so the provider's stated expectation carries a phantom device delta while the plan estimate (from the corrected accounting) is right. The executor already prints both figures, so this is an accounting nit, not a behavior bug.
The "a card exposing a GTT counter is an integrated GPU" heuristic would also label a discrete AMD card's GTT as iGPU. The card defines it this way and the count-once intent still holds, but the label can mislead.
i915/xe remain hardware-gated, correctly recorded as such in docs/IGPU.md.

Merging to done.

**Reviewed and merged.** PR #35 squashed onto main as b38f03b, branch deleted. CI was green on the rebased head 2ff1108 (CPU checks / check), the PR was mergeable, and each RAM-67 acceptance point is met: - GTT used/total are read from the drm class (telemetry/igpu.rs), bounded to 16 cards and summed, carried as gtt_used_bytes/gtt_total_bytes (protocol minor 7, both defaulted), and surfaced as the gpu_gtt capability. - Reported separately from host RAM and from dedicated VRAM: the breakdown copies the figures and ram_available_bytes is left exactly as the kernel reported it (daemon test integrated_gpu_memory_is_reported_separately_and_never_folded_into_host_ram). - An absent or unreadable iGPU reports unknown with a reason, never zero; the capability reads available, degraded, or unavailable to match. - Diagnostics (human and JSON) and the CLI show the GTT figures. - A provider marked igpu = true attributes a loaded model's whole size to host RAM and reports no separate device figure, so an otherwise idle workload holding an iGPU model keeps holding host RAM rather than being released on a false zero. Minor, non-blocking notes, suggested as a follow-up card rather than rework: - The unload projection in ollama.rs (operation()) still splits host/device using size_vram_bytes even when the endpoint is igpu = true, so the provider's stated expectation carries a phantom device delta while the plan estimate (from the corrected accounting) is right. The executor already prints both figures, so this is an accounting nit, not a behavior bug. - The "a card exposing a GTT counter is an integrated GPU" heuristic would also label a discrete AMD card's GTT as iGPU. The card defines it this way and the count-once intent still holds, but the label can mislead. - i915/xe remain hardware-gated, correctly recorded as such in docs/IGPU.md. Merging to done.

Breadway closed this issue

2026-10-12 00:39:21 +08:00

Rows
Columns

Support iGPU GTT memory and inference servers on iGPUs #34

How it was found

Scope

Acceptance

Notes