Add OpenVINO backend for Intel iGPU/dGPU (Arc)
All checks were successful
Mirror to GitHub / mirror (push) Successful in 1s
release / build (push) Successful in 2m12s

Mirrors the rocm/cuda pattern: ort::ep::OpenVINO with device_type "GPU"
(covers both Intel integrated graphics and Arc discrete GPUs through the
same EP), load-dynamic/dlopen so no OpenVINO toolkit is needed at build
time, and its own --openvino flag / backend = "openvino" config value.
Folded into the `full` feature alongside npu/rocm/cuda.

OpenVINO's provider options go through a generic key/value FFI interface
rather than a fixed C struct (unlike MIGraphX's OrtMIGraphXProviderOptions),
so it should be less exposed to the ABI-version-skew crash MIGraphX hit --
but that's inference from the EP's design, not verified against real
hardware. Like CUDA, this is compile-checked only: no Intel GPU in this
dev environment to runtime-verify against.

Version bump: breadmill 0.2.2 -> 0.2.3.
This commit is contained in:
Breadway 2026-07-03 22:58:27 +08:00
parent e5922e9c90
commit c6ed6a41d8
8 changed files with 112 additions and 35 deletions

View file

@ -21,10 +21,11 @@ Optional features:
| Feature | What it adds |
|---------|-------------|
| `npu` | AMD XDNA NPU via VitisAI ONNX Runtime EP (requires Ryzen AI SDK) |
| `rocm` | AMD iGPU via the MIGraphX ONNX Runtime EP (ROCm-backed) |
| `cuda` | NVIDIA GPU via the CUDA ONNX Runtime EP |
| `full` | All three of the above in one binary |
| `npu` | AMD XDNA NPU via VitisAI ONNX Runtime EP (requires Ryzen AI SDK) |
| `rocm` | AMD iGPU via the MIGraphX ONNX Runtime EP (ROCm-backed) |
| `cuda` | NVIDIA GPU via the CUDA ONNX Runtime EP |
| `openvino` | Intel iGPU/dGPU (Arc) via the OpenVINO ONNX Runtime EP |
| `full` | All four of the above in one binary |
```
# NPU build
@ -36,25 +37,29 @@ cargo build --release -p breadmill --features rocm
# CUDA (NVIDIA GPU) build
cargo build --release -p breadmill --features cuda
# OpenVINO (Intel iGPU/dGPU) build
cargo build --release -p breadmill --features openvino
# All backends in one binary (what the release build ships)
cargo build --release -p breadmill --features full
```
`rocm`/`cuda`/`npu` all use `ort`'s `load-dynamic` mode: at runtime, breadmill
dlopens whatever `libonnxruntime.so` the dynamic linker resolves (or
`rocm`/`cuda`/`npu`/`openvino` all use `ort`'s `load-dynamic` mode: at runtime,
breadmill dlopens whatever `libonnxruntime.so` the dynamic linker resolves (or
`ORT_DYLIB_PATH` if set). GPU acceleration only works if that ONNX Runtime
build actually has the matching execution provider compiled in — breadmill
logs a clear `Successfully registered` / `not enabled in this build` line for
this at startup (see [GPU backend notes](#gpu-backend-notes) below).
Because all three are dlopen-based, `full` doesn't require the NPU/ROCm/CUDA
toolkits to be installed at build time — only at run time, and only for
whichever single backend you actually select via `--npu`/`--rocm`/`--cuda`
or `backend` in config.toml. The **released binaries are built with
`full`**: same binary works CPU-only out of the box, and picks up NPU/ROCm/CUDA
acceleration on a machine that has the matching ONNX Runtime available,
without needing a different download. An explicit `--npu`/`--rocm`/`--cuda`
flag always overrides `backend` in config.toml, not the other way around.
Because all four are dlopen-based, `full` doesn't require the NPU/ROCm/CUDA/
OpenVINO toolkits to be installed at build time — only at run time, and only
for whichever single backend you actually select via
`--npu`/`--rocm`/`--cuda`/`--openvino` or `backend` in config.toml. The
**released binaries are built with `full`**: same binary works CPU-only out
of the box, and picks up NPU/ROCm/CUDA/OpenVINO acceleration on a machine
that has the matching ONNX Runtime available, without needing a different
download. An explicit `--npu`/`--rocm`/`--cuda`/`--openvino` flag always
overrides `backend` in config.toml, not the other way around.
## Setup
@ -117,6 +122,7 @@ breadmill status
breadmill --npu
breadmill --rocm
breadmill --cuda
breadmill --openvino
```
## Config
@ -137,7 +143,7 @@ snippet_len = 200 # max characters in result snippet
[model]
name = "nomic-embed-text-v1.5"
dim = 768
backend = "cpu" # "cpu", "npu", "rocm", or "cuda"
backend = "cpu" # "cpu", "npu", "rocm", "cuda", or "openvino"
```
`roots` and `excludes` support `~/` expansion. The index respects `.gitignore` files found during the walk.
@ -154,10 +160,11 @@ Set `backend = "npu"` in config (or pass `--npu`) when running a build compiled
### GPU backend notes
Both `rocm` and `cuda` need a system ONNX Runtime that was actually built with
the matching execution provider — the crate's own downloaded binary is CPU-only.
Point `ORT_DYLIB_PATH` at one, or install a distro package that provides
`libonnxruntime.so` with the EP baked in and let the dynamic linker find it.
`rocm`, `cuda`, and `openvino` all need a system ONNX Runtime that was
actually built with the matching execution provider — the crate's own
downloaded binary is CPU-only. Point `ORT_DYLIB_PATH` at one, or install a
distro package that provides `libonnxruntime.so` with the EP baked in and
let the dynamic linker find it.
**ROCm (`--rocm` / `backend = "rocm"`)** targets ONNX Runtime's **MIGraphX**
execution provider, not the classic `ROCMExecutionProvider`. Distro
@ -183,6 +190,18 @@ noticeable for interactive query embedding.
CUDA/cuDNN install. Unverified on real NVIDIA hardware in this repo — only
compile-checked, since development happened on an AMD-only machine.
**OpenVINO (`--openvino` / `backend = "openvino"`)** targets
`OpenVINOExecutionProvider` with `device_type = "GPU"`, covering both Intel
iGPUs and Arc dGPUs through the same EP (OpenVINO abstracts Intel's whole
hardware line — CPU/GPU/NPU — behind one provider and a device-type string).
Needs an OpenVINO-enabled ONNX Runtime and the OpenVINO runtime itself
installed. Its options go through a generic key/value FFI interface rather
than a fixed C struct (unlike MIGraphX's `OrtMIGraphXProviderOptions`), so it
should be less exposed to the kind of ABI-version-skew crash MIGraphX hit —
but that's inference from reading the EP's design, not something verified
against real Intel GPU hardware. Also unverified on real hardware in this
repo — only compile-checked, for the same reason as CUDA.
## Runtime paths
| Purpose | Path |