Fix ROCm backend (target MIGraphX EP) and add CUDA support
ROCm silently fell back to CPU: the code targeted ONNX Runtime's classic ROCMExecutionProvider, but distro ROCm-enabled ONNX Runtime builds (e.g. Arch's onnxruntime-rocm) are commonly compiled with --use_migraphx instead, and registration failures were invisible since breadmill never installed a tracing subscriber. Switches the rocm feature to target MIGraphX, adds a default tracing subscriber so EP registration success/failure is always visible, and fixes a real crash where MIGraphX's output sequence padding could index the attention mask out of bounds during mean-pooling. Also adds a CUDA backend (--cuda / backend = "cuda") mirroring the same ort execution-provider pattern, for NVIDIA hardware. Version bump: 0.1.0 -> 0.2.0.
This commit is contained in:
parent
d3843f3131
commit
2618a33fd5
10 changed files with 227 additions and 96 deletions
|
|
@ -80,3 +80,7 @@ Start from `breadbox/breadbox/src/main.rs`. **Reuse verbatim:** the gtk4-layer-s
|
|||
- nomic prefixes + mean-pool + normalize must match between index and query or recall collapses.
|
||||
- `ort` linking: prefer the crate's downloaded/bundled ONNX Runtime to avoid version skew with Arch's `onnxruntime`.
|
||||
- Office formats (docx/odt) are best-effort in v1; md/txt/org/pdf are the reliable path.
|
||||
- GPU EPs (ROCm/CUDA) fail to register silently at the ONNX Runtime level and fall back to CPU — always check
|
||||
startup logs for `Successfully registered` before trusting a GPU build is actually accelerating. See
|
||||
[README: GPU backend notes](README.md#gpu-backend-notes) for the MIGraphX-vs-ROCMExecutionProvider distinction
|
||||
and the per-shape JIT-compile-and-cache behavior that matters for interactive query latency.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue