Breadway/breadsearch
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Packages Wiki Activity Actions

Report queries served in the breadmill status #16

Merged
Breadway merged 1 commit from feature/status-queries-served into feature/model-unload-verbs 2026-10-20 08:15:33 +08:00
Conversation 0 Commits 1 Files changed 3 +8
Breadway commented 2026-10-16 16:34:44 +08:00
Owner
Copy link

Adds an optional queries_served counter to the status response so a client polling every few seconds can tell a mill that answered short queries between polls from an idle one. Related: Breadway/breadram RAM-82.

breadmill counts a query only when it reaches the embedder, so a "model loading" or "model not ready" answer does not count. The counter starts at zero with the daemon and is not reset by unload or reload. In StatusInfo the field is Option with serde default, so older daemons stay distinguishable from one that has served zero.

Based on feature/model-unload-verbs (bc4b2d9, PR 12), not main, because it extends the status shape that branch introduces. Retarget to main once PR 12 lands.

No unit test: serve.rs and breadsearch-shared have no test seam and dispatch needs a Store and an embedder. Verified live instead: three runs of the breadram contention benchmark against this build. cargo build --release --features breadmill/full and cargo test -p breadsearch-shared pass.

Adds an optional queries_served counter to the status response so a client polling every few seconds can tell a mill that answered short queries between polls from an idle one. Related: Breadway/breadram RAM-82. breadmill counts a query only when it reaches the embedder, so a "model loading" or "model not ready" answer does not count. The counter starts at zero with the daemon and is not reset by unload or reload. In StatusInfo the field is Option<u64> with serde default, so older daemons stay distinguishable from one that has served zero. Based on feature/model-unload-verbs (bc4b2d9, PR 12), not main, because it extends the status shape that branch introduces. Retarget to main once PR 12 lands. No unit test: serve.rs and breadsearch-shared have no test seam and dispatch needs a Store and an embedder. Verified live instead: three runs of the breadram contention benchmark against this build. cargo build --release --features breadmill/full and cargo test -p breadsearch-shared pass.
feat: report queries served in the breadmill status
All checks were successful
check / check (push) Successful in 2m18s
Details
7e5eef9b2e
Breadway merged commit 287ea872b2 into feature/model-unload-verbs 2026-10-20 08:15:33 +08:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Breadway/breadsearch!16
No description provided.