Add a vLLM adapter with memory-aware sleep and wake #29

Open

Breadway wants to merge 2 commits from feature/ram-14-vllm-adapter into main

pull from: feature/ram-14-vllm-adapter

merge into: Breadway:main

Breadway:main

Breadway:feature/bakery-breadram

Breadway:feature/ram-30-benchmarks

Breadway:fix/ram-79-honest-llama-capability

Breadway:fix/ram-84-provider-rearm

Breadway:fix/ram-78-pressure-sample-clock

Conversation 0 Commits 2 Files changed 3 +228 -2

Breadway commented 2026-10-11 20:41:36 +08:00

Owner

Copy link

Conservative vLLM observation and ordered restore contract. Part of the breadram implementation plan (breadboard RAM-14).

What this adds

• crates/breadram-core/src/adapter/vllm.rs: a read-only adapter that reads GET /is_sleeping on an explicitly configured endpoint. 404/501, denied access, worker failure, malformed JSON, and missing responses clear the previous reading. It never enables development mode, changes launch arguments, or sends a sleep/wake request.
• A RestoreState contract for the ordered level 2 restoration (allocate weights, collectively reload weights, allocate KV cache, then verify inference) that rejects incomplete, out-of-order, and failed-worker evidence.
• Registered in the adapter registry alongside the existing arms. docs/VLLM.md records the contract and the boundary.

Scope note

Partial delivery. Sleep state is not proof of idle requests or valid restored weights, and activity and memory remain unknown. The remaining RAM-14 work (request-lease drain and dispatch integration, bounded multi-step operations and confirmations, disabled-sleep and live-stream integration, worker failure handling on a real deployment, successful inference after restore, exact tested version/backend, independent memory measurement) is named in docs/VLLM.md.

Verification

Run on this branch rebased onto main (e4c5770):

• cargo fmt --all --check
• cargo clippy --workspace --all-targets --locked -- -D warnings
• cargo build --workspace --locked
• cargo test --workspace --locked

All pass. Rebase kept both the Hook and Vllm registry arms.

Closes RAM-14.

Conservative vLLM observation and ordered restore contract. Part of the breadram implementation plan (breadboard RAM-14). ## What this adds - crates/breadram-core/src/adapter/vllm.rs: a read-only adapter that reads GET /is_sleeping on an explicitly configured endpoint. 404/501, denied access, worker failure, malformed JSON, and missing responses clear the previous reading. It never enables development mode, changes launch arguments, or sends a sleep/wake request. - A RestoreState contract for the ordered level 2 restoration (allocate weights, collectively reload weights, allocate KV cache, then verify inference) that rejects incomplete, out-of-order, and failed-worker evidence. - Registered in the adapter registry alongside the existing arms. docs/VLLM.md records the contract and the boundary. ## Scope note Partial delivery. Sleep state is not proof of idle requests or valid restored weights, and activity and memory remain unknown. The remaining RAM-14 work (request-lease drain and dispatch integration, bounded multi-step operations and confirmations, disabled-sleep and live-stream integration, worker failure handling on a real deployment, successful inference after restore, exact tested version/backend, independent memory measurement) is named in docs/VLLM.md. ## Verification Run on this branch rebased onto main (e4c5770): - cargo fmt --all --check - cargo clippy --workspace --all-targets --locked -- -D warnings - cargo build --workspace --locked - cargo test --workspace --locked All pass. Rebase kept both the Hook and Vllm registry arms. Closes RAM-14.

Breadway added 1 commit 2026-10-11 20:41:36 +08:00

feat: add conservative vLLM observation and restore ordering 4533ca8469

Breadway added 1 commit 2026-10-11 20:43:25 +08:00

fix: indent the adapter registry match arm a2d368b022

Breadway referenced this pull request 2026-10-11 20:51:24 +08:00

Add the optional local inference gateway for request tracking #13

Breadway force-pushed feature/ram-14-vllm-adapter from a2d368b022 to e507f4fb7c 2026-10-12 09:22:53 +08:00 Compare

Breadway added 1 commit 2026-10-12 09:24:07 +08:00

docs: name the lease and gateway blockers for vLLM sleep control b1806e947b

Breadway force-pushed feature/ram-14-vllm-adapter from b1806e947b to 2a056b38de 2026-10-12 15:33:44 +08:00 Compare

Breadway force-pushed feature/ram-14-vllm-adapter from 2a056b38de to 00fe0cb531 2026-10-12 19:02:20 +08:00 Compare

Breadway force-pushed feature/ram-14-vllm-adapter from 00fe0cb531 to 7449472024 2026-10-12 21:24:38 +08:00 Compare

All checks were successful

Hide all checks

CPU checks / check (push) Successful in 38s

Details

This pull request has changes conflicting with the target branch.

• crates/breadram-core/src/adapter/mod.rs

View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Merge commit title Merge pull request 'Add a vLLM adapter with memory-aware sleep and wake' (#29) from feature/ram-14-vllm-adapter into main

Merge commit body Reviewed-on: https://git.breadway.dev/Breadway/breadram/pulls/29

Merge pull request 'Add a vLLM adapter with memory-aware sleep and wake' (#29) from feature/ram-14-vllm-adapter into main Reviewed-on: https://git.breadway.dev/Breadway/breadram/pulls/29 Copy merge commit message

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feature/ram-14-vllm-adapter:feature/ram-14-vllm-adapter

git switch feature/ram-14-vllm-adapter

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main

git merge --no-ff feature/ram-14-vllm-adapter

git switch feature/ram-14-vllm-adapter

git rebase main

git switch main

git merge --ff-only feature/ram-14-vllm-adapter

git switch feature/ram-14-vllm-adapter

git rebase main

git switch main

git merge --no-ff feature/ram-14-vllm-adapter

git switch main

git merge --squash feature/ram-14-vllm-adapter

git switch main

git merge --ff-only feature/ram-14-vllm-adapter

git switch main

git merge feature/ram-14-vllm-adapter

git push origin main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Breadway/breadram!29

No description provided.