Root cause (found by Opus 5 second-opinion review) of the freezing that
survived every prior fix tonight: openscreen's Sender caps in-flight
unacknowledged media at clamp(2*RTT, 66ms, 133ms) -- on a LAN that's
pinned at the 66ms floor, about two frames at 30fps. When EnqueueFrame
rejects a frame under that budget, facade.cc discarded the result
(`(void)video_sender->EnqueueFrame(frame)`) and moved on. But
vah264enc had already encoded the *next* frame as a P-slice depending
on the one that just got silently dropped -- the encoder has no idea
the drop happened, since it happens downstream of encoding, at this
FFI boundary. The receiver sees an unbroken frame-ID sequence (nothing
here told it otherwise) and decodes a P-slice against a reference
picture it never received: a stuck/corrupted frame until the next
regularly-scheduled key frame (up to ~2s, longer if that key frame is
itself dropped the same way -- explains the 20+s outlier). Zero
"receiver reported picture loss" lines across 195s of a visibly
freezing session confirms the receiver genuinely never noticed
anything was wrong, which a real decode-capability or packet-loss
problem would have triggered.
This also explains why moving to a faster network made it *worse*:
the in-flight budget is RTT-derived, not bandwidth-derived, so more
throughput doesn't raise the 66ms floor at all -- while 1080p tripled
the per-frame packet count, increasing how often frames missed that
window.
Fixed the actual corruption: both drop paths in facade.cc (the
EnqueueFrame rejection, and the pre-existing non-monotonic-capture-time
guard) now set a `frame_chain_broken` flag, consumed once by
breadcast_caststream_sender_needs_key_frame() so frame_pump_loop forces
a key frame on the very next frame instead of chaining more P-slices
onto a reference that no longer exists on the receiver. A separate flag
from the existing `needs_key_frame` atomic because SchedulePoll's 100ms
timer unconditionally overwrites that one with the Sender's own
(unrelated) NeedsKeyFrame() reading, which would have silently clobbered
this signal.
Also reverted the Cast Streaming pipeline from tonight's 1080p
experiment back to 1280x720 (build_video_pipeline_for_streaming +
VideoParams::default(), which must agree -- a mismatch there is a
separate protocol-level bug fixed earlier tonight). Not the root cause,
but a real contributing factor per the packet-count reasoning above, and
untangling it from the frame_chain_broken fix by changing both at once
would make the next test ambiguous. Kept the 6000/1500 kbps bitrate
range from earlier tonight, now actually paired with 720p for the first
time.
The deeper real fix -- raising openscreen's 66ms in-flight floor itself,
which trades latency for headroom -- is out of scope for tonight; this
targets the corruption mechanism (via the sanctioned, if awkward,
needs_key_frame signal) without touching vendored openscreen constants.
User is moving to a faster (150 Mbps) network and wants 1080p. Raised
build_video_pipeline_for_streaming and VideoParams::default() together
(they must agree -- see caststream.rs's doc comment on why a resolution
mismatch there is a protocol violation, not just soft video).
Left build_video_pipeline (the HLS/DLNA path) at 720p -- that one's
1280x720 choice is about an older Default Media Receiver's decoder
profile/level, unrelated to what's changing here.
Bitrate ceiling raised from 4-8x scaling but deliberately not straight
back up to the old 8 Mbps: real testing tonight showed the AIMD probe
pins to whatever MAX_BITRATE_KBPS is for the entire session once
estimated_bandwidth_bps() reports (unreliably -- flat ~20 Mbps most of
a session that was visibly stuttering) that there's headroom, and 8
Mbps sustained was more than the previous network+receiver could hold.
6 Mbps is a solid target for 1080p30 on its own merits. MIN_BITRATE_KBPS
bumped 1000->1500 to match (1080p needs more of a floor than 720p did
before it's a wall of blocking artifacts).
start()'s "never received an ANSWER" branch called session.stop()
directly (blocking, unbounded, not even off the async runtime's worker
thread) and capture.close().await with no timeout -- the exact same
freeze-the-whole-daemon hazard Self::stop() was just bounded against,
just at a different call site. An unresponsive receiver hits this path
by definition (that's what a negotiation timeout means), so it's not a
hypothetical: reproduced live just now, wedging the daemon for over a
minute with no way to recover short of kill -9.
Factored the bound into stop_session_bounded/close_capture_bounded so
both call sites share one implementation instead of drifting.
deadlock, and OFFER/encode resolution mismatch
Found live-testing on real hardware after the first round of fixes:
1. mDNS resolves one Chromecast on every local address it has (a private
IPv4 and a link-local IPv6 in the common case), as separate
CastDeviceFound events for the same id. The device map was a plain
HashMap::insert, so whichever address resolved last won -- and a bare
fe80:: address has no interface scope attached, so connecting to it
fails outright. This is what "failed to connect and launch the
Mirroring receiver" actually was; the message just didn't say why,
since daemon.rs was converting the anyhow::Error with to_string()
(Display, outermost .context() only) instead of "{e:#}" (full chain).
Fixed both: prefer an already-usable host over a link-local one
instead of always taking the newest resolution, and preserve the full
error chain to the GUI/bread event log.
2. The CASTV2 receive loop (run_io_loop) treated any error from
device.receive() as connection-fatal and ended the whole loop --
including a plain JSON deserialization failure on a single
MEDIA_STATUS message (rust_cast's struct requires an `images` field
this receiver didn't send). That killed the receiver-status/control
channel for the rest of every session, on the very first status
update, while the RTP stream itself kept flowing obliviously.
rust_cast::Error already distinguishes Io/Tls/Dns (actually fatal)
from Serialization/Parsing/etc (a bad message, not a dead socket) --
only end the loop on the former now.
3. CastSession::stop() blocks on an unbounded reply_rx.recv() waiting for
the io thread's device.receiver.stop_app() -- a network round trip
rust_cast gives no way to put a read timeout on. If the receiver ever
stops responding, that never returns, and since the daemon actor
processes one command at a time, a single wedged stop_cast freezes
every future IPC request too, recoverable only by killing the process
-- which is exactly what was observed live. Bounded both that wait and
join_pump's thread joins to 5s; past that, log a warning and tear down
anyway rather than hang forever. The abandoned thread(s) may leak, but
a leak beats an unrecoverable daemon.
4. VideoParams::default() advertised 1920x1080 in the OFFER while
build_video_pipeline_for_streaming actually encodes 1280x720 --
negotiated and actual resolution disagreeing is a real protocol
violation, not just soft video, and a plausible cause of a receiver
decoder corrupting or freezing outright rather than merely looking
worse. Made the OFFER match what's actually sent.
The receiver-status io loop treated any read error as a dead connection,
so one malformed/unexpected message (e.g. a MEDIA_STATUS missing a field
the struct requires) tore down the whole control channel for the rest of
the session even though the RTP stream was fine. Only end the loop on
Io/Tls/Dns errors now; log and keep going on Serialization/Parsing/etc.
mDNS resolves a single physical Cast device once per local address it
has, so the same device id can show up with a private IPv4 host and
again with a link-local IPv6 host. A bare fe80:: address has no zone id
attached, so connecting to it fails outright -- don't let it replace an
already-usable host in the device map just because it resolved more
recently.
e.to_string() on an anyhow::Error only prints the outermost .context()
message; the underlying cause (why connect_app/build_pipeline/etc.
actually failed) was being silently dropped before it ever reached the
GUI or bread event log. Use "{e:#}" instead.
Three independent problems found by reading the mirroring paths end to end.
1. Segfault on Cast session teardown. CastStreamSender::SchedulePoll
self-reschedules every 100ms with a raw `this` and was never cancelled,
so the one task that can be scheduled to run *after* an already-queued
teardown task would dereference the just-reset `environment` unique_ptr
(Environment::task_runner() dereferences a member immediately) --- a hard
null deref on openscreen's TaskRunner thread. TaskRunnerImpl's shutdown
has an explicit flushing phase and PlatformClientPosix::ShutDown()'s quit
task queues behind whatever is already pending, so this is a race the
teardown path can lose. Latch a `shutting_down` atomic before posting
teardown and check it in the poll and in every other posted task.
2. Encoder bitrate collapsing to the floor within seconds. The control loop
set `target = 0.85 * estimate` once a second unconditionally. openscreen's
BandwidthEstimator deliberately under-estimates capacity whenever the
transmit rate is below it and documents the required TCP-like response;
multiplying the target by <=0.85 every second instead walks 4000 kbps past
1500 in ~6s and pins it at the floor on a healthy LAN. Replaced with
proper AIMD (hold on a zero/unknown estimate, back off below it, probe up
10%/s otherwise), clamped to 1000..8000 kbps, with unit tests.
3. DLNA/HLS latency. Segment length is max(target-duration, GOP), so
target-duration=2 with a 2s GOP gave 2s segments, ~6s of renderer buffer,
plus 3 segments of stale video waited for before handover. 1s segments
(GOP halved to make that reachable), shorter playlist, and wait for 2
segments instead of 3.
Also hardened two paths into openscreen's fatal OSP_CHECK on strictly
increasing RTP timestamps: pull_encoded_frame no longer substitutes 0 for a
missing PTS (it skips the buffer), and facade.cc drops non-monotonic capture
times at the FFI boundary. Either could previously abort the daemon outright.
CastMirrorSession now owns the Arc<CastStreamSender> instead of leaving its
lifetime to whichever detached pump thread dropped the last clone, so the
blocking FFI destroy happens at a defined point in stop() with the pump
joins ordered around it.
Builds out the full v1 scope: a vendored+patched openscreen subset for
low-latency Cast Streaming (Mirroring receiver 0F5096E8) alongside the
existing Cast V2/HLS and new DLNA/AVTransport casting paths, breadcastd's
Idle/Casting state machine with a private IPC socket, the breadcast GTK4
popup as a thin IPC client, and bread.cast.*/bread.command.cast.* breadd
integration (device discovery, start/stop, mirroring lifecycle events).
Also adds bakery/systemd/Forgejo CI packaging.
Validated end-to-end against a real Chromecast/Google TV: negotiated
Cast Streaming session, live pipeline playback, and daemon+GUI click-to-cast/
stop through the actual popup.
