Fix silent frame-chain corruption on EnqueueFrame rejection; revert to 720p
Root cause (found by Opus 5 second-opinion review) of the freezing that survived every prior fix tonight: openscreen's Sender caps in-flight unacknowledged media at clamp(2*RTT, 66ms, 133ms) -- on a LAN that's pinned at the 66ms floor, about two frames at 30fps. When EnqueueFrame rejects a frame under that budget, facade.cc discarded the result (`(void)video_sender->EnqueueFrame(frame)`) and moved on. But vah264enc had already encoded the *next* frame as a P-slice depending on the one that just got silently dropped -- the encoder has no idea the drop happened, since it happens downstream of encoding, at this FFI boundary. The receiver sees an unbroken frame-ID sequence (nothing here told it otherwise) and decodes a P-slice against a reference picture it never received: a stuck/corrupted frame until the next regularly-scheduled key frame (up to ~2s, longer if that key frame is itself dropped the same way -- explains the 20+s outlier). Zero "receiver reported picture loss" lines across 195s of a visibly freezing session confirms the receiver genuinely never noticed anything was wrong, which a real decode-capability or packet-loss problem would have triggered. This also explains why moving to a faster network made it *worse*: the in-flight budget is RTT-derived, not bandwidth-derived, so more throughput doesn't raise the 66ms floor at all -- while 1080p tripled the per-frame packet count, increasing how often frames missed that window. Fixed the actual corruption: both drop paths in facade.cc (the EnqueueFrame rejection, and the pre-existing non-monotonic-capture-time guard) now set a `frame_chain_broken` flag, consumed once by breadcast_caststream_sender_needs_key_frame() so frame_pump_loop forces a key frame on the very next frame instead of chaining more P-slices onto a reference that no longer exists on the receiver. A separate flag from the existing `needs_key_frame` atomic because SchedulePoll's 100ms timer unconditionally overwrites that one with the Sender's own (unrelated) NeedsKeyFrame() reading, which would have silently clobbered this signal. Also reverted the Cast Streaming pipeline from tonight's 1080p experiment back to 1280x720 (build_video_pipeline_for_streaming + VideoParams::default(), which must agree -- a mismatch there is a separate protocol-level bug fixed earlier tonight). Not the root cause, but a real contributing factor per the packet-count reasoning above, and untangling it from the frame_chain_broken fix by changing both at once would make the next test ambiguous. Kept the 6000/1500 kbps bitrate range from earlier tonight, now actually paired with 720p for the first time. The deeper real fix -- raising openscreen's 66ms in-flight floor itself, which trades latency for headroom -- is out of scope for tonight; this targets the corruption mechanism (via the sanctioned, if awkward, needs_key_frame signal) without touching vendored openscreen constants.
This commit is contained in:
parent
a058482b39
commit
22a18eee1b
4 changed files with 55 additions and 26 deletions
|
|
@ -80,6 +80,27 @@ struct CastStreamSender {
|
|||
std::atomic<bool> needs_key_frame{true};
|
||||
std::atomic<int32_t> estimated_bandwidth_bps{kDefaultBandwidthEstimateBps};
|
||||
|
||||
// Set (never cleared except by the one consuming read, see
|
||||
// breadcast_caststream_sender_needs_key_frame below) whenever a frame gets
|
||||
// silently dropped after already being encoded -- either
|
||||
// Sender::EnqueueFrame() rejecting it (e.g. MAX_DURATION_IN_FLIGHT, the
|
||||
// in-flight budget openscreen enforces) or the non-monotonic-capture-time
|
||||
// guard below. Either way, vah264enc already encoded the *next* frame as a
|
||||
// P-slice depending on the one that just got dropped -- the encoder has no
|
||||
// idea the drop happened, since it happens downstream of encoding, at this
|
||||
// FFI boundary. Left alone, that reference is now dangling: the receiver
|
||||
// decodes it against whatever picture it last successfully received,
|
||||
// producing a stuck or corrupted frame that only resolves at the next
|
||||
// regularly-scheduled key frame (key-int-max=60 -- up to ~2s, longer still
|
||||
// if that key frame is itself dropped the same way). Forcing a key frame
|
||||
// on the very next enqueue turns "up to several seconds of corruption"
|
||||
// into "one dropped frame, then a clean resync" -- a separate flag rather
|
||||
// than reusing `needs_key_frame` directly because SchedulePoll's 100ms
|
||||
// timer unconditionally overwrites that one with the Sender's own
|
||||
// (unrelated) NeedsKeyFrame() reading, which would silently clobber this
|
||||
// signal before frame_pump_loop ever observed it.
|
||||
std::atomic<bool> frame_chain_broken{false};
|
||||
|
||||
// The caller's own user_data + callbacks, as passed to `_create`. Not
|
||||
// called directly -- session.h/message_port_bridge.h are instead given
|
||||
// trampolines below (with `this` as their user_data) so this struct can
|
||||
|
|
@ -282,6 +303,7 @@ int32_t breadcast_caststream_sender_enqueue_frame(CastStreamSender* sender,
|
|||
// here rather than passed through.
|
||||
if (sender->have_last_capture_time &&
|
||||
capture_time_us <= sender->last_capture_time_us) {
|
||||
sender->frame_chain_broken.store(true, std::memory_order_relaxed);
|
||||
return;
|
||||
}
|
||||
|
||||
|
|
@ -311,12 +333,15 @@ int32_t breadcast_caststream_sender_enqueue_frame(CastStreamSender* sender,
|
|||
ByteView(owned_data->data(), owned_data->size()));
|
||||
|
||||
// EnqueueFrame()'s result (e.g. MAX_DURATION_IN_FLIGHT under backpressure)
|
||||
// isn't propagated to the caller: by the time this runs, enqueue_frame()
|
||||
// has already returned 0 synchronously (this call is posted, not
|
||||
// immediate -- see facade.h's threading contract). Backpressure here just
|
||||
// means this one frame is dropped; the encoder finds out indirectly via
|
||||
// needs_key_frame()/estimated_bandwidth_bps() polling.
|
||||
(void)video_sender->EnqueueFrame(frame);
|
||||
// can't be propagated to enqueue_frame()'s caller synchronously -- by the
|
||||
// time this runs, that call has already returned 0 (this call is posted,
|
||||
// not immediate -- see facade.h's threading contract). What it *can* do
|
||||
// is flag the drop so the next frame comes in clean -- see
|
||||
// `frame_chain_broken`'s doc comment on why that matters here, not just
|
||||
// for the encoder's bitrate.
|
||||
if (video_sender->EnqueueFrame(frame) != Sender::OK) {
|
||||
sender->frame_chain_broken.store(true, std::memory_order_relaxed);
|
||||
}
|
||||
});
|
||||
|
||||
return 0;
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue