breadbar can freeze with its main loop idle and a dead breadd subscription (possible follow-on of the BAR-58 leak) #59

Open
opened 2026-10-07 22:26:49 +08:00 by Breadway · 0 comments
Owner

How found: while investigating BAR-58 the user reported the live bar was unresponsive (it has happened before). Process inspected with gdb on the live breadbar (22h uptime, 1.19 GB RSS). Confidence: medium on the symptoms, low on the cause.

Observed:

  • RSS frozen to within 4 KB for many minutes; main thread has had effectively zero context switches.
  • Main thread is idle in g_main_context_iteration -> ppoll with an INFINITE timeout and only 2 fds (glib wakeup eventfd, Wayland socket). No timers are registered at all.
  • A connection to breadd.sock (fd 29) has 81,818 bytes unread in breadbar's receive queue and 216,576 bytes stuck in breadd's send queue; it is not in the main poll set and not in any epoll set. Nothing reads it. breadd itself is healthy (bread ping ok).
  • Only 2 BreadClient subscription threads exist, though up to 3 subscriptions are created (widgets.** plus 2 launcher verbs in launcher_command.rs). Their Subscription handles are mem::forget'd (widgets/client.rs:56, launcher_command.rs:51), so a thread that exits early leaves its socket open and nobody reconnects.
  • relm4 ComponentSender::input() does .expect("The runtime of the component was shutdown...") (relm4 0.11 channel/component.rs:66). Every background thread in breadbar calls sender.input(), so if the App runtime stops, those threads panic and die silently (stderr goes to the launching tty).

Hypothesis (unverified): the same run had hit GLib "Too many GWeakRef registered" (reproduced in the BAR-58 harness, 78k per 3000 rebuilds with bread-theme v0.7.11), which could plausibly break the main loop or the App runtime and then kill the helper threads. Not proven. stderr from the stuck process was not available.

Suggested next steps: re-check after BAR-58's fix ships and breadbar runs for 40h+; if it still freezes, run it with stderr captured to a file (BREADBAR_LOG=debug) so the panic is visible; consider making the BreadClient subscription loop survive callback panics (catch_unwind) and making sender.input() failures non-fatal in background threads.

How found: while investigating BAR-58 the user reported the live bar was unresponsive (it has happened before). Process inspected with gdb on the live breadbar (22h uptime, 1.19 GB RSS). Confidence: medium on the symptoms, low on the cause. Observed: - RSS frozen to within 4 KB for many minutes; main thread has had effectively zero context switches. - Main thread is idle in g_main_context_iteration -> ppoll with an INFINITE timeout and only 2 fds (glib wakeup eventfd, Wayland socket). No timers are registered at all. - A connection to breadd.sock (fd 29) has 81,818 bytes unread in breadbar's receive queue and 216,576 bytes stuck in breadd's send queue; it is not in the main poll set and not in any epoll set. Nothing reads it. breadd itself is healthy (bread ping ok). - Only 2 BreadClient subscription threads exist, though up to 3 subscriptions are created (widgets.** plus 2 launcher verbs in launcher_command.rs). Their Subscription handles are mem::forget'd (widgets/client.rs:56, launcher_command.rs:51), so a thread that exits early leaves its socket open and nobody reconnects. - relm4 ComponentSender::input() does .expect("The runtime of the component was shutdown...") (relm4 0.11 channel/component.rs:66). Every background thread in breadbar calls sender.input(), so if the App runtime stops, those threads panic and die silently (stderr goes to the launching tty). Hypothesis (unverified): the same run had hit GLib "Too many GWeakRef registered" (reproduced in the BAR-58 harness, 78k per 3000 rebuilds with bread-theme v0.7.11), which could plausibly break the main loop or the App runtime and then kill the helper threads. Not proven. stderr from the stuck process was not available. Suggested next steps: re-check after BAR-58's fix ships and breadbar runs for 40h+; if it still freezes, run it with stderr captured to a file (BREADBAR_LOG=debug) so the panic is visible; consider making the BreadClient subscription loop survive callback panics (catch_unwind) and making sender.input() failures non-fatal in background threads.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Breadway/breadbar#59
No description provided.