Breadway/breadarr
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix real bugs found by an independent Opus review of the transcode feature

Browse source 
Requested and applied a full review of the AV1 transcode implementation.
Findings and fixes:

- Season-pack import bypassed upgrade_locked entirely: a season pack
  scored higher than a locally-transcoded file's stale release score
  would silently overwrite it. Fixed in import_season_pack_file (and
  mirrored into import_one for defense-in-depth) to check
  upgrade_locked before the score comparison, not after.
- The post-import enqueue hook only checked anime, silently skipping
  the HDR/2160p exclusions find_backlog_candidates applies to the
  backfill -- a freshly-grabbed HDR file would have gone through the
  unverified HDR-via-VAAPI path. Centralized all eligibility rules
  (anime, already-av1, HDR, height) into transcode::should_enqueue,
  used by both import_one and the newly-added season-pack enqueue
  hook (season packs previously had no transcode hook at all, despite
  being the actual headline use case -- 100GB+ season packs).
- find_backlog_candidates: a NULL-height row was included as a
  candidate but could never actually be claimed (claim_pending_jobs
  requires non-null height), permanently stuck pending. Fixed the
  WHERE clause.
- encode_and_verify now probes the real input file first and skips
  the encode entirely if it's already AV1 -- closes a narrow crash
  window where a rename-succeeded-but-DB-update-failed job would
  otherwise re-encode an already-transcoded file on retry.
- finalize_job's success path could leak a verified temp file and
  strand a job in 'running' forever if a filesystem operation failed
  partway through; now wrapped so any failure there cleans up and
  marks the job failed like every other error path.
- reset_orphaned_running_jobs now also sweeps each reset job's
  leftover temp file (job-id-scoped paths, so this lookup is
  unambiguous) rather than leaking them on a disk that's usually
  already tight on space.
- Added a unique index preventing two active jobs for the same file
  ever existing at once -- closes the remaining gap in last commit's
  concurrency fix (a daemon restart's reset could otherwise race a
  still-alive transcode-library backfill onto the same file).
- Made the VAAPI rate-control mode explicit (VBR) instead of
  driver-inferred.
- The daemon's transcode ticker awaited each cycle inline in the
  tokio::select! loop, blocking every other cycle (import, search,
  upgrade, reconcile) for the full multi-minute duration of an
  encode. Now spawns each cycle detached with a try-lock guard so
  overlapping ticks skip cleanly rather than stacking (the
  concurrency cap from the previous commit makes this safe).

Two items reviewed and deliberately left as documented, not fixed:
mp4 sources with mov_text subtitles will fail the encode cleanly (no
data loss, just no space saved) since Matroska can't hold that codec
via stream copy -- fixing this needs per-stream codec probing that
wasn't safe to add untested at this hour. Renaming an mp4 source's
extension to .mkv after transcoding is cosmetic (Jellyfin
content-sniffs fine) and left alone.
This commit is contained in:
Breadway 2026-07-25 00:38:51 +08:00
parent 576aad3bfe
commit 66d323b7f7
5 changed files with 395 additions and 65 deletions
Download patch file Download diff file Expand all files Collapse all files

51
breadarrd/src/main.rs
View file

@ -358,6 +358,16 @@ async fn background_loop(
let mut upgrade_ticker = tokio::time::interval(std::time::Duration::from_secs(
config.sources.upgrade_poll_interval_secs,
));
// Guards against the transcode ticker itself blocking every other cycle
// (import, search, upgrade, reconcile) for the full multi-minute
// duration of an encode — the ticker below spawns each cycle detached
// rather than awaiting it inline, and this is what stops two spawned
// cycles from running at once if one is still going when the next tick
// fires (a real possibility: files easily take longer to encode than
// `poll_interval_secs`). `claim_pending_jobs`'s own concurrency cap
// already makes overlap *safe*; this just keeps it from happening
// pointlessly.
let transcode_busy = std::sync::Arc::new(tokio::sync::Mutex::new(()));
let mut transcode_ticker = tokio::time::interval(std::time::Duration::from_secs(
config.transcode.poll_interval_secs,
));
@ -420,7 +430,7 @@ async fn background_loop(
_ = import_ticker.tick() => {
let result = {
let conn = conn.lock().await;
importer::run_import_cycle(&conn, &qbit, jellyfin.as_ref(), &config.qbit.category, &config.qbit.container_downloads_path, &config.qbit.host_downloads_path, config.transcode.enabled).await
importer::run_import_cycle(&conn, &qbit, jellyfin.as_ref(), &config.qbit.category, &config.qbit.container_downloads_path, &config.qbit.host_downloads_path, config.transcode.enabled.then_some(&config.transcode)).await
};
if let (Ok(stats), Some(n)) = (&result, &notifier) {
if stats.failed > 0 {
@ -553,18 +563,33 @@ async fn background_loop(
if !config.transcode.enabled {
continue;
}
let result = transcode::run_cycle(conn.clone(), config.transcode.clone(), jellyfin.as_ref()).await;
let record = match &result {
Ok(stats) => {
info!(?stats, "transcode cycle complete");
api::CycleRecord { at: chrono::Utc::now(), ok: true, detail: format!("{stats:?}") }
}
Err(e) => {
error!(error = %e, "transcode cycle failed");
api::CycleRecord { at: chrono::Utc::now(), ok: false, detail: e.to_string() }
}
let Ok(busy_permit) = transcode_busy.clone().try_lock_owned() else {
// A previous cycle is still running (this file's
// encode took longer than one poll interval) — skip
// this tick rather than spawning a second overlapping
// one; the still-running cycle will pick up any newly
// pending jobs on its own next iteration anyway.
continue;
};
cycle_status.lock().expect("cycle_status poisoned").last_transcode = Some(record);
let conn = conn.clone();
let cfg = config.transcode.clone();
let jellyfin = jellyfin.clone();
let cycle_status = cycle_status.clone();
tokio::spawn(async move {
let _permit = busy_permit;
let result = transcode::run_cycle(conn, cfg, jellyfin.as_ref()).await;
let record = match &result {
Ok(stats) => {
info!(?stats, "transcode cycle complete");
api::CycleRecord { at: chrono::Utc::now(), ok: true, detail: format!("{stats:?}") }
}
Err(e) => {
error!(error = %e, "transcode cycle failed");
api::CycleRecord { at: chrono::Utc::now(), ok: false, detail: e.to_string() }
}
};
cycle_status.lock().expect("cycle_status poisoned").last_transcode = Some(record);
});
}
_ = reconcile_ticker.tick() => {
let result = {
@ -879,7 +904,7 @@ async fn debug_import_cycle(config: &Config) -> Result<()> {
&config.qbit.category,
&config.qbit.container_downloads_path,
&config.qbit.host_downloads_path,
config.transcode.enabled,
config.transcode.enabled.then_some(&config.transcode),
)
.await?;
println!("{stats:?}");