asdfghj1237890/WebVideo2NAS

# WebVideo2NAS [](https://opensource.org/licenses/MIT) [](https://docs.docker.com/compose/) [](https://www.python.org/) [](https://developer.chrome.com/docs/extensions/) [](https://github.com/asdfghj1237890/WebVideo2NAS/releases/latest) **Languages**: **English** (`README.md`) | **繁體中文** (`README.zh-TW.md`) ## Table of Contents - [Overview](#overview) - [Quick Links](#quick-links) - [Key Features](#key-features) - [Technology Stack](#technology-stack) - [Project Structure](#project-structure) - [Requirements](#requirements) - [Getting Started / Installation](#installation) - [Usage](#usage) - [Configuration](#configuration) - [Security](#security) - [Limitations](#limitations) - [Troubleshooting](#troubleshooting) - [Contributing](#contributing) - [License](#license) - [Changelog](#changelog) - [Support](#support) ## Overview This system enables you to: 1. 🔍 Detect M3U8, MPD, MP4, and MOV video URLs in Chrome (including disguised streams) 2. 📤 Send URLs to your NAS with one click 3. ⬇️ Download through NAS-direct or browser-side mode for session-bound HLS/DASH streams 4. 💾 Store videos on your NAS storage ## System Architecture Chrome Extension → NAS Docker API → Worker mux/download → Video Storage └──── browser-side HLS/DASH segment upload ────┘  ### Backend Architecture  ## Quick Links

_{Chrome Extension Interface (Click to view full size)}

- **[🚀 Installation Guide](#installation)** - Complete setup instructions - **[📋 Technical Documentation](docs/)** - Architecture & specifications - **[🛠️ Developer Docs](docs/development/)** - Internal developer guide (8 chapters: getting started, architecture, chrome ext, worker pipeline, API, testing, CI/release, bug case studies) - **[🔒 Security Policy](#security)** - Security guidelines - **[🤝 Contributing](#contributing)** - How to contribute ## Key Features ### Chrome Extension - ✅ Automatic M3U8, MPD, MP4, and MOV URL detection - ✅ Deep manifest interception — detects disguised streams (e.g. `.jpg`-wrapped HLS) via fetch/XHR content inspection - ✅ One-click send to NAS - ✅ Side panel interface for easy access - ✅ Browser-side HLS/DASH mode for cookie/IP-bound streams - ✅ Live browser-side upload progress and NAS job progress - ✅ Trusted cross-site CDN allowlist with exact-host one-click add - ✅ Cookie & header forwarding for authenticated streams - ✅ Context menu integration - ✅ Configurable NAS endpoint ### NAS Docker Service - ✅ RESTful API for job management - ✅ **Multi-worker architecture** (3 workers by default in the Synology compose) for parallel processing - ✅ Browser-side staging APIs for segment upload + finalize - ✅ Multi-threaded segment downloader - ✅ FFmpeg-based video merging - ✅ Job queue with Redis - ✅ Progress tracking & notifications - ✅ Persistent storage with PostgreSQL - ✅ Periodic DB cleanup service (per-status retention + orphan partial-file removal) ## Technology Stack **Frontend:** - Chrome Extension (Manifest V3) - JavaScript ES6+ **Backend:** - Python 3.11+ (FastAPI) - FFmpeg - Redis - PostgreSQL - Docker & Docker Compose
## Project Structure webvideo2nas/ ├── chrome-extension/ # Chrome extension source ├── docs/ # User-facing documentation │ └── development/ # Developer-facing internal docs (architecture, │ # worker pipeline, testing, CI, bug case studies) ├── video-downloader/ # NAS downloader (Docker stack) │ └── docker/ # Docker services (API + Worker) ├── pics/ # Diagrams used by README └── README.md # This file ## Requirements ### For NAS - Docker & Docker Compose - 2GB+ RAM available - Storage space for videos - Network accessibility from Chrome device ### For Chrome - Chrome browser (v88+) - Developer mode enabled (for unpacked extension) ## Getting Started ### 📦 Installation **Prerequisites:** Docker 20.10+, Docker Compose v2, 2 GB+ RAM. Chrome must reach the NAS over the LAN. The actual application ships as a single multi-arch container at `ghcr.io/asdfghj1237890/webvideo2nas` (linux/amd64 + linux/arm64). The release zip contains **only the compose file** (~3 KB). #### 1. Get the compose files wget https://github.com/asdfghj1237890/WebVideo2NAS/releases/latest/download/WebVideo2NAS-downloader-docker.zip unzip WebVideo2NAS-downloader-docker.zip # → ./docker/ cd docker Pick the right compose file for your host: | Host | Run | |---|---| | **Synology NAS** | `mv docker-compose.synology.yml docker-compose.yml` | | **Anything else** (Linux / macOS / Windows Docker) | `mv docker-compose_not_synology.yml docker-compose.yml` | #### 2. Set up `.env` cp .env.example .env Edit `.env` and set the **two required** values: | Variable | How | |---|---| | `API_KEY` | `openssl rand -base64 32` — also paste this into the Chrome extension settings | | `DB_PASSWORD` | `openssl rand -base64 24` | All other variables ship with sensible defaults; comments in `.env.example` describe each (rate limiting, CORS, worker tuning, IP allowlist, SSRF guard, image tag pin). #### 3. Start the stack docker compose pull # pulls ghcr.io/asdfghj1237890/webvideo2nas:latest docker compose up -d curl -fsS -H "Authorization: Bearer YOUR_API_KEY" http://localhost:52052/api/health # → {"status":"healthy"}

Synology Container Manager (DSM UI alternative to CLI)

If you'd rather not SSH: 1. **Package Center** → install **Container Manager** (skip if already installed). 2. **File Station** — create / verify these paths and grant the project user read/write: - `/volume1/docker/video-downloader/` (project root: extract zip here, place `.env`) - `/volume1/docker/video-downloader/db_data/` (DB persistence) - `/volume1/docker/video-downloader/redis_data/` (Redis persistence) - `/volume1/docker/video-downloader/logs/` (logs) - `/volume1/video-downloader/downloads/` (downloaded videos — adjust the path to match your shared folder, and update the compose file's `volumes:` if it differs) 3. **Upload + extract** `WebVideo2NAS-downloader-docker.zip` to `/volume1/docker/video-downloader/` (gives `/volume1/docker/video-downloader/docker/`). 4. **Edit `.env`** in DSM Text Editor (or upload from PC) — set `API_KEY` + `DB_PASSWORD`. 5. **Container Manager → Projects → Create**: - Project name: `video-downloader` - Path: `/volume1/docker/video-downloader/docker` - Source: pick `docker-compose.synology.yml` - Finish the wizard — DSM auto-pulls the image from GHCR and brings everything up. 6. **Verify**: `http://YOUR_SYNOLOGY_IP:52052/api/health` (with `Authorization: Bearer ...`) returns `{"status":"healthy"}`.

#### 4. Install the Chrome extension 1. Clone the repo, or download `WebVideo2NAS-chrome-extension.zip` from the same release and unzip. 2. `chrome://extensions/` → enable **Developer mode**. 3. **Load unpacked** → select the `chrome-extension/` folder. 4. Open the extension **Settings**: - **NAS Endpoint**: `http://YOUR_NAS_IP:52052` (use the LAN IP, not `localhost`) - **API Key**: same value as `API_KEY` in `.env` 5. **Test Connection** → should say *connected*. #### Updating cd /path/to/docker-compose-folder docker compose pull docker compose up -d Synology UI: open the Project → **Action → Pull** → **Restart**. #### Common issues | Symptom | Likely cause | |---|---| | `/api/health` returns **401** | `Authorization: Bearer ` header missing or mismatched against `.env` | | Worker container shows **unhealthy** | Pre-1.9.2 templates inherit the API healthcheck. Upgrade to ≥ 1.9.2 (`docker compose pull`) — fixed compose disables the inherited check | | Synology can't write to `/downloads` | Check folder permissions in DSM File Station (project user needs read/write) | | Anything else | See [Troubleshooting](#troubleshooting) | ## Usage 1. Browse to any video streaming site 2. When a video URL (M3U8/MPD/MP4/MOV) is detected, the extension side panel lists it 3. Click extension icon to open side panel, or right-click → "Send to NAS" 4. For browser-side HLS/DASH, press play on the page first so the player issues the current session token 5. Video downloads automatically to your NAS (with cookies/headers for authenticated streams) 6. Monitor NAS-direct or browser-side upload progress in the side panel 7. Access completed videos in `/downloads/` (or `/downloads//` if a per-profile subfolder is configured) ## Configuration ### Environment Variables The full list with inline comments lives in [`.env.example`](video-downloader/docker/.env.example). The two **required** values are `API_KEY` and `DB_PASSWORD`; everything else has sensible defaults. The handful you'll most likely tune: | Variable | Default | Effect | |---|---|---| | `IMAGE_TAG` | `latest` | Pin to a specific release (e.g. `3.1.0`) instead of tracking latest | | `LOG_LEVEL` | `INFO` | `DEBUG` for verbose troubleshooting; `WARNING` to quiet down | | `MAX_DOWNLOAD_WORKERS` | `20` | Per-worker thread pool for HLS segment downloads | | `FFMPEG_THREADS` | `2` | Threads ffmpeg uses during merge | | `RATE_LIMIT_PER_MINUTE` | `10` | Per-IP API rate limit (0 disables) | | `ALLOWED_CLIENT_CIDRS` | _(empty)_ | Comma-separated CIDRs permitted to call the API; empty = no restriction | | `SSRF_GUARD` | `false` | `true` blocks downloads targeting private/loopback/link-local hosts | | `CLEANUP_INTERVAL_SECONDS` | `3600` | How often `db_cleanup` prunes finished jobs (keeps latest 100 per status: completed/failed/cancelled). Partial files for failed/cancelled are also rm'd. | ### Worker Scaling The default compose runs **3 download workers**. For higher throughput copy the `worker3` block into `worker4` / `worker5` / etc. For lower-spec hosts delete the `worker3` (or `worker2`) service. ### Extension Settings In `chrome://extensions/` → **WebVideo2NAS** → **Settings**: - **NAS Endpoint**: `http://YOUR_NAS_IP:52052` (LAN IP, not `localhost`) - **API Key**: same value as `API_KEY` in `.env` - **Auto Detect**: surfaces M3U8/MP4 URLs as you browse - **Notifications**: completion alerts ## Security ⚠️ **Important:** - **Don't expose this service directly to the public internet.** Keep it on your LAN, or behind a VPN (Tailscale, WireGuard, etc.). - Keep `API_KEY` secret. Generate strong: `openssl rand -base64 32`. Never commit `.env`. - For tighter access control, set `ALLOWED_CLIENT_CIDRS` to your LAN range and `SSRF_GUARD=true`. - Pin `IMAGE_TAG` to a specific version and review the changelog before upgrading. - Out of scope: DRM bypass, public-internet hosting, multi-tenant deployments. ### Reporting a Vulnerability Please open a [GitHub Security Advisory](https://github.com/asdfghj1237890/WebVideo2NAS/security/advisories/new). **Do not** open a public issue. When reporting, include: type of issue, affected file path / commit, reproduction steps, and (if possible) PoC and impact assessment. ## Limitations - ❌ DRM-protected content not supported - ❌ Some streaming sites use additional encryption - ❌ Requires network connectivity between Chrome and NAS - ℹ️ Download speed limited by network and NAS hardware ## Troubleshooting For first-run / install issues see the [Common issues table](#common-issues) at the end of Installation. ### Extension can't connect to NAS - `http://YOUR_NAS_IP:52052` — use the LAN IP, not `localhost` - `docker compose ps` — confirm `video_api` is `Up` and `(healthy)` - Synology / Linux firewall blocking 52052? ### Download fails - `docker compose logs -f worker` — failure reason is usually one error line - For authenticated streams: confirm the extension captured cookies for the manifest's domain (extension Settings → check the captured-headers panel) - HTTP 403/474 from segment downloads usually means the URL has expired — re-detect from a fresh page load - Disk full? `df -h /downloads` ### Slow downloads - Lower `MAX_DOWNLOAD_WORKERS` in `.env` (NAS CPU saturated) - The site may be throttling; check segment download rate in worker logs - Network: verify NAS upload bandwidth from another LAN device ## License MIT License — see [LICENSE](LICENSE). ## Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

Full Changelog (click to expand)

### [3.1.0] - 2026-05-08 #### Added - **Per-user trusted-CDN allowlist for the v3.0 browser-side gate**. The same-site safety gate refuses cross-site CDN streams (page on a brand domain, manifest on a separate CDN eTLD+1) to mitigate split-horizon-DNS / internal-CA misuse on the client. But that also blocks legitimate streams whose HMAC tokens are bound to the browser's IP — NAS-direct can't reach those either. New `chrome.storage.sync.trustedCdnSuffixes` (default empty) lets the user opt in per host suffix. Strict dotted-suffix match (`cdn.example.com` matches `media.cdn.example.com` but never `evilcdn.example.com`); hard rejections (private IP, localhost, IPv6 reserved, HTTPS-only, malformed URL) still fire — the allowlist only relaxes the same-site check. - **Sidepanel UI for trusted CDNs lives next to the detected-pane**: per-tile **`+`** button (top-right of the thumbnail, hover-reveal) derives the exact URL host and stores it in one click; trusted CDNs textbox sits in a collapsed `

` below the section label with a count badge. `+` button shows green `✓` (disabled) on tiles whose host is already covered. - **Browser-side upload progress is now surfaced live**. `segmentDownloader.runJob` emits `{track, seq, done, total}` per media segment via its `onProgress` callback; offscreen.js relays a throttled `BROWSER_JOB_PROGRESS` (200 ms; first/last events always send) to the SW; the SW broadcasts `action: 'browserJobProgress'` to the sidepanel, which keeps a `liveBrowserProgress` map and re-applies it after every `loadRecentJobs()` poll so the API's stale `progress=0` doesn't overwrite. The progress ring + percentage now move continuously through the segment-upload phase instead of staying blank until the brief finalize moment. #### Fixed - **Active browser-side jobs sorted to the bottom of the recent-jobs list**. `STATUS_RANK_ACTIVE` and `STATUS_RANK_FAILED` had no entry for `browser_pending` / `browser_uploading` / `browser_finalizing`, so they fell through to the `?? 99` rank fallback. Added the three states alongside their NAS-direct counterparts (`browser_uploading` ↔ `downloading`, `browser_finalizing` ↔ `merging`, `browser_pending` ↔ `pending`). #### Notes - The trusted-CDN allowlist also relaxes the master-URL CORS-relax DNR decision (`masterTrustedForDnr`) — without that, the manifest response comes back opaque cross-origin and `_wv2nasFetchManifestInBrowser` can't read it as `manifest_text`. - The variant-URL trust check (master → variant) stays strict regardless of allowlist — that boundary is structural integrity, not user-config. Otherwise an allowlisted master could surface variants on attacker-controlled hosts that share an unrelated allowlisted suffix. - The progress pipeline only writes to live state when the job exists in the local `jobs` array AND has status `browser_*`; pruning runs on every `loadRecentJobs()` to bound the in-memory map. - 22 new tests in this release: 16 covering the trusted-CDN matcher / gate / typosquat boundary, 1 covering one-click trusted-CDN host derivation, 5 covering `runJob.onProgress` (done/total counters, init-segment exclusion, multi-track flatten, callback-throws-but-upload-continues, omitted-callback). 300/300 green. ### [3.0.0] - 2026-05-07 #### Added - **Browser-side HLS/DASH pipeline** for streams whose tokens or cookies are bound to the user's browser session (signed CDN URLs, paid-streaming sites, anything where NAS-direct gets 401/403). The extension now fetches the manifest, downloads each media segment, AES-128-decrypts where applicable, and uploads each segment to a per-job NAS staging directory; the worker is reduced to ffmpeg mux only. A new offscreen document owns the long-lived fetch loop so it survives the SW's 30-second idle eviction; `chrome.declarativeNetRequest` session rules spoof Referer / Origin / User-Agent + relax CORS per-host so credentialed segment fetches behave like the original player. - **Same-site safety gate (`_wv2nasIsManifestUrlSafeForBrowser`)** on every URL the extension is about to fetch in browser context. Rejects HTTPS-only failures, private/loopback/link-local/CGN/TEST-NET IPv4 + IPv6 reserved literals, localhost, malformed URLs, AND DNS hostnames that aren't same-site with the page that surfaced them (split-horizon-DNS / internal-CA mitigation — a public-looking name on a corporate machine could resolve to intranet content the browser would then post to NAS as `manifest_text`). 14 rounds of Codex adversarial-review hardened the gate against typosquats, redirect-following, variant URLs on different sites, oversize-manifest DoS, AES-key URI off-CDN, and per-URL header scoping leaks. - **Server-side always-on `_enforce_plan_url_safety`** at `/api/jobs/init` walks every URL in the plan (segment / init / AES-key URIs) and rejects the whole plan if any points at a non-public address or non-http(s) scheme. Always on regardless of `SSRF_GUARD` env var (which only protects `/api/download`). - **DB schema**: new `mode` (`'nas_direct' | 'browser'`) + `staging_dir` columns on `job_metadata`; new statuses `browser_pending` / `browser_uploading` / `browser_finalizing`. Stale-browser-job reaper at worker boot (>6 h pre-finalize → failed). DNR-rule slots are persisted per active job and recovered on SW restart so a SW eviction mid-upload doesn't strand session rules. #### Notes - This is the v2.5.x design landing as a major release. The default for `useBrowserSide` flipped to ON for HLS/DASH (MP4 still goes NAS-direct because there's no payoff for a single GET). - See [docs/development/03-chrome-extension.md](docs/development/03-chrome-extension.md#8-browser-side-pipeline-v30) for the request/response flow + state machine. ### [2.3.9] - 2026-05-05 #### Fixed - **Hidden-mode nav badge displayed `100` overflowing the supposedly-circular pill shape** when AV-task history hit the cap. Root cause was structural: `.nav-count` was a default `display: inline` `` inheriting `line-height: 1.6` from `.nav-item`, so on `padding: 1px 7px` + `font-size: 12px` the rendered text box was 19.2 px tall while the padded container was only ~14 px, pushing characters outside the rounded border. The "circle" only looked circular for 1-2 digit content because flex centering masked the overflow at small widths; "100" exposed the broken layout. Two-part fix: (1) `.nav-count` rebuilt with `display: inline-flex` + `align-items/justify-content: center` + explicit `height: 18px` + `line-height: 1` + `flex-shrink: 0`, so the badge owns its layout instead of inheriting from the parent, and (2) `options.js` caps the displayed value at `99+` once `rows.length >= 100`, so the badge never has to render literal 3-digit counts. #### Changed - **`AV_HISTORY_MAX` bumped from 100 to 200** in `chrome-extension/background.js`. With the badge now handling 3-digit counts cleanly, doubling the cap gives users meaningful history headroom. FIFO trim behaviour (newest unshifted, list capped to MAX) unchanged. Extension manifest: `2.3.1` → `2.3.9` (catching up several releases of stale version string). ### [2.3.8] - 2026-05-05 #### Fixed - **`backfill_suspect.py` crashed with `TypeError: ... got multiple values for argument 'declared_duration'`** the moment it tried to evaluate the first job. The script's `_Shim` class hoisted `DownloadWorker._compute_suspect_reason` (a `@staticmethod`) as an instance attribute, then accessed it through an instance — Python's descriptor protocol re-bound the function as a regular method, so calling `shim._compute_suspect_reason(declared_duration=...)` slotted `shim` into the first positional argument, conflicting with the explicit `declared_duration=` kwarg. Fix: drop the `_Shim` wrapper entirely; both helpers (`_probe_duration_seconds` and `_compute_suspect_reason`) are called directly off `DownloadWorker`. Also converted `_probe_duration_seconds` to `@staticmethod` (it never used `self` anyway, and is now consistent with the `_probe_duration_float` helper added in v2.3.5). ### [2.3.7] - 2026-05-05 #### Fixed - **CI red after v2.3.6** because the new `merge()` test read `stdin.getvalue()` on a `BytesIO` that the production code had already closed (`merge()` closes stdin once it's done streaming segments through). Test now uses a tiny `_CapturingBytesIO` subclass that snapshots its contents into a `.captured` attribute on `close()`, so the test reads the captured snapshot regardless of stream state. Local `vitest`/`pytest` runs all green; CI back to green. ### [2.3.6] - 2026-05-05 #### Fixed - **HLS merge produced ~half-length mp4 even when every segment downloaded successfully**. Worker would log `Download complete: 1216/1216 segments successful`, ffmpeg merge returned `returncode == 0`, the output mp4 was ~770 MB — but actual playback duration was 3158 s instead of the m3u8's declared 7299 s (~43%). Root cause: the old merge command used ffmpeg's **concat demuxer** (`-f concat -safe 0 -i list.txt -c copy`) without explicit `duration` directives in the list. Each `.ts` segment's internal PTS started from 0 (HLS-spec-legal), so the concat demuxer had to compute cross-segment offsets from each input's reported timestamps + heuristics. On certain streams the heuristic miscomputed offsets, producing PTS overlap between consecutive segments → mp4 muxer (which requires monotonic PTS) silently dropped every "time-reversed" packet. No error, no warning, just half the output. Fixed by replacing concat demuxer with byte-concatenation through ffmpeg's stdin: `ffmpeg -f mpegts -i pipe:0 -c copy -bsf:a aac_adtstoasc out.mp4`, with the worker writing each segment's bytes (1 MB chunks via `shutil.copyfileobj`) into ffmpeg's stdin in order. MPEG-TS is byte-concatenable by design (188-byte packet stream), so ffmpeg sees a single continuous stream and never has to compute offsets. Implementation includes background drain threads on stdout/stderr to avoid pipe-buffer deadlock and a 15-minute timeout safety net. The existing `merge_with_re_encode` fallback (concat demuxer + transcode) is preserved as a final safety path — transcode regenerates PTS so the demuxer bug can't manifest there. #### Notes - Detailed root-cause walkthrough and decision rationale documented in [`docs/development/08-bug-case-studies.md`](docs/development/08-bug-case-studies.md) §1. - Pre-v2.3.6 jobs may have suffered the same silent truncation. The existing `backfill_suspect.py` tool (re)probes their actual duration vs declared and flags ratio < 0.85 as suspect, surfacing in the chrome sidepanel with a Re-fetch button. ### [2.3.5] - 2026-05-05 #### Added - **Worker diagnostic logs for the HLS download / decryption pipeline**, to make root-causing future "downloaded but wrong" cases tractable without round-tripping through extra reproductions. Two new instrumentation points: (1) `_get_key_bytes` now logs the AES-128 key endpoint's Content-Type, length, and full hex, plus a WARNING when all 16 bytes fall in the printable-ASCII range (sometimes legitimate — some hosts genuinely use ASCII-text keys — but worth surfacing as it's an unusual key shape worth eyeballing); (2) new `_diagnose_segment_durations` runs after segment download finishes, sample-probes 5 segments (start, 25%, 50%, 75%, end) with ffprobe and compares the actually-decoded duration to the m3u8's `#EXTINF` declaration. Both are pure observability — neither fails the job — and together they give enough signal to distinguish "decryption broken", "individual segments wrong", and "merge step lost data" in a handful of log lines. #### Changed - **Reverted the v2.3.4 worker-side heuristic relaxation** that would have suppressed the `actual_duration < 0.85 * declared_duration` SUSPECT flag for jobs that had downloaded 100% of segments. The flag is the last line of defence against silent truncation in the worker pipeline — relaxing it without a compensating signal turned out to mask a real partial-output case. v2.3.5's diagnostics replace it as the primary debugging aid; the SUSPECT heuristic remains in its v2.3.3 form. The chrome-extension cross-tab fix from v2.3.4 is unaffected — that part stays. ### [2.3.4] - 2026-05-05 #### Fixed - **Cross-tab URL leak when sending detected videos**. Sidepanel's `sendToNAS` now passes the source tab's `tabId` along with the URL, and the extension's `findBestCapturedEntry` substitution logic hard-filters captured headers by `entry.tabId === sourceTabId`. Previously the captured-header picker scored entries by URL-origin match alone, which on multi-tab sessions of the same site (the canonical multi-tab use case for this extension) didn't disambiguate between tabs — every captured manifest from any tab on the same origin scored equally and the most-recent timestamp tiebreaker silently rewrote the user's clicked URL to whichever same-origin tab's video was most recently played, sending the wrong video to NAS. Tab id is intrinsic to the network capture (Chrome's webRequest events carry it) and survives redirects, so it's the definitive same-tab signal. When the caller doesn't supply a tab id (e.g. orphan / service-worker capture path), the function falls back to strict initiator equality — still tighter than the old origin-prefix scoring. Helper hoisted to module scope for testability + 3 new vitest regression tests covering the multi-tab scenario, orphan fallback, and within-tab clean-URL → tokenized-variant substitution. ### [2.3.3] - 2026-05-05 #### Added - **Third worker container (`worker3`)** in `docker-compose.synology.yml`, identical to `worker` / `worker2`, sharing the same image and Redis queue. Three workers means roughly 3× concurrent download throughput — bounded by NAS network and disk I/O rather than the worker process itself. - **`db_cleanup` service**: a new lightweight container running an `sh` loop that every `CLEANUP_INTERVAL_SECONDS` (default 3600) prunes the `jobs` table and removes orphaned partial files on disk. Keeps the latest 100 jobs **per terminal status** (`completed`, `failed`, `cancelled`) — so a long-running deployment doesn't accumulate unbounded historical rows that slow down the sidepanel's `/api/jobs?limit=...` query. Also `rm`'s `file_path` for `failed`/`cancelled` rows that left partial mp4 fragments under `/downloads`. #### Changed - **Worker startup now reaps zombie jobs** at boot. New `_reap_zombie_jobs()` runs once when worker container starts, marks any job in `downloading`/`processing` status with `started_at > 2 hours ago` as `failed` with reason "Worker restarted while job was in progress (zombie reaped after 2h)". Idempotent across multiple workers booting simultaneously (PG row locks serialise; second writer sees no matching rows). The 2-hour floor protects legitimately long HLS jobs from being clobbered when a sibling worker restarts. - **Cancellation also cleans up the partial output** instead of leaving a half-merged file behind for `db_cleanup` to find later. Worker checks `is_job_cancelled()` at every transition (post-download, pre-merge, post-merge), unlinks the partial file before raising. ### [2.3.2] - 2026-05-05 #### Fixed - **HLS download progress callback wrote to the DB on every segment** (1216 segments → 1216 `UPDATE jobs SET progress=...` round-trips per job). The v2.3.1 throttle covered the worker→DB write but not the per-segment callback that triggered it; this fix brings them in sync. Progress writes now happen at most every 2 s, plus a final guaranteed write when the job completes 100 % so the user never sees a stuck progress bar. Reduces DB load on the typical multi-thousand-segment HLS download by ~600×. ### [2.3.1] - 2026-05-05 #### Changed - **Tighter progress refresh** for active downloads. Worker reports progress every ~2 s (was variable, sometimes 5–10 s gaps when ffmpeg merge was running silently). Sidepanel polls `/api/jobs?limit=20` every 2 s (was 5 s). Net result: download bars update smoothly instead of feeling stuck during long segment phases. Both ends throttled symmetrically so neither side hits API rate limits even with multiple concurrent in-flight jobs. ### [2.3.0] - 2026-05-04 #### Added - **AV-task gets a secondary-source fallback when the primary URL template doesn't produce a manifest.** New two-phase pipeline in `handleAvTaskFetch`: **phase 1** is the original v2.2.0 behaviour — opens the user-configured `hidden_mode.url_template` in a **background tab**, fully automatic, the site's own JS produces a signed m3u8 and the existing detection pipeline ships it to NAS without bothering the user. **Phase 2** only kicks in if phase 1 times out (no manifest in 60 s): opens a hardcoded secondary search site in an **active foreground tab** so the user can click the download button and solve any required CAPTCHA — the resulting signed mp4 request is picked up by the same `maybeFireAvTaskAutoSend → sendToNAS` path and shipped as a single direct mp4 (no HLS, no segment auth, no cookies). The history row stays `pending` across the transition; its `url` field updates from primary → secondary so the table reflects which site is currently being attempted. #### Changed - **`maybeFireAvTaskAutoSend` gained a phase-aware URL filter**: during the secondary phase, only URLs whose hostname is on the secondary CDN (e.g. `dl*.example.com` patterns rather than the apex that serves the page chrome) qualify for auto-send. Without this filter the play page's preview-clip mp4s would race the real download and ship a 30-second teaser to the NAS. The primary phase keeps the original "first eligible URL wins" behaviour. - **Primary phase fast-fails on HTTP 4xx/5xx** instead of waiting the full 60 s timeout. New `chrome.webRequest.onHeadersReceived` listener watches the helper tab's `main_frame` response — if the status code is ≥ 400 (e.g. the page returns 404 because the code's path doesn't exist on the primary site), the tab is closed immediately and the secondary fallback opens within milliseconds rather than after a minute of dead air. Filtered to `main_frame` only so that ad subframes and API errors don't trigger spurious failovers. The secondary phase isn't covered — its search page returns 200 with empty results when a code is unknown, so HTTP status can't classify success/failure there. #### Notes - **Order is "automatic-first, manual-fallback"** by design: most codes resolve on the primary site inside the first ~5–15 s without the user noticing anything happened, and the secondary site only intrudes (active tab popping to the front) for the codes that the primary genuinely can't deliver. Worst-case end-to-end timeout is 120 s (60 s primary + 60 s secondary) when the primary stalls without erroring, but a 404 now flips to secondary in under a second. - **Phase 2 is not "hidden"** — the secondary tab opens in foreground because CAPTCHA solving requires user interaction (Anthropic safety policy forbids auto-solving CAPTCHAs, and the secondary site doesn't expose the signed download without it). If the user isn't around to solve the captcha, the 60 s timer expires and the row is marked `failed`. - The secondary site's URL pattern is hardcoded — no new option added. The existing `hidden_mode.url_template` setting still controls the primary phase, so swapping the primary site (or pointing it at a 404 to force the secondary every time) remains a one-line config change in `hidden_mode.toml`. - The signed secondary mp4 carries an `expires=` query param — a few minutes in the future when issued. The pipeline forwards the URL to NAS within ~4 s of capture (existing `AV_TASK_AUTOCLOSE_DELAY_MS` window), well inside the validity. If the NAS queue is backed up enough that the URL expires before the worker dequeues, the existing v2.2.3 anti-hotlink Re-fetch path takes over (re-open the secondary tab, solve captcha again, resend). ### [2.2.3] - 2026-05-04 #### Changed - **Anti-hotlink failures now surface a Re-fetch button** instead of silently sitting in `failed`. When the worker aborts a job because the CDN started serving anti-hotlink PNG placeholders mid-download (signed-URL token expired, or the session/cookie context the CDN wanted got invalidated), the sidepanel now renders the same warm-tone Re-fetch block the suspect-file path uses (v2.1.22) — one click re-opens the original `source_page` in a new tab, the extension grabs fresh m3u8 + segment tokens, and the user resends as normal. Re-uses the existing `source_page` field on `job_metadata` (already populated since v2.1.22), the existing button + toast UI, and the same `suspect.refetch.*` i18n strings. Adds a new label key `suspect.label.refetch` ("Re-fetch needed" / "需要重新抓取" / "需要重新抓取") because the original `suspect.label` ("Probably wrong") is a misnomer for a job that never produced a file. - **Anti-hotlinking errors now classify as `error.tokenExpired`** in the failed-job error-details block (was `error.generic`). The recovery is identical to a CDN-token-expiry — refresh the source page and resend — so the existing tokenExpired solution copy applies cleanly. No new i18n needed. - **Worker stops retrying segments once it confirms the response is an anti-hotlink placeholder**: the `download_segment` retry loop now short-circuits when the failure message contains `anti-hotlinking`, since retrying the same URL with the same session and same auth produces the same PNG. Saves ~16 dead HTTP requests per segment and lets the existing 5-segment hotlink-count guard (`worker.py:1152`) trip in <1s instead of ~4s. The other retry paths (timeouts, transient 5xx, etc.) still get the full 3 retries with exponential backoff. #### Notes - The Re-fetch button only renders when `job.source_page` is recorded — pre-v2.1.22 jobs without a captured source URL still surface the error-details block but no actionable button (same as before for any job with no source_page). - The new `isHotlinkFail` detection in `sidepanel.js` is a substring match on the worker's existing abort message ("Download aborted: Server blocked segment downloads (anti-hotlinking protection)…"). No worker→extension contract change. ### [2.2.2] - 2026-05-04 #### Changed - **AV-task auto-send now uses the source page's `