feat: consolidate on the shadcn frontend and harden playback

Frontend - Replace the Ant Design app with the shadcn/ui app as the single frontend, keeping the portfolio side-by-side revertible in this commit. - Drop the abandoned mdui experiment and the AntD tree so there is one app, one router and one design system to maintain. Live playback - Stop racing the encoder. lowLatencyMode + liveSyncDurationCount:1 targeted a single 4s segment, so playback ran at up to 1.10x and the forward buffer bottomed out at 0.1s, re-buffering every segment. Measured over 40s: buffer min 0.1s -> 8.9s, rate 1.00 constant, 0 stalls. - Remove the 落后 labels and the page-level edge-snapping timer that fought hls.js over two different definitions of "latency". - Live can no longer be paused: the control is not rendered while it plays, so the icon can never misreport state. - Fix autoplay: muted autoplay started silently and never unmuted, and the gesture that restored sound also toggled playback off. One owner now. Source ("原画") rendition - Publish the publisher's stream remuxed losslessly via -c:v copy alongside the transcoded ones, so a source option exists even when every ladder entry downsizes. - Give renditions an ASCII path slug: the HLS path is part of the URL and MediaMTX rejects non-alphanumeric paths, so 原画 could never be served. Backend - Supervise MediaMTX from the backend: start it at boot and restart it on exit, never killing an origin it did not start. - Clear stale HLS output before restarting an encoder; an ENDLIST-less playlist left by a dead encoder hangs hls.js forever. - Mark JSON timestamps as UTC. utcnow().isoformat() emitted a naive string that JS read as local time, so every viewer heartbeat looked 8 hours old in UTC+8 and the live board's active-viewer count was permanently zero. - Rebind a viewer session when it arrives on another stream instead of failing, and survive the StrictMode double-heartbeat race on first insert. - Replace the Linux-only /proc and os.getloadavg metrics with psutil. - Complete MediaMTX's cookieCheck redirect inside /hls proxying, and hold the httpx client open for the streamed body. Housekeeping - Ignore RainOSS (it stores accessKeySecret), runtime output, TLS keys and .venv; remove generated screenshots, load test scripts and stray files. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-09-22 18:07:56 +08:00
1 parent 186e32caf1
commit 996e3fea39
@@ -22,8 +22,16 @@ class MediaMTXService:
                if resp.status_code == 200:
                    data = resp.json()
                    for item in data.get("items", []):
-                        if item.get("name") == stream_key:
-                            return {"ready": item.get("ready", False), "source_ready": item.get("sourceReady", False)}
+                        # Paths are named after the RTMP URL key, so a publisher on
+                        # rtmp://host/live/<key> shows up as "live/<key>", never as
+                        # the bare key. This MediaMTX exposes the reader list itself
+                        # rather than a readersCount/sourceReady pair.
+                        if item.get("name") in (stream_key, f"live/{stream_key}"):
+                            return {
+                                "ready": item.get("ready", False),
+                                "source_ready": bool(item.get("source")),
+                                "readers": len(item.get("readers") or []),
+                            }
        except Exception:
            return None
        return None
@@ -0,0 +1,158 @@
+import asyncio
+import os
+import shutil
+import subprocess
+
+import httpx
+
+from config import settings
+
+
+def _log(message: str) -> None:
+    print(f"[mediamtx] {message}", flush=True)
+
+
+def resolve_binary(configured: str) -> str | None:
+    """Find the MediaMTX executable: the configured name on PATH, else the repo copy.
+
+    The repository ships both an ELF `mediamtx` and a Windows `mediamtx.exe`, so a
+    bare-name fallback has to prefer the platform's own extension first — picking
+    the other one fails at exec time with "not a valid Win32 application".
+    """
+    found = shutil.which(configured)
+    if found:
+        return found
+    if os.path.dirname(configured):
+        return configured if os.path.isfile(configured) else None
+
+    repo_dir = os.path.dirname(os.path.abspath(settings.media_origin_config))
+    candidates = [f"{configured}.exe", configured] if os.name == "nt" else [configured, f"{configured}.exe"]
+    for candidate in candidates:
+        path = os.path.join(repo_dir, candidate)
+        if os.path.isfile(path):
+            return path
+    return None
+
+
+class MediaMTXManager:
+    """Keeps the MediaMTX origin alive.
+
+    The backend owns the media origin: it is the only thing that can tell whether
+    the origin is actually serving (its own /v3 API), and RTMP ingest is dead
+    without it while transcoders keep burning CPU against a missing source. When
+    an external supervisor already runs MediaMTX, the health probe below sees it
+    and this manager stays out of the way.
+
+    Only the process this manager started is ever stopped, so an externally
+    managed origin is never killed on shutdown.
+    """
+
+    def __init__(self) -> None:
+        self._process: subprocess.Popen | None = None
+        self._failures = 0
+        self._binary_missing_logged = False
+
+    async def is_healthy(self) -> bool:
+        """True when the origin API answers, regardless of who started it."""
+        try:
+            async with httpx.AsyncClient(timeout=2) as client:
+                response = await client.get(
+                    f"http://{settings.media_origin_host}:{settings.media_origin_api_port}/v3/paths/list"
+                )
+                return response.status_code == 200
+        except Exception:
+            return False
+
+    def _started_process_alive(self) -> bool:
+        return self._process is not None and self._process.poll() is None
+
+    def start(self) -> None:
+        """Launch MediaMTX unless one is already running. Never raises."""
+        if self._process is not None:
+            # A process we started exited; clear it so the next tick can retry.
+            if self._process.poll() is not None:
+                _log(f"process exited with code {self._process.returncode}")
+                self._process = None
+            else:
+                return
+
+        binary = resolve_binary(settings.media_origin_binary)
+        if binary is None:
+            # No MediaMTX anywhere is a deployment gap, not a crash: the API still
+            # serves, the admin UI can explain it, and RTMP ingest is simply down.
+            if not self._binary_missing_logged:
+                _log(f"'{settings.media_origin_binary}' not found on PATH or in mediamtx/; RTMP ingest unavailable")
+                self._binary_missing_logged = True
+            return
+
+        config_path = settings.media_origin_config
+        command = [binary]
+        if config_path and os.path.isfile(config_path):
+            command.append(config_path)
+        else:
+            _log(f"config not found at {config_path}; running with defaults")
+            config_path = ""
+
+        try:
+            self._process = subprocess.Popen(
+                command,
+                cwd=os.path.dirname(os.path.abspath(config_path)) if config_path else None,
+                stdout=subprocess.DEVNULL,
+                stderr=subprocess.DEVNULL,
+            )
+        except FileNotFoundError:
+            if not self._binary_missing_logged:
+                _log(f"failed to launch {binary}; RTMP ingest unavailable")
+                self._binary_missing_logged = True
+            self._process = None
+        except Exception as error:  # pragma: no cover - defensive
+            _log(f"failed to start: {error!r}")
+            self._process = None
+
+    async def ensure_running(self) -> bool:
+        """Start the origin if it is neither healthy nor already running.
+
+        Returns True when this call started it.
+        """
+        if not settings.media_origin_autostart:
+            return False
+        if await self.is_healthy() or self._started_process_alive():
+            self._failures = 0
+            return False
+        # Back off after repeated failures so a broken binary cannot be relaunched
+        # every supervisor tick, which would spam processes and logs.
+        if self._failures:
+            self._failures -= 1
+            return False
+        self.start()
+        if self._process is None:
+            self._failures = 12
+            return False
+        self._failures = 0
+        _log(f"started (pid {self._process.pid})")
+        return True
+
+    async def wait_until_ready(self, timeout: float = 15.0) -> bool:
+        deadline = asyncio.get_running_loop().time() + timeout
+        while asyncio.get_running_loop().time() < deadline:
+            if await self.is_healthy():
+                return True
+            await asyncio.sleep(0.5)
+        return False
+
+    def stop(self) -> None:
+        """Stop only the process this manager started."""
+        process, self._process = self._process, None
+        if process is None or process.poll() is not None:
+            return
+        try:
+            process.terminate()
+            process.wait(timeout=5)
+        except Exception:
+            try:
+                process.kill()
+            except Exception:
+                pass
+
+
+mediamtx_mgr = MediaMTXManager()
@@ -7,9 +7,24 @@ from services.multicast_detector import multicast_detector
 from services.transcoder import transcode_mgr
 from config import settings
 import json
+import re
 from datetime import datetime


+# MediaMTX rejects any HLS path that is not alphanumeric ("invalid path name: can
+# contain only alphanumeric characters"), and the label is also used as a directory
+# name, so every rendition needs a path-safe slug of its own.
+def hls_path_slug(label: str) -> str:
+    """Filesystem- and MediaMTX-safe rendition path for a display label."""
+    slug = re.sub(r"[^A-Za-z0-9]", "", label or "")
+    return slug or "rendition"
+
+
+def with_hls_path(rendition: dict) -> dict:
+    """Attach the `path` an admin-configured rendition is stored and served under."""
+    return {**rendition, "path": hls_path_slug(rendition.get("label", ""))}
+
+
 class StreamService:
    def __init__(self, db: AsyncSession):
        self.db = db
@@ -25,6 +40,56 @@ class StreamService:
                pass
        return settings.transcode_profiles.get(stream.transcode_profile, settings.qualities)

+    @staticmethod
+    def source_label() -> str:
+        """Display name of the stream-copy rendition, or '' when it is disabled.
+
+        A blank label would leak into the HLS path and produce a URL like
+        /hls/<key>//index.m3u8, so it degrades to "off" instead.
+        """
+        return settings.source_rendition_label.strip() if settings.source_rendition_enabled else ""
+
+    @classmethod
+    def source_slug(cls) -> str:
+        """Path segment for that rendition, safe for MediaMTX and the filesystem."""
+        if not cls.source_label():
+            return ""
+        return hls_path_slug(settings.source_rendition_slug)
+
+    def watch_renditions(self, stream: Stream) -> list:
+        """Renditions a viewer may pick: the stream-copy rendition first, then the configured ones.
+
+        Each entry carries both the label shown to viewers and the `path` used in
+        HLS URLs — MediaMTX only accepts alphanumeric HLS paths, so a Chinese label
+        such as "原画" cannot double as a directory name.
+
+        The source entry carries no size: the input resolution is only known to
+        ffmpeg, so the player measures it from the media it ends up decoding.
+        """
+        renditions = [with_hls_path(r) for r in self.effective_renditions(stream)]
+        label, slug = self.source_label(), self.source_slug()
+        if not label or not slug or not renditions or renditions[0]["path"] == slug:
+            return renditions
+        return [{"label": label, "path": slug, "width": None, "height": None, "copy": True}] + renditions
+
+    def transcode_renditions(self, stream: Stream) -> list:
+        """Renditions to hand to the transcoder, including the stream-copy source rendition.
+
+        Carries the same `path` values as watch_renditions so the files ffmpeg
+        writes are the files the player asks for.
+
+        The copy entry is the top rendition with its label replaced, so it carries
+        every key the transcoder reads. ffmpeg ignores those numbers for a stream
+        copy, but the supervisor in main.py compares TranscodeManager's recorded
+        signature against this list every tick — an entry missing a field would
+        both crash the start and make the signature drift.
+        """
+        renditions = [with_hls_path(r) for r in self.effective_renditions(stream)]
+        label, slug = self.source_label(), self.source_slug()
+        if not label or not slug or not renditions or renditions[0]["path"] == slug:
+            return renditions
+        return [{**renditions[0], "label": label, "path": slug, "copy": True}] + renditions
+
    async def create_stream(self, title: str, description: str | None = None,
                            cover_url: str | None = None,
                            scheduled_at: str | None = None,
@@ -32,6 +97,7 @@ class StreamService:
                            replay_url: str | None = None,
                            chapters: list[dict] | None = None,
                            transcode_profile: str = "standard",
+                            renditions_json: str | None = None,
                            homepage_visible: bool = True,
                            remote_control_enabled: bool = False) -> Stream:
        import uuid
@@ -49,6 +115,7 @@ class StreamService:
            replay_url=replay_url,
            chapters_json=json.dumps(chapters or [], ensure_ascii=False),
            transcode_profile=transcode_profile,
+            renditions_json=renditions_json,
            homepage_visible=homepage_visible,
            remote_control_enabled=remote_control_enabled,
            stream_key=uuid.uuid4().hex,
@@ -108,7 +175,7 @@ class StreamService:
                started_at=datetime.utcnow(),
            )
            rtmp_url = media_service.get_rtmp_url(stream_key)
-            transcode_mgr.start_transcode(stream_key, rtmp_url, self.effective_renditions(stream))
+            transcode_mgr.start_transcode(stream_key, rtmp_url, self.transcode_renditions(stream))

    async def set_stream_ended(self, stream_key: str):
        stream = await self.get_stream_by_key(stream_key)
@@ -125,7 +192,7 @@ class StreamService:
        stream = await self.get_stream_by_key(stream_key)
        if not stream:
            return {"lines": [], "qualities": [], "multicast_available": False, "selected_node": None}
-        qualities = self.effective_renditions(stream)
+        qualities = self.watch_renditions(stream)
        best_node = lb.select_best_node(client_ip)
        if not best_node:
            best_node = lb.get_enabled_nodes()[0] if lb.get_enabled_nodes() else None
@@ -142,12 +209,13 @@ class StreamService:
                "qualities": [],
            }
            for quality in qualities:
-                label = quality["label"]
+                # The URL segment is the rendition's path slug, not its label:
+                # labels are display text and may be Chinese.
                line_info["qualities"].append({
-                    "label": label,
+                    "label": quality["label"],
                    "width": quality["width"],
                    "height": quality["height"],
-                    "url": f"{base_hls}/{label}/index.m3u8",
+                    "url": f"{base_hls}/{quality['path']}/index.m3u8",
                })
            lines.append(line_info)

@@ -5,15 +5,67 @@ from config import settings

 class TranscodeManager:
    _processes: dict[str, list[subprocess.Popen]] = {}
+    # Signature of the last successful start, so a supervisor can tell whether a
+    # running set of processes still matches the stream's current configuration.
+    _signatures: dict[str, str] = {}
+    _failed_starts: dict[str, int] = {}
+
+    @staticmethod
+    def _signature(qualities: list[dict]) -> str:
+        return "|".join(
+            f"{q.get('label')}:{q.get('width')}x{q.get('height')}:{q.get('bitrate')}:"
+            f"{'copy' if q.get('copy') else q.get('codec', 'libx264')}"
+            for q in qualities
+        )
+
+    def is_running(self, stream_key: str) -> bool:
+        """True when at least one transcode process exists and none have exited."""
+        procs = self._processes.get(stream_key)
+        if not procs:
+            return False
+        return any(proc.poll() is None for proc in procs)
+
+    def needs_restart(self, stream_key: str, qualities: list[dict]) -> bool:
+        if not qualities:
+            return False
+        return self._signatures.get(stream_key) != self._signature(qualities) or not self.is_running(stream_key)
+
+    def _clear_stale_output(self, hls_dir: str) -> None:
+        """Drop the previous run's playlist and segments.
+
+        ffmpeg writes a live playlist with no #EXT-X-ENDLIST, so a playlist left
+        behind by an encoder that died looks to hls.js exactly like a stream that
+        simply stopped: it keeps reloading a file that will never change and
+        buffers forever. Removing it makes the player see a missing manifest
+        instead, which it retries until the supervisor has the encoder back up.
+        """
+        if not os.path.isdir(hls_dir):
+            return
+        for name in os.listdir(hls_dir):
+            if name.endswith(".ts") or name.endswith(".m3u8") or name.endswith(".m3u8.tmp"):
+                try:
+                    os.remove(os.path.join(hls_dir, name))
+                except OSError:
+                    # A player holding the file open (Windows) is fine: it will be
+                    # overwritten by the restarting encoder.
+                    pass

    def start_transcode(self, stream_key: str, input_url: str, qualities: list[dict]):
-        if stream_key in self._processes:
+        if stream_key in self._processes and self.is_running(stream_key):
            return
+        self.stop_transcode(stream_key)

        procs = []
        segment_seconds = max(1, settings.hls_segment_seconds)
        for quality in qualities:
            label = quality["label"]
+            # The stream-copy ("原画") rendition carries the top rendition's numbers
+            # only so the signature stays complete; ffmpeg copies the input rather
+            # than encoding to them. See StreamService.transcode_renditions.
+            is_copy = bool(quality.get("copy"))
+            # Files land under the rendition's path slug, never its label: MediaMTX
+            # refuses non-alphanumeric HLS paths and the label may be Chinese.
+            path_slug = quality.get("path") or label
            width = quality["width"]
            height = quality["height"]
            fps = quality.get("fps", 30)
@@ -24,20 +76,10 @@ class TranscodeManager:
            audio_bitrate = quality.get("audio_bitrate", "128k")
            keyframe_seconds = quality.get("keyframe_seconds", segment_seconds)
            keyframe_frames = keyframe_seconds * fps
-            hls_dir = os.path.join(settings.hls_runtime_dir, stream_key, label)
+            hls_dir = os.path.join(settings.hls_runtime_dir, stream_key, path_slug)
            os.makedirs(hls_dir, exist_ok=True)
-            cmd = [
-                "ffmpeg", "-y",
-                "-i", input_url,
-                "-c:v", codec, "-preset", preset,
-                "-r", str(fps), "-b:v", bitrate, "-crf", str(crf),
-                "-s", f"{width}x{height}",
-                "-sc_threshold", "0",
-                "-g", str(keyframe_frames),
-                "-keyint_min", str(keyframe_frames),
-                "-force_key_frames", f"expr:gte(t,n_forced*{keyframe_seconds})",
-                "-copyts", "-start_at_zero",
-                "-c:a", "aac", "-b:a", audio_bitrate,
+            self._clear_stale_output(hls_dir)
+            hls_output = [
                "-f", "hls",
                "-hls_time", str(segment_seconds),
                "-hls_list_size", "6",
@@ -45,6 +87,44 @@ class TranscodeManager:
                "-hls_segment_filename", os.path.join(hls_dir, "segment_%03d.ts"),
                os.path.join(hls_dir, "index.m3u8"),
            ]
+            # NOTE: ffmpeg's -reconnect* flags are HTTP/HTTPS-only protocol
+            # options and make RTMP inputs fail with "Option not found".
+            # RTMP drop recovery is handled by the supervisor in main.py,
+            # which restarts exited encoders on its next tick.
+            hls_input = ["-rw_timeout", "10000000", "-i", input_url]
+            if is_copy:
+                # Source ("原画") rendition: keep the publisher's video as-is and only
+                # re-wrap it, so the output keeps the exact incoming resolution,
+                # frame rate and bitrate. Video copy costs no CPU; audio is re-encoded
+                # because MPEG-TS cannot carry every codec a publisher might send, and
+                # one AAC track is a rounding error next to the video encoders.
+                cmd = (
+                    ["ffmpeg", "-y"] + hls_input
+                    + [
+                        "-c:v", "copy",
+                        "-c:a", "aac", "-b:a", "192k",
+                        "-copyts", "-start_at_zero",
+                        "-map", "0:v:0", "-map", "0:a:0?",
+                    ]
+                    + hls_output
+                )
+            else:
+                cmd = (
+                    ["ffmpeg", "-y"] + hls_input
+                    + [
+                        "-c:v", codec, "-preset", preset,
+                        "-r", str(fps), "-b:v", bitrate, "-crf", str(crf),
+                        "-s", f"{width}x{height}",
+                        "-sc_threshold", "0",
+                        "-g", str(keyframe_frames),
+                        "-keyint_min", str(keyframe_frames),
+                        "-force_key_frames", f"expr:gte(t,n_forced*{keyframe_seconds})",
+                        "-copyts", "-start_at_zero",
+                        "-c:a", "aac", "-b:a", audio_bitrate,
+                    ]
+                    + hls_output
+                )
+
            try:
                proc = subprocess.Popen(
                    cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
@@ -54,6 +134,8 @@ class TranscodeManager:
                pass

        self._processes[stream_key] = procs
+        self._signatures[stream_key] = self._signature(qualities)
+        self._failed_starts[stream_key] = 0 if procs else self._failed_starts.get(stream_key, 0) + 1

    def stop_transcode(self, stream_key: str):
        procs = self._processes.pop(stream_key, [])
@@ -72,4 +154,4 @@ class TranscodeManager:
            self.stop_transcode(key)


-transcode_mgr = TranscodeManager()
+transcode_mgr = TranscodeManager()