
JFrog: PixelSmash
JFrog Security Research discloses CVE-2026-8461 ("PixelSmash"), a critical heap out-of-bounds write in FFmpeg's MagicYUV decoder caused by a rounding mismatch between chroma-plane allocation and slice-height calculation. The team escalates the OOB write into reliable remote code execution by hijacking the AVBuffer.free function pointer to call system(), demonstrating full RCE via a 50KB crafted AVI against a Jellyfin media server (through automatic library/ffprobe scanning) and a Nextcloud instance (via video thumbnail preview generation), plus DoS crashes across mpv, Kodi, OBS Studio, Immich, PhotoPrism and vLLM. The post frames the bug as a software supply-chain issue since libavcodec is a silent transitive dependency in hundreds of downstream media applications, and covers detection, patching, and build-time workarounds.
What it is
PixelSmash is CVE-2026-8461, a heap out-of-bounds write in FFmpeg's MagicYUV decoder (CVSS 8.8), disclosed by JFrog Security Research. The summary covers the RCE demonstrations; the blog additionally supplies a fixed version, a detection command, two workarounds, and the specific attack surface — which is why this document exists.
How it works
The bug is triggered by malformed slice_height values in a crafted MagicYUV bitstream, causing an out-of-bounds write into a heap-allocated frame buffer. A valid delivery container is AVI, MKV, or MOV. JFrog escalated the crash to reliable RCE; the patch rejects the malformed slice heights, and the page notes the reference MagicYUV encoder always emits aligned slice heights, so the check only rejects malicious input.
Preconditions / Affected
The MagicYUV decoder is enabled by default in every upstream FFmpeg build and every distro package tested (Ubuntu, Debian, Fedora, Arch, Alpine) before version 9.0. Exploitation requires only delivering a media file to something that decodes video via libavcodec — no authentication, privileges, or prior access. Trigger paths include a user opening or even just browsing a folder containing the file (thumbnail generators used by GNOME/KDE/XFCE), server-side auto-processing (Jellyfin, Emby, Nextcloud, Immich, plus chat platforms Slack/Discord/Telegram and cloud transcoders AWS MediaConvert/Cloudflare Stream), and NAS/smart-TV appliances (Synology, QNAP).
What to do
Upgrade FFmpeg to the fixed version (9.0 or later). Detect the vulnerable decoder:
ffmpeg -decoders 2>/dev/null | grep magicyuvOutput containing VFS..D magicyuv means the build is vulnerable.
If you cannot upgrade immediately, disable the decoder at build time:
./configure --disable-decoder=magicyuv [your other flags]
make && make installOr apply the minimal 7-line patch to libavcodec/magicyuv.c:
if (s->slice_height <= 0 || s->slice_height > INT_MAX - avctx->coded_height) {
av_log(avctx, AV_LOG_ERROR, "invalid slice height: %d\n", s->slice_height);
return AVERROR_INVALIDDATA;
}
+ if ((s->slice_height >> s->vshift[1]) <= s->interlaced) {
+ av_log(avctx, AV_LOG_ERROR, "impossible slice height\n");
+ return AVERROR_INVALIDDATA;
+ }
+ if ((avctx->coded_height % s->slice_height) && ((avctx->coded_height % + s->slice_height) >> s->vshift[1]) <= s->interlaced) {
+ av_log(avctx, AV_LOG_ERROR, "impossible height\n");
+ return AVERROR_INVALIDDATA;Caveats
The downstream projects (Jellyfin, mpv, Nextcloud, Immich, OBS, vLLM, and others) did not introduce the bug and, per JFrog, "most have no mechanism to detect or mitigate it independently" — so patching a single application is insufficient if other bundled/linked FFmpeg copies exist on the same host. Full RCE was demonstrated against Jellyfin and Nextcloud; the other listed products were confirmed to crash, not proven exploitable to RCE.
CVEs named here
- CVE-2026-8461
Technologies named here
- FFmpeg
- GDB
- AddressSanitizer
- strace
- Nextcloud
- JFrog Security Research Center
- OBS Studio
- vLLM
- Emby (crash confirmed)
- ffmpegthumbnailer (crash confirmed, used by GNOME, KDE, XFCE)
- ffprobe
- Immich (crash confirmed)
- Jellyfin
- Jellyfin 10.11.9 (RCE demonstrated via library scan)
- jellyfin-ffmpeg 7.1.3 (Jellyfin's bundled FFmpeg, RCE demonstrated)
- Kodi (crash confirmed)
- MagicYUV
- mpv (crash confirmed)
- PhotoPrism (crash confirmed)
- Plex (builds FFmpeg with a minimal decoder allow-list, used as a defensive comparison)
Techniques named here
- Heap-based Buffer Overflow
- Reverse shell
- Function pointer overwrite (AVBuffer.free hijack)
- Heap layout calibration
- Zero-click delivery via monitored watch folder (torrent download)
Concepts named here
- ASLR
- Cost amplification in cloud pipelines
- Decoder allow-listing
- Silent heap corruption
- Software supply chain vulnerability
Related in Supply Chain


