Whisper it: FFmpeg 8 can now subtitle your videos on the fly
- Reference: 1756402152
- News link: https://www.theregister.co.uk/2025/08/28/ffmpeg_8_huffman/
- Source link:
[1]At the start of the week , the FFmpeg project [2]released its eighth major version. It's codenamed "Huffman" after the [3]Huffman code algorithm , which was invented in 1952, making it one of the [4]oldest lossless compression algorithms .
We last looked at FFmpeg version 6.1 [5]in late 2023 , in case you need to refresh your memory for what it is and does. Since then, there was 7.0 last April and 7.1 in September, but after an unusually long interval, this is the first release of 2025 – and it's quite a big one.
[6]
The [7]changelog lists 30 significant changes, of which the top new feature is [8]integrating Whisper . This means [9]whisper.cpp , which is [10]Georgi Gerganov 's entirely [11]local and offline version of [12]OpenAI's Whisper automatic speech recognition model. The bottom line is that FFmpeg can now automatically subtitle videos for you.
[13]
[14]
This release uses the Vulkan API to do some hardware acceleration of some of its codecs. This includes some FOSS formats, such as encoding [15]AV1 format , and both encoding and decoding [16]FFv1 and the [17]WebM group's [18]VP9 format . It also supports [19]Apple's ProRes RAW . It can handle AV1 files even if they use [20]common encryption (CENC) , and supports all types of SCC [21]screen content coding .
This is a little surprising – in a good way – because as The Register [22]said when first covering Vulkan a decade ago, it's a 3D graphics API. Its codename was glNext and it was intended to replace OpenGL one day. An API for hardware-accelerated 3D rendering is not an obvious fit for video coding, but using Vulkan APIs to handle some of the pixel data in video frames means that you get to use the computer's GPU to perform the work, not the CPU – and you do so in a cross-platform sort of way that doesn't even need to know what GPU it's using, let alone being tied to specific GPU models, drivers, or OSes.
[23]
FFmpeg now supports VVC – Versatile Video Coding ( [24]ITU H.266 ) – using [25]VAAPI , Intel's [26]open spec for GPU acceleration. As [27]Phoronix reported , this code was contributed by Intel late last year. VVC is the successor to [28]H.265 HEVC , which in turn is the successor to [29]H.264 AVC . That last standard is one of the most widely used video standards, and also widely hardware-accelerated. Huffman can also handle VVC inside [30]Matroska container files .
[31]GhostBSD 25.02 adds 'Gershwin' desktop for a Mac-like twist
[32]VirtualBox 7.2 fixes flaky 3D guests and adds Arm-on-Arm support
[33]Thunderbird 142 lands with modest upgrades – plus talk of Pro service ahead
[34]Saved you a click: Firefox 142 offers AI summaries of links
It's not all about cool new standards, though. For instance, FFmpeg 8 can newly decode [35]RealVideo 6.0 , which [36]dates back to at least 2015.
(Incidentally, the same author who wrote that overview of RealVideo 6.0 also has a [37]history of the controversial project lead Michael Niedermayer, whose – [38]presumably temporary – resignation we covered in 2015. We say "presumably temporary" as he tagged this version.)
In 2022, we [39]reported that Google dropped JPEG-XL from Chromium . That doesn't seem to have killed the new format: Huffman can animate displaying JPEG-XL files. It has improved handling of AVP, the [40]Advanced Professional Video format. It can now decode speech encoded in the [41]G.728 low-bandwidth format, and Sanyo's LD-ADPCM format used for WAV files from some Sanyo digital voice recorders. Good news for owners of gadgets such as the ICR-1000 and ICR-B150.
The FFmpeg project – the name [42]means "fast forward MPEG," incidentally – has quite a turbulent history. This has included attempted forks, such as LibAV, of which Clément [43]"ubitux" Bœsch [44]has a history . Reddit also has a [45]pithy executive summary .
[46]
LibAV – not to be confused with the [47]FFmpeg library called libav – thrived for a while. Debian switched to it, but switched back again for reasons that [48]LWN explained a decade ago . The rival project was [49]abandoned in 2020 .
These days, FFmpeg is an essential tool. We found Wikipedia's [50]list of projects using it eye-opening, and we recommend developer and prolific blogger Drew deVault's article, [51]In praise of ffmpeg . ®
Get our [52]Tech Resources
[1] https://ffmpeg.org/index.html#news
[2] https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/release/8.0:/RELEASE_NOTES
[3] https://www.w3schools.com/dsa/dsa_ref_huffman_coding.php
[4] https://ethw.org/History_of_Lossless_Data_Compression_Algorithms#History
[5] https://www.theregister.com/2023/11/24/ffmpeg_6_1/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aLDRdkQhL9a1kkOpVVaRngAAAA0&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[7] https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/release/8.0:/Changelog
[8] https://code.ffmpeg.org/FFmpeg/FFmpeg/commit/13ce36fef98a3f4e6d8360c24d6b8434cbb8869b
[9] https://github.com/ggml-org/whisper.cpp
[10] https://github.com/ggerganov
[11] https://dataloop.ai/library/model/ggerganov_whispercpp/
[12] https://github.com/openai/whisper
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aLDRdkQhL9a1kkOpVVaRngAAAA0&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aLDRdkQhL9a1kkOpVVaRngAAAA0&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[15] https://aomedia.org/specifications/av1/
[16] https://github.com/FFmpeg/FFV1
[17] https://www.webmproject.org/
[18] https://www.webmproject.org/vp9/
[19] https://support.apple.com/en-us/102124
[20] https://www.iso.org/standard/84637.html
[21] https://visionular.ai/av1-screen-content-coding/
[22] https://www.theregister.com/2015/03/03/here_comes_vulkan_the_next_generation_of_the_opengl_graphics_api/
[23] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aLDRdkQhL9a1kkOpVVaRngAAAA0&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[24] https://www.itu.int/rec/T-REC-H.266
[25] https://www.intel.com/content/www/us/en/developer/articles/technical/linuxmedia-vaapi.html
[26] https://www.freedesktop.org/wiki/Software/vaapi/
[27] https://www.phoronix.com/news/VVC-VA-API-FFmpeg-Decode
[28] https://www.itu.int/rec/T-REC-H.265
[29] https://www.itu.int/rec/T-REC-H.264
[30] https://www.matroska.org/index.html
[31] https://www.theregister.com/2025/08/27/ghostbsd_2502/
[32] https://www.theregister.com/2025/08/23/virtualbox_72_fixes_3d_guests/
[33] https://www.theregister.com/2025/08/22/thunderbird_142/
[34] https://www.theregister.com/2025/08/22/firefox_142/
[35] https://wiki.multimedia.cx/index.php/RealVideo_6
[36] https://codecs.multimedia.cx/2017/10/rmhd-a-more-detailed-look/
[37] https://codecs.multimedia.cx/2023/01/ffhistory-michael-niedermayer/
[38] https://www.theregister.com/2015/08/05/ffmpeg_leader_steps_down/
[39] https://www.theregister.com/2022/10/31/jpeg_xl_axed_chrome/
[40] https://www.ietf.org/archive/id/draft-lim-apv-00.html
[41] https://www.itu.int/rec/T-REC-G.728/
[42] https://ffmpeg.org/pipermail/ffmpeg-devel/2006-February/010315.html
[43] https://github.com/ubitux
[44] https://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html
[45] https://www.reddit.com/r/linux/comments/vvdxn/the_ffmpeglibav_situation/
[46] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/applications&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aLDRdkQhL9a1kkOpVVaRngAAAA0&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[47] https://trac.ffmpeg.org/wiki/Using%20libav*
[48] https://lwn.net/Articles/650816/
[49] https://web.archive.org/web/20200812182017/https://lists.libav.org/pipermail/libav-devel/2020-April/086589.html
[50] https://en.wikipedia.org/wiki/FFmpeg#Projects_using_FFmpeg
[51] https://drewdevault.com/2022/10/12/In-praise-of-ffmpeg.html
[52] https://whitepapers.theregister.com/
You get subtitles, you get subtitles, you all get subtitles!
Such is FFMPEG's ubiquity I expect that all systems will have auto generated subtitles now.
Re: You get subtitles, you get subtitles, you all get subtitles!
As long as they're not hardcoded/burned-in.
There's a special circle of Hell for people who do that .
Re: You get subtitles, you get subtitles, you all get subtitles!
mkv allows for separate subtitle tracks.
Any hardware recommendations?
I admit that I have no interest in gaming, and therefore don't know much about GPUs. That said, I do use FFMPEG a lot, mainly convering H264 videos to H265 (for reasons, mainly disc space).
So what are the requirements for this use case? Will a £100 Radeon show a marked reduction in encoding time, or do I need to spend £1000 on a higher spec card? I'm a Debian Linux user, so obviously Vulkan 1.3 support is a given, but what else is needed? I assume more £££ = more speed, but as it's doing maths rather than outputting to a screen, I wonder if a lesser card is good enough.
Anyone?
Re: Any hardware recommendations?
My experience will all those GFX-card encoders is that they target streaming. Try to use higher compression setting and you may get a hard freeze on nvidia (in Windows Terms: Your mouse is stuck, you don't even get a blue screen). But since you use debian you won't have that crash issue, but possible the same limitations: It cannot beat software encoding regarding size vs. quality.
MSU encoding tests at [1]https://compression.ru/video/codec_comparison/index_en.html may be the one you should look for since they do quite extensive testing. But does not include your specific scenario, but allows comparison hardware vs software.
[1] https://compression.ru/video/codec_comparison/index_en.html
Re: Any hardware recommendations?
Ah, so are you saying this new speech to text option will only work if using the GPU for hardware encoding then?
I was hoping it would be a separate thread and do the sub-title generation purely from the audio track leaving me to choose how the video source is encoded.
On the other hand, I think my initial reaction was "ooooh! Shiny!" and on reflection I don't really have all that much use for subtitling in the original language, more the occasional need to subtitle in English from a non-English soundtrack where I can't find one already done on t'internet :-)
Although as I tyoed that, I wondered if FFMPeG can generate textual subtitle files from non-English source and I could then run through a translator.
Re: Any hardware recommendations?
You might be able to script that, but you might need mkvtoolnix on top. ffmpeg internally does use the mkvtoolnix muxer for mkv. But subtitle extration, then send through some translator, save it as new subtitle file next to the old one, that sounds like mkvtoolnix for extracting to me. Which can handle various source types, not just mkv. Regarding some API to auto-translate: Sorry never needed that. English and German are always available, quite often both.
Re: Any hardware recommendations?
#MeToo :-)
Re: Any hardware recommendations?
Depends on your CPU, and Intel GPUs including iGPUs can actually be pretty good for video encoding/decoding even if they are not so good for anything else.
What! No Rust version? lol