Does this finally enable dynamically generating subtitles for movies with AI?

▲ jeroenhd a day ago | parent | next [-]

Docs say:

    If set, the transcription output will be sent to the specified file or URL
    (use one of the FFmpeg AVIO protocols); otherwise, the output will be logged as info messages.
    The output will also be set in the "lavfi.whisper.text" frame metadata.
    If the destination is a file and it already exists, it will be overwritten.

    @item format
    The destination format string; it could be "text" (only the transcribed text will be sent to the destination), "srt" (subtitle format) or "json".
    Default value: @code{"text"}

I don't know if this can embed the subtitles, but it does support generating accompanying srt files.

Of course, you could already do that by just manually calling whisper on files, but now you don't need to export parts or transformed media files to feed into whisper.

▲ regularfry a day ago | parent | prev | next [-]

If you have enough processing power. Without a GPU it's going to lag.

	▲	jeroenhd 16 hours ago \| parent \| next [-]
		In my experience, a small/tiny whisper model has pretty okay English decoding speed on something relatively modern even without GPU support. There's a bunch of latency in the process (because of technological limitations) but the optimised C++ version shouldn't pose too much of a problem unless you're running in power saving mode. Battery life may be a problem on older laptops, though.
	▲	KeplerBoy a day ago \| parent \| prev [-]
		Whisper is pretty fast.

▲ diggan a day ago | parent | prev [-]

Finally? I think VLC demo'd this a while ago at some conference where they had a table, if I remember correctly.

▲

SSLy a day ago | parent | next [-]

VLC and ffmpeg are unrelated projects

▲

demurgos 20 hours ago | parent [-]

I'm not very familiar with them, but I always assumed that there is a lot of overlap between the maintainers of both projects.

▲

SSLy 20 hours ago | parent [-]

Well, they are just unrelated. VLC has a plugin to access ffmpeg codecs via libav*, that's about it.

	▲	guipsp 17 hours ago \| parent [-]
		They are not completly unrelated. There is significant overlap. FFMPEG also uses libs from VLC.

▲

mmmpetrichor 20 hours ago | parent | prev [-]

I've been waiting a while now for automatic translated subtitles in vlc. I thought it would be here by now. I'm probably underestimating the difficulty but I'm surprised some video player hasn't done it by now. (as far as I know).

	▲	jeroenhd 16 hours ago \| parent [-]
		A lot of subtitles from commercial media use a subtitle format that's essentially a bitmap that the video player overlays on top of the video. There are tools to decode this using OCR, but it's not something I'd enable by default. For text/srt subtitles, translation would probably be easier. There's a plugin for that already if you're okay with online translation services: https://github.com/nopium/vlc-trans-lua