I did use whisper last night to get the captions out of a video file. The standard whisper tool from OpenAI uses CPU. It took more than 20 minutes to fully process a video file that was a little more than an hour long. During that time my 20-Core CPU was pegged at 100% utilization and the fan got very loud. I then downloaded an Intel version that used the NPU. CPUs stayed close to 0% and fans remained quiet. Total task was completed in about 6 minutes.

NPUs can be useful for some cases. The AI PC crap is ill thought out however.

▲

BeetleB a day ago | parent | next [-]

I suggest trying whisper-cpp if you haven't. It's probably the fastest CPU only version.

But yeah, NPUs likely will be faster.

▲

Neywiny a day ago | parent | next [-]

Depending on the part, it's likely the iGPU will be even faster. The new panther lake has iGPUs with either 80% or 250% the performance of the NPU when at the higher end. But on lower end models, it's lower but still within the same performance class

▲

satvikpendem 9 hours ago | parent | prev [-]

faster-whisper can be faster in many cases, even on CPU.

▲

BeetleB 8 hours ago | parent [-]

Looking at that page, it doesn't seem particularly faster than whisper-cpp, except when using batches - but I'm not clear on what that means.

Does it have a command line utility I can embed into my scripts?

	▲	satvikpendem 8 hours ago \| parent [-]
		Batching is essentially running multiple instances at once, ie bundling 8 segments and running them simultaneously on the processing unit, but which obviously takes more RAM to do. Notice, however, that if you drop the precision to int8 from fp16, you use basically the same amount of RAM as whisper.cpp yet it completes in a fraction of the time using batching [0]. Yes, if you check their community integrations section on faster-whisper [1], you can see a lot of different CLIs, GUIs, and libraries. I recommend WhisperX [2], it's the most complete CLI so far and has features like diarization which whisper.cpp does not have in a production-ready capacity. [0] https://github.com/SYSTRAN/faster-whisper#benchmark [1] https://github.com/SYSTRAN/faster-whisper#community-integrat... [2] https://github.com/m-bain/whisperX

▲

zamadatix a day ago | parent | prev [-]

If you mean OpenVINO, it uses CPU+GPU+NPU - not just the NPU. On something like a 265K the NPU would only be providing 13 of the 36 total TOPS. Overall, I wish they would just put a few more general compute units in the GPU and have 30 TOPS or something but more overall performance in general.