Linux kernel 7 enables the NPU on Linux. You can use fastflowLM with lemonade now.

It is quite slow, but if you want to compute embeddings in the background it’s fine.

I didn’t find it more energy efficient than just using the GPU for time insensitive tasks though.