Note that the NPU models/kernels this uses are proprietary and not available as open source. It would be nice to develop more open support for this hardware.

▲

plagiarist 7 hours ago | parent | next [-]

I bought one of their machines to play around with under the expectation that I may never be able to use the NPU for models. But I am still angry to read this anyway.

	▲	zozbot234 7 hours ago \| parent [-]
		AMD/Xilinx's software support for the NPU is fully open, it's only FFLM's models that are proprietary. See https://github.com/amd/iron https://github.com/Xilinx/mlir-aie https://github.com/amd/RyzenAI-SW/ . It would be nice to explore whether one can simply develop kernels for these NPU's using Vulkan Compute and drive them that way; that would provide the closest unification with the existing cross-platform support for GPU's.

▲

swiftcoder 9 hours ago | parent | prev [-]

Are they? The docs say "You can also register any Hugging Face model into your Lemonade Server with the advanced pull command options"

	▲	zozbot234 9 hours ago \| parent [-]
		That won't give you NPU support, which relies on https://github.com/FastFlowLM/FastFlowLM . And that says "NPU-accelerated kernels are proprietary binaries", not open source.