| ▲ | GeekyBear 4 days ago | |||||||||||||||||||||||||||||||||||||||||||
> Hopefully Apple optimizes Core ML to map transformer workloads to the ANE. If you want to convert models to run on the ANE there are tools provided: > Convert models from TensorFlow, PyTorch, and other libraries to Core ML. | ||||||||||||||||||||||||||||||||||||||||||||
| ▲ | ls-a 4 days ago | parent | next [-] | |||||||||||||||||||||||||||||||||||||||||||
I thought Apple MLX can do that if you convert your model using it https://mlx-framework.org/ | ||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||
| ▲ | coffeecoders 4 days ago | parent | prev [-] | |||||||||||||||||||||||||||||||||||||||||||
It is less about conversion and more about extending ANE support for transformer-style models or giving developers more control. The issue is in targeting specific hardware blocks. When you convert with coremltools, Core ML takes over and doesn't provide fine-grained control - run on GPU, CPU or ANE. Also, ANE isn't really designed with transformers in mind, so most LLM inference defaults to GPU. | ||||||||||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||||||||||