| ▲ | nottorp 7 hours ago | |
Hmm. Can this "AI" hardware - or any other "AI" hardware that isn't a GPU - be used for anything other than LLMs? YOLO for example. | ||
| ▲ | jsight 2 minutes ago | parent | next [-] | |
I'd expect that kind of thing to be the primary use case, tbh. Maybe even running whisper models? If it could run whisper, it'd be a solid addition to a pi based home assistant setup. | ||
| ▲ | dismalpedigree 7 hours ago | parent | prev [-] | |
Yes. The Hailo chips are mainly for AI vision models. This is the first time I have seen them pushed for LLM. They are very finicky and difficult to setup outside of the examples. Documentation is inconsistent and the models have to be converted to a different format to run. It is possible to run a custom yolo8 model, but is challenging. | ||