▲ | lukev 2 hours ago | |
Actually I think even this makes it look scarier than it is if you're on an M-series Apple. In that case, the steps to building llama.cpp are: 1. Clone the repo. 2. Run `make`. To start chatting with a model all you need is to: 1. Download the model you want in gguf format that will fit into your hardware (probably the hardest step, but readily available on HuggingFace) 2. Run `./llama-server -m model.gguf`. 3. Visit localhost:8080 | ||
▲ | int_19h 12 minutes ago | parent [-] | |
On a Mac, if all you want is to just use it directly, it is also readily available from Homebrew. |