Remix.run Logo
lukev 2 hours ago

Actually I think even this makes it look scarier than it is if you're on an M-series Apple.

In that case, the steps to building llama.cpp are:

1. Clone the repo.

2. Run `make`.

To start chatting with a model all you need is to:

1. Download the model you want in gguf format that will fit into your hardware (probably the hardest step, but readily available on HuggingFace)

2. Run `./llama-server -m model.gguf`.

3. Visit localhost:8080

int_19h 12 minutes ago | parent [-]

On a Mac, if all you want is to just use it directly, it is also readily available from Homebrew.