What effort do you folks take to see your models actually running on hardware such as AMD Strix Halo or Apple M3M4?

I get the sense that AI is at the “hobby kit computing” stage where they used to dump all the components in a box and give you a schematic and a soldering iron and happily say “you make it work!”

And that worked in the early days of computing because there was a small number of people really motivated for the outcome.

But fully assembled and packaged and tested in a nice looking box is where the real demand turned out to be.

I’m looking forward to the day Google doesn’t just dump a model and say “you do the rest”.

I want to fire up Ubuntu on a Strix Halo and say apt install then load the browser interface. Or just download and run a Mac installer and have it just work.

Arcane complex multi step build install configure processes for AI need to end in favor of one click install. I’m not interested in the process of making it run.

▲ canyon289 6 days ago | parent | next [-]

I don't think we dumped the model and say you do the rest?

Myself and my colleagues spent many days transforming the weights into various open compatible formats. And its not just us there's many orgs and partners dedicating their time, resources, and companies to making all open models easy to use.

I encourage you to explore the solutions provided by them. We linked some in our blog post here, and there's more. They've all done a fantastic job building frankly an insane amount of infrastructure, documentation, and community support in the last 2+ years. Some of them are here in this HN thread answering questions.

	▲	kwerk 6 days ago \| parent [-]
		Thank you. And thank you for your kindness in these threads. It’s appreciated by the people who aren’t commenting as much

▲ dist-epoch 6 days ago | parent | prev | next [-]

Here you go, one click installer - https://lmstudio.ai

▲

andrewstuart 6 days ago | parent [-]

I’m talking about the supplier doing the packaging.

	▲	garbageman 6 days ago \| parent [-]
		Then use ChatGPT/Gemini/Claude on your phone. They are giving it away for free - if you NEED a local LLM, the least you can do is spend the 2 minutes to download LMSTudio and pick a model.

▲ freehorse 6 days ago | parent | prev [-]

Running this on your mac takes less of the effort of writing this comment (assuming you have homebrew installed)

1. open terminal.app

2. run:

    brew install llama.cpp
    llama-cli -hf ggml-org/gemma-3-270m-GGUF -c 0 -fa -p "hello"