Thanks for the suggestions! Apologies again I'm pretty bad at packaging, so hence the current setup.

1. So I added a `check_llama_cpp` which checks if llama.cpp does exist and it'll use the prebuilt one https://github.com/unslothai/unsloth-zoo/blob/main/unsloth_z...

2. Yes I like the idea of determining distro

3. Agreed on bailing - I was also thinking if doing a Python input() with a 30 second waiting period for apt-get if that's ok? We tell the user we will apt-get some packages (only if apt exists) (no sudo), and after 30 seconds, it'll just error out

4. I will remove sudo immediately (ie now), and temporarily just do (3)

But more than happy to fix this asap - again sorry on me being dumb

▲

mkl 3 days ago | parent [-]

It shouldn't install any packages itself. Just print out a message about the missing packages and your guess of the command to install them, then exit. That way users can run the command themselves if it's appropriate or add the packages to their container build or whatever. People set up machines in a lot of different ways, and automatically installing things is going to mess that up.

▲

danielhanchen 3 days ago | parent | next [-]

Hmmm so I should get rid of the asking / permissions message?

▲

mkl 3 days ago | parent [-]

Yes, since you won't actually need the permissions.

▲

danielhanchen 3 days ago | parent [-]

Hmmm I'm worried people will really not get on how to install / compile / use the terminal hmmm hence I thought permissions were like a compromise solution

▲

solarkraft 3 days ago | parent | next [-]

I think that it is, quite a good one, even:

- Determine the command that has to be run by the algorithm above.

This does most of the work a user would have to figure out what has to be installed on their system.

- Ask whether to run the command automatically.

This allows the “software should never install dependencies by itself” crowd to say no and figure out further steps, while allowing people who just want it to work to get on with their task as quickly as possible (who do you think there are more of?).

I think it would be fine to print out the command and force the user to run it themselves, but it would bring little material gain at the cost of some of your users’ peace (“oh no it failed, what is it this time ...”).

	▲	danielhanchen 3 days ago \| parent [-]
		Oh ok! I would say 50% of people manually install llama.cpp and the other 50% want it to be automated

▲

segmondy 3 days ago | parent | prev [-]

Don't listen to this crowd, these are "technical folks". Most of your audience will fail to figure it out. You can provide an option that llama.cpp is missing and give them an option where you auto install it or they can install it themselves and do manual configuration. I personally won't tho.

▲

Computer0 3 days ago | parent | next [-]

Who do you think the audience is here if not technical. We are in a discussion about a model that requires over 250gb of ram to run. I don't know a non-technical person with more than 32gb.

	▲	pxc 3 days ago \| parent [-]
		I think most of the people like this in the ML world are extreme specialists (e.g.: bioinformaticians, statisticians, linguists, data scientists) who are "technical" in some ways but aren't really "computer people". They're power users in a sense but they're also prone to strange bouts of computing insanity and/or helplessness.

▲

danielhanchen 3 days ago | parent | prev [-]

I think for a compromise solution I'll allow the permission asking to install. I'll definitely try investigating pre built binaries though

▲

solarkraft 3 days ago | parent | prev [-]

This is an edge case optimization at the cost of 95% of users.

	▲	mkl 3 days ago \| parent [-]
		95% of users probably won't be using Linux. Most of those who are will have no problem installing dependencies. There are too many distributions and ways of setting them up for automated package manager use to be the right thing to do. I have never seen a Python package even try.