(1) Removed and disabled sudo

(2) Installing via apt-get will ask user's input() for permission

(3) Added an error if failed llama.cpp and provides instructions to manual compile llama.cpp

▲ mFixman 3 days ago | parent [-]

Maybe it's a personal preference, but I don't want external programs to ever touch my package manager, even with permission. Besides, this will fail loudly for systems that don't use `apt-get`.

I would just ask the user to install the package, and _maybe_ show the command line to install it (but never run it).

▲ lucideer 3 days ago | parent | next [-]

I don't think this should be a personal preference, I think it should be a standard*.

That said, it does at least seem like these recent changes are a large step in the right direction.

---

* in terms of what the standard approach should be, we live in an imperfect world and package management has been done "wrong" in many ecosystems, but in an ideal world I think the "correct" solution here should be:

(1) If it's an end user tool it should be a self contained binary or it should be a system package installed via the package manager (which will manage any ancillary dependencies for you)

(2) If it's a dev tool (which, if you're cloning a cpp repo & building binaries, it is), it should not touch anything systemwide. Whatsoever.

This often results in a README with manual instructions to install deps, but there are many good automated ways to approach this. E.g. for CPP this is a solved problem with Conan Profiles. However that might incur significant maintenace overhead for the Unsloth guys if it's not something the ggml guys support. A dockerised build is another potential option here, though that would still require the user to have some kind of container engine installed, so still not 100% ideal.

▲ danielhanchen 3 days ago | parent [-]

I would like to be in (1) but I'm not a packaging person so I'll need to investigate more :(

(2) I might make the message on installing llama.cpp maybe more informative - ie instead of re-directing people to the docs on manual compilation ie https://docs.unsloth.ai/basics/troubleshooting-and-faqs#how-..., I might actually print out a longer message in the Python cell entirely

Yes we're working on Docker! https://hub.docker.com/r/unsloth/unsloth

▲ lucideer 3 days ago | parent [-]

> Yes we're working on Docker!

That will be nice too, though I was more just referring to simply doing something along the lines of this in your current build:

  docker run conanio/gcc11-ubuntu16.04 make clean -C llama.cpp etc etc...

(likely mounting & calling a sh file instead of passing individual commands)

---

Although I do think getting the ggml guys to support Conan (or monkey patching your own llama conanfile in before building) might be an easier route.

	▲	danielhanchen 3 days ago \| parent [-]
		Oh ok I'll take a look at conanfiles as well - sorry I'm not familiar with them!

▲ danielhanchen 3 days ago | parent | prev | next [-]

Hopefully the solution for now is a compromise if that works? It will show the command as well, so if not accepted, typing no will error out and tell the user on how to install the package

▲ solarkraft 3 days ago | parent | prev [-]

I like it when software does work for me.

Quietly installing stuff at runtime is shady for sure, but why not if I consent?

	▲	danielhanchen 3 days ago \| parent [-]
		Do you think it's ok for permissioning I guess? I might also add a 30 second timer and just bail out if there's no response from the user