Remix.run Logo
themanualstates 2 hours ago

That’s useless without describing WHY you chose those flags, and how you did the optimisation…

halJordan an hour ago | parent [-]

The switches are all in the -h of llama.cpp (although the maintainers have a tendency to use the word in its definition). The actual values are essentially just what alibaba recommends. So you just need their model card. I would not call it highly optimized, more appropriately tuned.