I follow llama.cpp pretty closely as I use either llama.cpp itself or projects that depend on it all the time, and one thing that I don't think gets talked about is the sheer scale of community involvement. It seems like a logistical nightmare, but somehow thousands of different contributors are opening hundreds of PR's every week and getting them merged in to support various hardware or implement a new pattern or algorithm from a recent research paper. It's really quite awe inspiring for me to see, and think it is in no small part because of the leadership of ggerganov - so I'm happy to see that he is sticking around and plans to keep building this incredibly useful tool that has grown into a huge community at this point.

▲

nikwen 2 days ago | parent | next [-]

Here's why I think it's working so well, as someone who's participating in llama.cpp development and who has participated in many other open source projects:

1) Reviews are super fast. Sometimes I get the first review 5 minutes after submitting a PR. Today I had a PR merged with two reviews within 42 minutes. This is incredible work by the whole team. It keeps contributors motivated. The pace is intoxicating. On other projects, I've sometimes been waiting months for a review.

2) Georgi motivates people by giving them responsibility. There's no gatekeeping as with other projects. He happily delegates. You do good work? It's appreciated, and you get the freedom you need to make an impact. That feels awesome.

I've rarely become addicted to an open-source project so quickly.

	▲	random3 a day ago \| parent [-]
		That’s sounds so cool! I wonder how they get to do high throughput contributions. Is the structure very flat, or plugin based? I’ve always admired this project from the outside and glad to learn it’s doing so well. Looks like they have something really good going on. Kudos to the team!

▲

springtimesun 2 days ago | parent | prev [-]

I have been using a PR branch to run GLM 5.3 flash locally so I have been watching the dueling PRs develop to implement it and all the comments and reviews. You're right the scale of it is huge. The speed with which maintainers and community are responding is also super impressive.

I know there's a lot of discussion about AI overwhelming OS maintainers and I can see that. It does seem like llama.cpp is successfully riding that dragon right now though.

	▲	nikwen 2 days ago \| parent [-]
		Until 1-2 months ago, AI-written code was not allowed in llama.cpp. That changed, but it's still not allowed to have AI write your posts for you (issues, PR descriptions, etc.). In my experience, banning AI-written prose leads to much better contributions than on projects where it's allowed. Of course, llama.cpp also gets lots of AI slop PRs from first-time contributors, but from what I've seen, maintainers simply ignore the bad submissions and prioritize the good contributions. I think that's probably one of the best ways to handle the slop overwhelm.