Remix.run Logo
jeffmess 8 hours ago

Doubt it: https://github.com/ziglang/zig/pull/24536

omnimus 7 hours ago | parent | next [-]

When somebody comments PR with “Incredible work, Jacob. It is an honor to call you my colleague.” then it's safe to assume it's out of the ordinary contribution. Pretty much falling outside of the “in all likelyhood”.

3000 line LLM commit is not that.

defmacr0 5 hours ago | parent | next [-]

Also 95% of those 30k lines changed are fully self-contained inside of the aarch64 directory and of the remaining changes it looks like the majority is just adding "aarch64" as another item into an existing list. There are a few core changes that to me look like they could be done in their own PRs, but also core maintainers get to decide if they want to apply bureaucracy to their own work.

thomascountz 7 hours ago | parent | prev | next [-]

No description provided. I love this PR. But yeah, try being anyone besides Jacob and submitting that!

KronisLV 6 hours ago | parent [-]

> In successful open source projects you eventually reach a point where you start getting more PRs than what you’re capable of processing. Given what I mentioned so far, it would make sense to stop accepting imperfect PRs in order to maximize ROI from your work, but that’s not what we do in the Zig project. Instead, we try our best to help new contributors to get their work in, even if they need some help getting there. We don’t do this just because it’s the “right” thing to do, but also because it’s the smart thing to do.

I feel like if their goal is to prioritize contributors over contributions, it'd also logically follow that they should try to have descriptions where possible? Just to make exploring any set of changes and learning easier? Looked it over briefly, no Markdown or similar doc changes there either.

I mean the changes can be amazing, it's just that adding some description of what they are in more detail, alongside the considerations during development, for new folks or anyone wanting to learn from good code would also be due diligence.

vga1 7 hours ago | parent | prev [-]

How would you differentiate a 3000 line LLM commit made by the best models and good AI processes from a 3000 line commit made by the best human developer?

edit Okay, I set the bar too high here with "best human developer" and vague "good AI processes". My bad. Yes, LLM is not quite there yet.

vurudlxtyt 7 hours ago | parent | next [-]

A personal relationship and trust, as seems to be the case here?

7 hours ago | parent [-]
[deleted]
wiseowise 7 hours ago | parent | prev | next [-]

By using my brain.

dnikolovv 6 hours ago | parent [-]

Don't be ridiculous! We don't do that anymore.

maccard 3 hours ago | parent | prev | next [-]

The post that inspired this post [0] says:

> So while one could in theory be a valid contributor that makes use of LLMs, from the perspective of contributor poker it’s simply irrational for us to bet on LLM users while there’s a huge pool of other contributors that don’t present this risk factor.

> The people who remarked on how it’s impossible to know if a contribution comes from an LLM or not have completely missed the point of this policy and are clearly unaware of contributor poker.

The point isn't about the 3000 line PR, it's about do we think the submitter is going to stick around.

[0] https://kristoff.it/blog/contributor-poker-and-ai/

saagarjha 7 hours ago | parent | prev | next [-]

Read it?

7 hours ago | parent [-]
[deleted]
IshKebab 7 hours ago | parent | prev [-]

It's still fairly obvious just by skimming the code. The best AI models are still quite far from the best human developers in ability and especially in code quality.

matwood 6 hours ago | parent [-]

When the best AI models are the same or better than the best[1] human developers, what then?

We're already at the point talking about best vs. best.

IshKebab 6 hours ago | parent | next [-]

If that happens and we have a way of reliably knowing if some code is produced to that high quality, then I think we probably can accept that AI coding is the only sensible option.

We definitely are not close to that point though and it's unclear if/when we will get there.

vga1 4 hours ago | parent [-]

It seems to me that people might be arguing from conflicting hidden premises here. "AI Coding" is a spectrum that could mean something as simple as letting the LLM proofread your changes and then act on those with your own human brain, or it could mean just telling the agent what you want and let it rip and tear until it is done.

If I do the latter and submit a PR to something like Zig, I'll be certainly caught doing it and rightfully chastised. If I do the former, my PR will be better without anybody besides myself having any way of knowing how it got better. Probably I do something in between when I contribute to open-source these days.

Blanket banning all of these seems like a bad idea to me. It actively gates people like myself from contributing, because I respect these people and projects that much. It feels like I would be doing something they find disgusting if my work has touched an LLM and I obviously don't want to do that to people I respect. But it's fine, there are plenty of things to do in the world even when some doors are closed.

I do not presume to have any say on Zig project's well argued decisions[0] -- I'm not really even their user let alone someone important like a contributor. Their point of preferring human contact is superb, frankly. Probably a different kind of problem in an open-source project staffed with a lot of remote working people, where human contact is scarce.

https://kristoff.it/blog/contributor-poker-and-ai/

irishcoffee 5 hours ago | parent | prev [-]

How can AI possibly be better than “the best” when the corpus of training data now includes its own slop in addition to all the code by new devs/lazy devs/bad devs scattered all over the internet? Law of averages applies here.

vga1 4 hours ago | parent [-]

Because LLM models are obviously much more than the sum of their parts.

irishcoffee 2 hours ago | parent [-]

Oh, which parts are those? Do tell!

flohofwoe 7 hours ago | parent | prev | next [-]

Jacob is part of the core team, not a random outside contributor.

slekker 7 hours ago | parent | prev [-]

Very different context: that PR is from a maintainer, and trusted member of Zig, which surely discussed the implementation/design internally as well