Remix.run Logo
TomasBM 10 hours ago

It's a fair policy. Getting those verbose, AI-authored walls of text is very annoying, especially when you're expected to thoroughly review it. It's like a denial-of-service attack on the human mind. I can only imagine how frustrating this can get in open projects that get a lot of contributions.

However, I don't think this will discourage AI-based coding at all. In fact, I see two potential outcomes of these policies:

- Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and the size of contributions stay the same, but the style gets quirkier.

- Positive: Submitters actually provide to-the-point, no-bullshit commits and comments - "here's the code, here's why I made that change, here are the effects of that change". Even if AI-generated, these small contributions may become much easier to verify & validate. We may even see some standardization in terms of what qualifies as an appropriately sized contribution, what requires more thorough review (e.g., adding unverified dependencies), etc.

I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.

ivorius 10 hours ago | parent | next [-]

> - Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and the size of contributions stay the same, but the style gets quirkier.

From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.

> Positive: Submitters actually provide to-the-point, no-bullshit commits and comments

That would be a dream.

QuantumNomad_ 9 hours ago | parent | next [-]

> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.

True. At least with a policy about it, the project maintainers can unilaterally close such PRs without further internal or external discussion on any case-by-case basis.

maybewhenthesun 7 hours ago | parent [-]

Dingdingding, we have a winner. The main use of such a policy is to be able to just close those giant wall-of-text PRs and have something to point to when people start to scream it's not fair.

tayo42 4 hours ago | parent | next [-]

Why is a policy necessary. you were never entitled to have your pr merged in the first place? If pr wasn't reviewable pre AI I'd expect it to be closed or ignored too

the_hoser 4 hours ago | parent [-]

The policy isn't necessary to close the PR. The policy just helps to shut down the ensuing discussion after closing the PR. It helps in quickly dealing with well-meaning onlookers asking for clarification when you block PRs from the account.

tayo42 2 hours ago | parent [-]

Still, your not even entitled to a discussion though.

bluGill 2 hours ago | parent | next [-]

Before AI a large pull request was enough effort to make that you could assume good faith work on the part of everyone. Likely even if it is bad for architectural reasons it is solving an itch other users have and it is reasonable for them to want an explanation why you refused someone who made this much effort. And since the effort required meant it didn't happen often it wasn't a big deal to provide that.

These days large PRs are easy to create and so humans need to shut them down.

LordDragonfang 16 minutes ago | parent | prev [-]

Whether or not someone is entitled to something has very little bearing on whether someone believes they are entitled to something (and are willing to waste everyone's time to make a stink about it). Having clear rules to point to, even after the fact, is surprisingly effective in mitigating that.

Or, put more bluntly, your belief in what people ought to feel entitled to has no bearing on what they do believe, and policy needs to address the latter, not the former.

mcphage 6 hours ago | parent | prev [-]

> when people start to scream it's not fair

Or LLMs, as we have seen.

chrisjj 6 hours ago | parent [-]

Or, who knows, the "AI" might gain sufficient intelligence to read the policy...

julianlam 5 hours ago | parent [-]

Then we'll amend the policy to instruct LLMs to author PRs as Fred Flintstone, yabba dabba doo!

stronglikedan 2 hours ago | parent | prev | next [-]

> That would be a dream.

We just recently started that policy so we'll see how it goes. If anything, having it stated as policy lets us filter out these requests without spending brain tokens on them.

whateverboat 8 hours ago | parent | prev | next [-]

But now with AI, this should be "easier" for some definition of easy. In the sense that in the past, this might have taken 15 minutes to write, now with AI, this can take 5 minutes to write by first getting AI to produce a summary and then using human judgement to make it better. So, it's a good idea now to actually demand the dream.

ivorius 8 hours ago | parent [-]

If people knew how to get AI to write terse, focused summaries, sure, that might help. I haven't seen many that do (well, ignoring the toupee fallacy).

Though the most important aspect is that we need to know the motivation and thought process, and all AI can do is fabricate a 'plausible' one.

adalacelove 7 hours ago | parent | next [-]

Reading AI PRs reminds me of Monty Python's holy grenade:

"And the Lord spake, saying, ''First shalt thou take out the Holy Pin. Then shalt thou count to three, no more, no less. Three shall be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who, being naughty in My sight, shall snuff it.'

dsign 6 hours ago | parent [-]

I wouldn't mind reading that and having a good chuckle while processing an MR, as long as the comment had been crafted by a person. But now I think writing in grunts is gonna become the thing. "pete? you good boy? we won't hire you/ paper you gave HR girl with older gigs? remember it pete my boy? too many words/ words were too long/ dots you used dots/ you scratched long dash with knife, but baby saw scratch/we don't ai pete/ask if they have job in next cave/good luck."

lokar 3 hours ago | parent | prev | next [-]

I have spent many years reviewing and editing design and product documentation from more Jr engineers and product managers.

It was a constant struggle to get them to be concise, one I mostly lost.

chrisjj 6 hours ago | parent | prev | next [-]

> If people knew how to get AI to write terse, focused summaries...

... then flooded maintainers would be doing it.

ray_kay777 6 hours ago | parent | prev [-]

I've found that the instructions "be extremely concise" gets me much closer to output that's actually sensible/helpful rather than another wall of text.

snarfy 6 hours ago | parent | prev | next [-]

They could allow AI PRs, but then have another AI PR reviewer reject them if they do not match the definitions for `to-the-point` `no-bullshit` commits.

skeeter2020 3 hours ago | parent | next [-]

Please provide 3 examples where layering on MORE of the offending technology has solved the problem. Spam? Malware & Viruses? Customer Service? Hiring & Recruitment?

mort96 6 hours ago | parent | prev [-]

And who pays for the (likely significant, and controllable by everyone) tokens such a system would use?

julianlam 5 hours ago | parent [-]

A small 4-9B model would be able to run cheaply for this sort of work.

krainboltgreene 5 hours ago | parent [-]

What question do you think you're answering?

Cpoll 4 hours ago | parent [-]

The Godot Foundation pays for it if it furthers their mission.

mort96 4 hours ago | parent [-]

Does it?

DonsDiscountGas 2 hours ago | parent | prev | next [-]

I've always instructed Claude to check the policies first, frankly I'm surprised it's not smart enough to do that already. Would be easy to add to a system prompt. But usually it doesn't matter because many projects have no policies, or maybe they exist but only in hidden forum posts issues or something.

watwut 9 hours ago | parent | prev | next [-]

> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.

The policy allows the reviewer to reject it on the "AI" grounds.

dspillett 8 hours ago | parent [-]

> allows the reviewer to reject it on the "AI" grounds

… but still unfortunately leaves reviewers having to spend time checking submissions and rejecting them.

jon-wood 8 hours ago | parent | next [-]

At least half the people firing off LLM generated PRs will have left the "Coauthored-by: Claude" line on it allowing automated rejections.

ivorius 8 hours ago | parent [-]

Unfortunately, only a single PR like this comes to mind. Most AI authors we've seen were identifiable mainly by overly verbose PR descriptions, meaningless code changes and copy-pasting more AI output when questioned.

fendy3002 8 hours ago | parent | prev [-]

oh but it'll be very2 helpful and the time spent will be short. It's easy to verify:

* new contributor?

* more than 10 files affected (higher count are more valid)?

* wall of text on description without screenshots, etc?

just close the PR as AI, and then the contributor can challenge it if they feel it should not.

HelloNurse 6 hours ago | parent [-]

A contributor in good faith is going to accept criticism and resubmit an improved change: less files modified, more explanation, more focus, references to actual tickets and discussion with actual developers.

paulddraper 6 hours ago | parent | prev [-]

> That would be a dream.

“Mission. Fucking. Accomplished.”

https://xkcd.com/810/

dspillett 8 hours ago | parent | prev | next [-]

> I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.

The problem is that a lot of AI contributions are lazily produced without review. Those that have been properly reviewed for correctness (tested to ensure actually working with no obvious undesirable side effects, tweaked where needed to be readable and understandable, fitting the other guidelines of the project, etc) will be indiscernible from human-only contributions, but there are a lot of people who make no such effort so the majority are not nearly this good.

mexicocitinluez 5 hours ago | parent [-]

> The problem is that a lot of AI contributions are lazily produced without review.

That sounds like a contributor problem. Not an AI problem.

I still don't understand a "no AI" policy whose only purpose is to weed out bad PRs. You should be weeding out bad PR's regardless of their source. I don't see why treating a purely human-authored, but bad, piece of code should be treated any differently than an AI-authored one.

All they've accomplished is creaking an environment where good code can't be submitted unless the submitter lies.

lokar 3 hours ago | parent | next [-]

It’s just a social thing. Two identically bad submissions have different social contexts.

krainboltgreene 5 hours ago | parent | prev | next [-]

Probably because a human authored contribution, no matter how bad, can be trained to make it good and also improves the community.

mexicocitinluez 3 hours ago | parent [-]

> can be trained to make it good and also improves the community.

AI can be trained. Also, AI can create code that improves the community. It's replies like this that leave me even more confused.

skydhash 2 hours ago | parent [-]

Human being trained is already proven (that’s how most maintainers came to be). Can you explain how AI can be trained in the above context?

SpicyLemonZest 5 hours ago | parent | prev [-]

You should not be weeding out bad PRs regardless of their source! A pull request is a social artifact whose value and meaning is dependent on its author; bad PRs from a human author often mean things such as "I'd like to learn how this works and join your community". So it can be both satisfying and worthwhile to spend your effort on cleaning it up, even if it starts to take as much or even more effort than doing it yourself would have.

You're not the first person I've seen argue that authorship doesn't matter, so I don't want to blame you for it, but I really don't understand where that idea is coming from. To me it seems obviously wrong.

KronisLV 4 hours ago | parent | next [-]

> You should not be weeding out bad PRs regardless of their source! A pull request is a social artifact whose value and meaning is dependent on its author; bad PRs from a human author often mean things such as "I'd like to learn how this works and join your community".

I think the difference in perspective might come from the fact that to many people the code and features matters more than any community or the idea of participating in it. If it works, it works.

Or maybe they’re not even indifferent about the community, just upset at people throwing away working code.

SpicyLemonZest 4 hours ago | parent [-]

The Godot maintainers have decided they don't support that perspective. In their words, they value "being cautious about feature creep" and "dedicated to high code quality"; they don't accept that all working code should be merged.

Aspiring contributors who'd like to make a different tradeoff are of course free to make a fork. But then all of the stuff in their fork won't benefit from the participation of the community, which I suspect most such people do value even if they identify as a "code first" person.

mexicocitinluez 3 hours ago | parent | prev | next [-]

> A pull request is a social artifact whose value and meaning is dependent on its author;

Says who? How can you say I'm categorically wrong when your entire point rests upon an opinion?

SpicyLemonZest an hour ago | parent [-]

Says the definition? I don't really understand your response. A pull request is a request from Alice (the author) that one of Barbara, Chris, Daniel (the maintainers) should pull her code into a particular branch.

Many communities do have a norm that all authors are to be presumed equal, as long as they're prepared to take advice and learn from it. (That's where all experts start, after all.) That's the norm that Godot are trying to protect here. If they don't stop accepting AI-authored contributions, they worry, reviewers will start to implicitly load-shed by not reading PRs from people they don't recognize.

danaris an hour ago | parent | prev [-]

So you can run your project that way.

You don't get to dictate that other people run their projects that way.

> A pull request is a social artifact whose value and meaning is dependent on its author

...and the project to which it is submitted.

SpicyLemonZest is not the sole arbiter of what PRs mean and stand for.

SpicyLemonZest 27 minutes ago | parent [-]

I was explaining why the Godot maintainers have chosen to enact the policy described in the source article, in response to a comment saying that they should not have enacted it. I don't understand why you think I'm dictating or arbitrating anything.

captainbland 9 hours ago | parent | prev | next [-]

> It's like a denial-of-service attack on the human mind.

I think this may be an example of deliberate hostile design, attempting to force users to adopt LLM based solutions to then summarise the vast output. Pushing back against AI contributions as such in this context makes sense, especially in software with an existing proven track record of great value delivery like Godot.

someonebaggy 8 hours ago | parent | next [-]

There's no chance that anyone saw that far ahead in the future and planned it. It's emergent behaviour.

x3ro 6 hours ago | parent | next [-]

Who says anything about „this far in the future“? It’s enough for Anthropic et al to realize this one or two model versions ago, see it as a strategic advantage and push for that behavior.

jayd16 4 hours ago | parent | prev | next [-]

Big tech: "Should we add any functionality at all to filter AI slop in any of our platforms?" "...nah"

Its not 500 moves ahead.

Yizahi 3 hours ago | parent | prev [-]

While it certainly didn't enter a mind of any director making decisions (because they can't comprehend not defecting in a prisoner's dilemma, being sociopaths), it was plainly obvious to every person even remotely connected to IT in the past two decades. If one makes a better and faster spam generator and the same unchanged program also works in reverse, by sifting through spam and condensing it to a readable summary, that it will be immediately co-opted in a spam arms race by all sides of the war and become essentially mandatory.

ahartmetz 5 hours ago | parent | prev [-]

AI also works better with concise, focused, high information density text. So AI-spam text hurts both humans and AIs, but humans more so than AIs. It is always a negative, except for the "competition" between (human with) AI and human without AI.

whateverboat 8 hours ago | parent | prev | next [-]

This was the original rule in linux kernel as well. No more than 200 loc per patch. We should also introduce this to git commits and pull request descriptions:

1. 400 chars/10 lines per commit

1b. Not more than 3 commits in the initial pull request

2. 20 lines of explanation for pull request

3. not more than 3 pull request open at any one time

TomasBM 6 hours ago | parent [-]

Seems like this policy would apply pretty well regardless of who/what generated the code.

mexicocitinluez 5 hours ago | parent [-]

YES! "No AI" policies that are purely based on technical grounds make no sense to me. Bad PR's are bad PR's regardless of their source.

Are we really in a situation where good code that solves a problem won't be merged because the person the person checked the "I used AI" box on the PR?

Ban PR's that are too big, don't have a clear purpose, touch too many areas, etc.

overgard 30 minutes ago | parent [-]

It's really a question of how much time you're willing to spend sorting through spam. "No AI" might be a blunt hammer, but the people submitting slop aren't reading guidelines anyway, and it's easier just to reject it early. Frankly, I'm sure if people wanted to sneak in an AI generated code by carefully reviewing it and making sure it's targeted and well tested... I'm sure they could, but those people aren't the problem.

mikepurvis 4 hours ago | parent | prev | next [-]

In my day job, I do a lot of AI coding but almost never have Claude actually create the PR titles or descriptions for me. It produces too much content, and the justification/background sections are often not quite right.

Most PRs to me are not coming out of nowhere anyway, rather they're "here's the linked issue, I started out addressing it by doing X and Y, but then Y got hairy so I switched to Z, hope that makes sense but happy to discuss further as well."

And most feedback is not "let's have you explain the design to me in a diff comment" but rather please explain this design in a code comment so that the next reader of the source will have your context.

lokar 3 hours ago | parent | next [-]

I think OSS maintainers are in the middle of intersecting trends:

- tough hiring market, especially for more Jr candidates

- the perception (true or not) that OSS contributions help get attention from recruiters

- LLMs making it very easy to generate “contributions”

8note 2 hours ago | parent | prev [-]

i want to figure out some extra practice - get a step of claude sending me a PR, then me accepting it after review, and then rewriting the merge to be a new PR for general review

peepee1982 8 hours ago | parent | prev | next [-]

If a commit is written by AI but reads as authored by a human, the developer has done their job and nothing will be flagged.

If commits written by AI wouldn't be substantially different, there would be no need to reject them.

So I agree with you that it won't discourage AI-based coding. But that's not even the intent.

clktmr 6 hours ago | parent | prev | next [-]

> I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.

It's pragmatic. Linus once said, the reason C++ is not allowed in the kernel is to keep the C++ people out.

ahartmetz 5 hours ago | parent [-]

Joke's on him, many Rust people are current or former C++ people.

RicardoLuis0 4 hours ago | parent [-]

just because someone writes or wrote C++ doesn't make them a "C++ person", and "C++ people" are very much against rust

ahartmetz 3 hours ago | parent [-]

Only true Scots... err C++ people are against Rust, I see :>

WhitneyLand 5 hours ago | parent | prev | next [-]

How strict is this no AI policy?

Say AI is used to identify and rewrite a single function that improves performance or fixes a bug, then the developer carefully reviews and tests it and submits a nice tight PR with all human communication.

So they don’t want that? They would just reject it?

If I’m understanding correctly, under the policy the higher performance function / bug free submission would be rejected and they could ask for a rewrite.

Should it then be rewritten from scratch, and clean room engineered so it doesn’t resemble the AI too much?

betorabinovich 3 hours ago | parent [-]

from TFA:

> The Foundation says we can expect Godot's contributing policy to soon include explicit rejections of AI-authored code, noting that contributors should only use AI assistance for "menial things" and must disclose its use. Additionally, the Foundation will reject any AI-generated text in human-to-human communications, saying it's "a basic principle of respect"—though it says machine translations "are still acceptable" if the original text was human-authored.

As long as your bots aren't contributing low-effort garbage in a push to give their operator some of those tasty internet brownie points you should be fine

basch 3 hours ago | parent | prev | next [-]

All these projects should seamlessly run a fork in parallel that accepts AI and has AI for review and approval. Both camps are happy.

Basically a play sandbox for contributors to not get jaded. A honeypot to contain the verbosity vomit, while also serving as positive public relations by keeping young contributor morale from starting in the basement.

Everyone has been that person once early in their life who is told they aren’t welcome and never comes back. Maybe it was SourceForge or IRC, maybe it was Wikipedia.

overgard 28 minutes ago | parent | next [-]

What's the point of this fork if it's just a landing spot for stuff that's not really wanted? Wouldn't that just be more condescending then telling people what's actually required for a contribution? Besides, the nature of forking is you can just do it yourself anyway. If people love their AI changes they can just make their own fork.

Plus I don't know how you could do this "seamlessly" -- someone has to manage merge conflicts, and as the codebases diverge it's just going to get more and more gnarly. (this is the reason most people don't maintain their own forks in the first place)

marcosdumay 2 hours ago | parent | prev | next [-]

Do you volunteer to maintain the sloppy fork?

endominus an hour ago | parent [-]

I think most pro-AI people would be happy to let an AI maintain the sloppy fork. What reason would they have to complain, after all?

TomasBM 2 hours ago | parent | prev [-]

This is actually a good idea.

I mean, not sure if everyone wants that for their project, and there will surely be plenty of trade-offs.

But it would be a very good compromise: You (the maintainer) get only human-generated PRs in the canonical project, and they (pro-AI contributors) get a lower-threshold sandbox to play with. Best case scenario, you cherrypick the pre-filtered golden nuggets to bring back to the canonical project.

basch 38 minutes ago | parent [-]

Precisely. Cherrypick nuggets from poo. It's there if you care to wade into it. May occasionally strike gold.

ahartmetz 6 hours ago | parent | prev | next [-]

Yes - if I can tell that you used AI (except maybe because of an unnaturally high work rate, or obviously an AI declaration, which is good!), you fail. Keep up the quality and I don't care too much.

I have some misgivings about AI, but I'm not a fundamentalist - you can't be or the machine will squish you, frankly - but please, don't spam me with text or code that could be much shorter. Relevant quotes:

"I didn't have time to write a short letter, so I wrote a long one instead"

"Brevity is the soul of wit"

Bukhmanizer 2 hours ago | parent | prev | next [-]

I recently wrote a tool to help me read AI generated PRs and it’s pretty sad that it’s got to this point.

reactordev 6 hours ago | parent | prev | next [-]

My agents operate on their own branch for a feature, they commit code changes after each step or phase with a description of what was changed, why, and what’s left.

This helps with PR reviews as it prevents a giant wall of text but it’s still verbose. However doing it this way cuts down on the wall of text at the expense of increased PR frequency.

onesandofgrain 10 hours ago | parent | prev | next [-]

The whole point of not-accepting AI authored code is because this line is not respected=>"Submitters actually provide to-the-point, no-bullshit commits and comments". You're putting way too much faith into the human minds ability to resist clout-chasing. AI isn't able to humanize code without human supervision.

blackoil 8 hours ago | parent | prev | next [-]

Better way is to provide a Claude.md with strong stylistic guidelines and loc requirements. Else it will be a chicken and mouse game of what is from AI.

ivorius 8 hours ago | parent [-]

I made a PR like that, but it was rejected by the community (for some valid reasons and some not so valid). https://github.com/godotengine/godot/pull/118681

someguynamedq 4 hours ago | parent [-]

Which goes to show that despite all of the rationalization both here and in the comments of that PR, the push to ban AI is religious not reasoned.

thewhitetulip 4 hours ago | parent | prev | next [-]

DoS attack is exactly what I describe an AI generated PR!!

chrisjj 6 hours ago | parent | prev | next [-]

> I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.

Perhaps reconsider "If your feedback on PRs is just being absorbed by a machine and not going towards mentoring a potential future maintainer..."

flexagoon 9 hours ago | parent | prev | next [-]

> Submitters just add stylistic markers to make their accounts and output seem human-generated

https://xkcd.com/810/

CamouflagedKiwi 9 hours ago | parent [-]

Not quite accomplished, if it's creating text on the pull requests that looks sufficiently human-like, but you're still worried about the quality of the code and that the submitter doesn't understand it.

someguynamedq 4 hours ago | parent [-]

No different than a human written PR

CamouflagedKiwi 2 hours ago | parent [-]

Right but as they mentioned, at least then they are communicating with a human about it, not going back and forth with a machine which they clearly do not enjoy.

shimonamit 5 hours ago | parent | prev | next [-]

[flagged]

sixtyj 10 hours ago | parent | prev [-]

[flagged]

grey-area 9 hours ago | parent [-]

If you understood the change, writing a short description of the problem and the fix yourself would be trivial.

sixtyj 9 hours ago | parent [-]

Efficiency is the key. I haven’t written any issue before so LLM was much quicker than manual experiment. I have personally checked the result before submission.

So why the hate? :)

unfocso 9 hours ago | parent | next [-]

You "personally checked" the result (generated by an LLM, a huge black box with extensive knowledge of all fields) to the best of your knowledge. There is a mismatch between what the machine knows (and has done as the result of it) and what you think you know.

Implementing a fix implies knowledge of the inner workings that brought you to it. A fix made by a LLM does not give you that.

sixtyj 5 hours ago | parent [-]

Why do I argue here anyway? :)

Before sending it I have tried the patch locally. It worked.

So I sent the proposal. And it was accepted by the author.

cyclopeanutopia 9 hours ago | parent | prev | next [-]

Efficiency rarely is the key.

chrisjj 6 hours ago | parent | prev [-]

> Efficiency is the key.

... and includes the reviewer efficiency disrespected by your verbose bot.