Remix clone Hacker News

Threw a random PR at it… of the 11 issues it flagged, only 1 was appropriate, and that one was also caught by pylint :(

(mixture of 400 lines of C and 100 lines of Python)

It also didn't flag the one SNAFU that really broke things (which to be fair wasn't caught by human review either, it showed in an ASAN fault in tests)

▲

allisonee 4 days ago | parent [-]

sorry to hear that it didn't catch all the issues! if you downvote/upvote or reply directly to the bot comment @mrge-io <feedback>, we can improve it for your team.

We take all these into consideration when improving our AI, and your direct reply will fine tune comments for your repository-only.

▲

eqvinox 4 days ago | parent [-]

That's good to know, but —assuming my sample of size 1 isn't a bad outlier, I should really try a few more— there's another problem: I don't think we'd be willing to sink time into tuning a currently-free subscription service that can be yanked at any time. And I'm in a position to say it is highly unlikely that we'd pay for the service.

(We already have problems with our human review being too superficial; we've recently come to a consensus that we're letting too much technical debt slip in, in the sense of unnoticed design problems.)

Now the funny part is that I'm talking about a FOSS project with nVidia involvement ;D

But also: this being a FOSS project, people have opened AI-generated PRs. Poor AI-generated PRs. This is indirectly hurting the prospects of your product (by reputation). Might I suggest adding an AI generated PR detector, if possible? (It's not in our guidelines yet but I expect we'll be prohibiting AI generated contributions soon.)

	▲	allisonee 4 days ago \| parent [-]
		totally get where you're coming from--many big open source repos have also been using it for a while and have seen some FP but have generally felt that the quality overall was worth it. would love to continue having you try it out, but also understand that maintaining a FOSS project is a ton of work! if you have specific feedback on the pr--feel free to email at contact@mrge.io and i'll take a look personally and see if we can adjust anything for your repo. nice idea on the fully AI-generated PRs! something in our roadmap is to better highlight PRs or chunks that were likely auto-gened. stay tuned !