Remix.run Logo
gchamonlive 8 hours ago

How you type is a poor proxy for code quality. Code quality is a good proxy for code quality. Inspect the code, build a verification pipeline for it, use agents to explore the code and the architecture, see if you can unearth anything fowl.

SwellJoe 8 hours ago | parent | next [-]

I'm not judging based on how they type. I can't see how they type, they vibed the README.

And, it's not my monkey. You can inspect the code, build a verification pipeline for it, use agents to explore the architecture and see if you can unearth anything fowl.

My heuristic is to dismiss purely vibe-coded apps from people I don't know, particularly for security sensitive stuff. If the README is written by a human and is coherent and exhibits some kind of desire and competence to make good software on the part of the author, I'm more likely to trust they drove their agents with care.

Here's the thing: you can make good software with agents, if you exhibit good judgement and put yourself in the path as a gate on quality. Too many clues point at this being loop engineering. And, C for this task, given 100% agent authorship, gives me the ick. Seems like bad judgement or opting out of making judgement calls.

gchamonlive 7 hours ago | parent [-]

Took a look at the readme and seems coherent enough. A readme is also a technical entrypoint, no problem in parts of it being generated, specially if you have quick start, tables and loose documentation there you need updating.

zamadatix 8 hours ago | parent | prev | next [-]

How big a video file is a poor proxy for the encode, quality is a good proxy. The problem is finding the actual quality of a video file is a hell of a lot more work and resources than using a proxy to see if doing so is a good use of our resources. See if you can go the extra mile you described for a few hours/dollars tonight and let us know what you find, it would be appreciated!

gchamonlive 7 hours ago | parent [-]

I do for projects I have interest in running and for my own projects. For instance I had agro with https://github.com/Mord3rca/gamma-launcher so I pulled my trusted agent, hammered the code with architectural and static analysis and made it my own here https://gitlab.com/gabriel.chamon/yagi

zamadatix 6 hours ago | parent [-]

Oh, I'll just have to stick with what info SwellJoe provided using the proxy on this project for now then as it seems none of us have the time or resources to go much further :/. Thanks!

gchamonlive 6 hours ago | parent [-]

You are welcome! My point stands though. Evidence of agentic loop is a bad proxy for code quality. If you are interested in a project you need to do the legwork, vibecoded or not.

zamadatix 5 hours ago | parent | next [-]

Sure - that's always valid - but now I know I'm no longer interested before having put the legwork in! A cool security project that has been reviewed multiple ways already is one thing, a C project nobody else (even the author) was very involved with is another. I don't need to put the legwork in on this codebase to know it's not worth putting the manual legwork in for because I can already get such unverified things out of AI the same as the author, so i'd just review my projects the same as you'd review yours. That's what the proxy provided, not an override of an actual review.

gchamonlive 5 hours ago | parent [-]

That's the spirit. I was thinking about this that you are saying in general, and agents now make for a case that it's often better for you to rollout your own specialized solution than to adopt a more generic project, so it's really important to know when not to waste time with a project like this.

I am currently doing DevOps work describing a legacy infrastructure as code and instead of buying into a paid or opensource tool, it's nicer for me to roll out my own because even though the task isn't trivial, I can custom tailor the solution exactly to how I'm going to use it to conduct the project.

zamadatix 5 hours ago | parent [-]

I can't wait until the really good auditing tools are so cheap we can just run them on everything we see! It's justifiable for work projects but not "fun" yet for me :/

gchamonlive 5 hours ago | parent [-]

I got lucky I think, in which the workflow of using agents is quite fun and addictive for me. But to audit anything you need a basic level of understanding of the underlying architecture, because we aren't there yet to just leave the agent working unattended and have plausible results, so any audit requires careful human involvement.

It's just that... Take this tiny project of mine I did for the fun of it https://gitlab.com/gabriel.chamon/thoracic-atlas-viewer. Basically went to an online interactive thoracic atlas and saw that the files were 404ing for download, but the visualization worked, so I gathered evidence and asked the agent to rebuild the archive for me. I still need to hack together an offline viewer, but it's amazing how much you can do just by having a general idea of what you need to do.

zamadatix 4 hours ago | parent [-]

Yeah, it's more about having a better 0 effort proxy than replacing the need for other audits after (from the ones I've messed with at least).

5 hours ago | parent | prev | next [-]
[deleted]
6 hours ago | parent | prev [-]
[deleted]
yjftsjthsd-h 8 hours ago | parent | prev [-]

It's not "how you type", it's "whether any human so much as laid eyes on the code". I wouldn't automatically discard code from an LLM, but let's put the goalposts where they actually are.

gchamonlive 7 hours ago | parent [-]

How do you know nobody laid eyes on the code in the project?

SwellJoe 7 hours ago | parent [-]

Have you look at the commits? A bunch of empty ones (seemingly all empty ones, though there must be some real ones in there somewhere). It's a bizarre looking repo. I don't even know how you make a repo look like that, but it certainly doesn't allow tracing of what code was added when and why.

It looks suspicious, even if it isn't. All those empty commits look like an intentional obfuscation of something, though I have no idea what.

gchamonlive 6 hours ago | parent [-]

It's indeed strange, it's likely some agent running amok. Doesn't look good indeed, but if I was interested in the project I'd still investigate the code itself