FOSS, Basically it plugs into your existing coding agent workflow and every time you push to prod it'll throw up a questionnaire that checks if you understand how the code you're pushing works.

I certainly feel like certain AI coding practices I have leave me with nothing learned in a frustrating way. Sometimes the most effective way to getting what I want is just describing a goal really clearly, and then bashing the AI until it gets there. That bashing process can take a lot of energy and yet I don’t gain useful skills or a “better way” that I would if I was still coding. I generate more training data for BigAI corp, but that’s a slow return cycle.

Not sure if this is the right solution, but I would like to learn/understand/grow more as I pursue reckless efficiency.

Have you ever heard the idiom "ambulance at the bottom of the cliff"?

After noticing a drop in my understanding of the code I ship, I built and started using a similar skill ("Quiz me on the PR or the current branch"). I didn't tie it to a specific commit, though - to me it makes more sense to look at the PR as a whole.

So far I was positively surprised with the questions it could come up with.

Lol I built this and immediately got frustrated with it and stopped using it. I think I don't like being questioned by someone who I expect to work like magic.

I can kinda understand the motivation here: you push a bunch of slop (or is it deep, genius code?) but you have a kind of nagging suspicion that you should get to know it more familiarly, like back in the good old days a few months back when you sort of understood what was going on. I... don't... think this is going to work...? Remember Confirmation Fatigue? Soon we will have agents that interact with Sphinx for us.

I considered doing this with an open source app I have that let's you get a visual summary on a PR + get quizzed on it, but ultimately decided it's better to keep it an optional thing for people that want to understand AI PRs rather than force it as a gate, the volume is slowly becoming too high for even that. I agree with the sentiment behind this though as I felt this was needed as well.

It’s ironic that all negative comments on here are dismissive because “slop’s baaaaad”. Yes, slop is bad. Carelessness is bad. Lack of understanding is bad. It always has been. Code has been copied from stackoverflow. Built-in methods or std libraries have been used in a wrong way. Edge cases haven’t been covered. Rate-limiting and other hardening tasks haven’t been taken care of until the system broke. The code wasn’t DRY.

Now, people are building tools that not only hide the solutions to these problems in some obscure coding tool, markdown file, or agent harness, but they build gates that help developers to elevate themselves from the darkness of not knowing better. The number of developers roughly doubles every ~3 years, so at any given time, half of the people building software have less than 3 years of experience. Most of them didn’t even study CS before starting their careers. Tools like this will be the new warning voice that told people like us 5 to 10 years ago to never copy code from stackoverflow that you didn’t understand. It’s great.

I think this would be killer for OSS maintainers.

We went from micromanaging code to micromanaging tools and markdowns.

Honestly I love to see people working on this problem ! I actually built a similar project myself (https://github.com/bhurghundii/vibecheck.dev) except it prevents you from merging on the PR level. I’d love to get in touch to talk about it sometimes.

Edit: I see there are a lot of people questioning the value of this. The thing is that there are teams where people are just pumping tons of AI code without understanding their work (literally take the ticket and dump it) and shipping it. culture change is the right answer but v hard but this at least slows things down and is quick to implement

So who is the master of whom?

Whenever I stop for a moment to think about it:

It is absolutely batsh*t crazy insane what SWE has come to in the last cple years.

And yet part of me is loving it.

Its the right problem statement, but we normally let the code review process take care of this.

Humans are now test subjects in the ci pipeline

I mean I see the idea, but at least I use LLMs for code generation in two distinct ways.

The first is when I care about the code. That's when I actually read what it produces and force it to use my mental model of how things should be.

The second is when I just need a solution that sorta fits the requirements provided. Then I don't read the code because I really do not care.

So this hook would either ask me things I know, or it would ask me things I do not care about.

But I guess this is very workflow-dependent.

This is cute and novel, but practically useless. For those who use AI efficiently and effectively, this is useless. For those who don't, this becomes friction they don't want. Who really needs this?

a tool written by AI (mostly) to check code written by (possibly other) AI.

however, it still feels like a lot of work. what if AI hallucinates during the check?

https://www.sphinx-doc.org/

Your slop generator should check if a name exist before claiming it.

"Outsource your thinking"

I'm sorry, what?

People who care about the problem will definitely not be using this as a method to "understand" their code. People who don't care about this problem will continue to not care.

So, who is this for?