There were some good insights in there. I like the idea of changing the hiring interview process to focus on testing code review ability. I feel like this would have been useful even before AI.

A candidate who can identify tradeoffs present in some code and make insightful comments is likely good at systems thinking. It's a highly effective way to test someone's knowledge, intelligence and taste.

It's actually brilliant because it provides the company with a way to actually improve their engineering posture since the company could land on a candidate who is more skilled than the engineers doing the interviewing. Many times in my career, I met engineers who seemed mediocre at first because they didn't know a fraction of what I knew, but I later found out they also knew a fair amount that I didn't know.

I've even had an extreme experience in the past year where a colleague seemed to struggle with basic code constructs and for the first month I was thinking to myself "this guy doesn't know how to code". He asked really basic questions and said he didn't know because his background was Python and not Typescript... But I know both of these languages and I just couldn't make sense of this explanation given the kinds of questions he asked and mistakes he made. If I had been asked to review him at the time, I would probably have given him a pretty bad review but after about 1 month and a half, this guy accelerated and literally became the most productive engineer on the team. No leetcode coding test could have predicted this.

Most leetcode tech interviews are a series of puzzles which most company insiders can solve but they never include problems that the candidate could solve but which the interviewer could not.

Leetcode interviews are horrible because they test a tiny subset of moderately difficult questions under time constraints and ignore a much larger set of problems that are much more complex. There is an incorrect assumption that someone who can solve extremely complex problems can also solve moderately complex problems under time constraints. This is absolutely not the case. It's almost mutually exclusive in fact since people who work on complex problems don't have the time or interest to practice solving simpler problems so they can never solve those fast enough to compete with fresh university grads who have been practicing those for years and don't know anything else.

On a different topic, I was sceptical of this comment:

"I honestly think you can have a fifteen thousand line PR and say, I need a human to review these three lines."

15k lines is a lot of code. I could destroy any software project, irreparably with 15k lines of code and not one engineer out of hundreds would recognize it unless they read carefully. You can absolutely destroy a codebase with 15k lines of code, without any obvious backdoors or malicious code. How would I do it? I would invent counter-productive abstractions and write a lot of unit tests for them to lock down the design... Then I would watch other engineers build on top to further lock it down... Let the flawed design accumulate debt for a few years until the entire codebase becomes slow, insecure and totally unmaintable. Nobody would ever remember that I'm the one who set the project on a bad course. Nobody ever suspects the person who invents the complex abstractions and who everyone comes to with questions... Also most engineers are afraid to ask why we need a SocketContextManager or a TaskContextSwitcherMediator or the TaskOrchestrator and TaskOrchestratorFactory that comes with it... Nobody will ask why we need 10 different Helm charts to split things up into microservices... Nobody will ask why we need 1000 top level dependencies with a total of 10k nested dependencies. Nobody will question those decisions because they're afraid they will look dumb for questioning complexity.

You cannot question system complexity without acknowledging that you find something to be complex... And this can be turned against you; "Oh so this is too complex for you?"

So my view is that every single one of these 15k lines needs thorough analysis. Each of those 15k lines represent the branches from which the next generation of twigs will sprout. If that branch isn't pointing in the right direction, better just cut it off as soon as possible before it becomes a central part of the codebase.

Counterpoint: you're doing a 15k line concrete implementation of a spec, or a module with fixed integration points.

There could still be a lot subtly wrong, but there are going to be stress points and high leverage validation methods you can use to avoid reading the whole 15k.

Yeah, that didn’t make any sense. If it’s functional, it impacts the system. 15k lines sure sounds like a lot of bloat.