I’ve been conducting AI Coding interviews at my job. Candidate will screen share and use AI-assisted coding environment and tools of their choice.

I ask them to implement xyz thing. What I’m looking for is how they interact with the AI agent. Do they ask the agent to plan first? Do they review the plan? Etc. It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such tools daily.

There are a series of follow-up questions about how to productionize the toy system, which gives some additional signal about how well they understand what they’re making. I sprinkle these in when there’s dead air waiting for the bot to think.

I think we’ll need to evolve and refine this problem and process as the models continue to improve.

▲

Muromec 2 hours ago | parent [-]

>It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such tools daily.

It's also the kind of stuff that somebody can learn in a week, so not hiring the right person who just didn't spend this week of time yet for whatever reason is a loss.

▲

pmg101 2 hours ago | parent [-]

Yeah it tells you nothing of value. It's the Recency Bias taken to its ultimate conclusion isn't it. Anything I learned about AI I learned in the last few months but somehow that's as important as all the experience of my decades long software career? Come on.

▲

Muromec 2 hours ago | parent [-]

It signals you don't have enough time or will to chase the latest stuff, otherwise you would have learned it with other cool kids half a year earlier. It carries some signal, but I would not choose based on this alone. I would even consider selecting against this to a certain point, but I also work in a place that COBOL on a mainframe. YMMV.

	▲	dan_q 2 hours ago \| parent [-]
		> I would even consider selecting against this to a certain point I agree. A technology professional who views themselves as "one of the cool kids" (read: easily manipulated by social media) is a legitimate security threat, as are many of the popularly promoted approaches to "LLM-assisted software developement".