Attempting a serious but not-a-certified-whatever answer: "Points" do have meaning when properly used as a kind of moving-average tool for forecasting within a particular context.

Problems arise when people try to perma-peg them to particular tasks, or (worse) man-hours or (much worse) man-hours across teams. Even just encouraging the humans to answer in terms of hours/days taints the accuracy of the forecast by introducing a kind of bias.

Claude, vibe code me an entire startup, the actual product doesn't matter, but it should all be based on the incredible pun "turn 'sorry' points into story points."

/goal get accepted into Y Combinator, you have an unlimited token budget, be bold.

EDIT: no, do not just make a product that gives away your unlimited token budget to users for free!

I'm primarily using local models. I turn on thinking visible and try to review at speed what it's doing.

Other than that, no. I drag myself out to poke around occasionally because at times the local models get to far into context and refuse to do simple tasks.

This is a recurring issue for us. Very small team. We move fast and pretty loose.

Dev + AI spend 3-4 hours on a project plan, there's a "wait a minute" moment, and finally they spend another hour dialing it back to a solution that could have been built, tested, and deployed in 2 hours.

Example: Someone was setting up a dev environment with multiple DB migrations from different branches - AI planned this wild 8 phase solution with a pretty fancy cutover event.

In review I essentially said... "Wait, isn't this a dev environment? It doesn't need 0 downtime, why not just destroy and recreate the DB" and it turned into a <1000LOC script.

Technically the original plan would have worked, it would have been more robust, but it would have taken a good deal more time to implement.

Some of this falls on the devs to know what fits our team well, what's realistic, what's obviously overengineered, etc... But some of it feels like AI just defaults to the most complex version of a thing. I catch it SUPER frequently. (And unfortunately some devs think that more complexity means it's a better solution)

Matches my experience to a tee.

And don’t forget the company also spent a bunch of money in tokens for the initial author to implement the thing poorly.

It's impressive for its total size if you need local inference.

Though it's significantly slower in Token/s and also thinks a lot more without matching the same intelligence (xhigh qwen27b scores lower than haiku's medium setting, and haiku-med is $0.05 per task compared to Qwen27B's $1.01 on AA's comparison)

It seems like more a backup if you need to work offline, imo, unless time doesn't matter and/or your electricity is free. Or you want independence from the labs (fair enough).

For the last few weeks I've been using Luna with Medium Reasoning for routine debugging, installing/updating dependencies, test creation/fixes, CLI/config miscellaneous problem solving, etc and it's been solid. I can let it churn away for a half hour and it barely moves the needle on remaining usage on my Plus plan. Previously I had been using Sol Low/Medium and would frequently hit the 5 hour limit doing those kind of tedious automation and routine problem solving tasks.

Our ai basic analysis for SRE / k8s based platform is haiku and its surprisngly good.

I wouldn't even tried it, i would still just go with even Opus (we don't have that many alerts) but it really surpsied me.

When i ran into usage limits a few days ago i switched most to Sonnet and again was surprised how good it is now.

As a point of reference, I tried an experiment with Claude Code and the latest Opus the other day. It was work for my company, so this was using API tokens and not the individual user plans that have non-commercial terms.

A simple task, migrating a typical password reset flow as part of updating a long-lived web application from legacy libraries and software architecture to modern equivalents, apparently cost roughly the same as 2 months of Pro subscription, over the equivalent of about half a working day in wall time.

It produced code of decent quality at a small scale, but it wasn’t always on point architecturally. It also had a tendency to drift off topic and try to tangle up other changes it decided should be made with the main change we were supposed to be working towards. So even for a routine task, based on a plan developed using the harness first and with the agents working under close supervision, a near-SOTA model is still producing quality on par with a decent mid-level developer but substandard for anyone senior+ in this case.

Moreover, based on a direct comparison with other migration tasks of similar complexity that I’d already done by hand, it was actually a bit slower overall to work this way. I had to babysit Claude throughout and review everything it proposed carefully, both to avoid subtle errors (it would have made several) and to prevent drifting off track. I also had to spend a significant amount of time cleaning up its final output to an acceptable standard after the session. Those two overheads more than cancelled out the much faster code generation an LLM offers under favourable conditions.

So for now, I remain sceptical about these high multiples of improved productivity that I keep seeing claimed online from people who are apparently writing almost everything using AIs now. I could certainly have achieved a multiple of my normal productivity by YOLOing everything without reviewing it in detail and then accepting the output code without tidying anything up. However, I doubt this codebase would still have been good enough for normal human developers to work on it reasonably after even 10 or 20 AI-led sessions like that. The architecture would have degraded significantly and the test suite would have been large and largely pointless. And again, this wasn’t rocket science in this experiment, it was completely unremarkable maintenance of a relatively small and simple web application.

Is there something in the ToS that says you can't give each of your employees a personal $200/mo subscription... and if they happen to use it at work, well, that's their business?

Even if so, it might be worth spinning up a separate LLC for each division, given what they are charging for the API.

> The good thing with velocity is that it doesn't care which way you're going as long as you're going there fast, so you can never be wrong

Velocity famously being a vector with a direction. Go in the wrong direction you'd have negative velocity.

Your description would be better for the concept of speed (IE the magnitude of velocity or directionless velocity).