Remix.run Logo
▲ pu_pe 2 hours ago

> It’s got some very expensive practices, where near-free practices would produce the same or better results.

I think tokens are near-free already, even for those of us outside Anthropic.

More importantly, your philosophy comes from a time where thinking had to be front-loaded: you try to predict everything that can go wrong, then design deterministic tests that can catch those things and a decision tree to figure out what to do in each case. This approach was the only viable way until now, but maybe that will change. Having the equivalent of a senior engineer thoroughly review each error with full context of what the application and feature is trying to achieve could perhaps uncover different insights, or cover edge cases more smoothly.

▲Grimburger an hour ago | parent | next [-]

> tokens are near-free already

How long do you think that will last?

Am over 11 billion tokens this month on a subscription. In API usage its close to $5000 and there's still 13 days left in the month.

Eventually the music in musical chairs stops. Already many anecdotal reports of the always opaque limits being lowered across the board. On a gemini pro account for personal use I can use a weeks worth of Argon in about 20 minutes on agy in the last month.

ChatGPT Pro I can use the weekly limit in under 2 days, this is with significant context clearing and handoffs.

The cost to provide frontier models isn't cheap and it's entirely subsidised by a winner-takes-all mentality from investors which in all likelihood won't come about.

▲olkre 14 minutes ago | parent [-]

The cost halves every year so far. Current level of llm will get cheaper

▲curt15 5 minutes ago | parent [-]

Cost or price? ANT's shareholders and creditors aren't going to wait forever for ROI.

▲skohan an hour ago | parent | prev | next [-]

It's great to have that option available, but it's incredibly inefficient to just throw GPU hours at a problem that can be solved deterministically.

▲pydry an hour ago | parent [-]

Good for anthropic margins though. The absolute last thing they want with the upcoming IPO is people conserving tokens and ruining the already dying hockey stick narrative.

▲castis an hour ago | parent | prev | next [-]

> Having the equivalent of a senior engineer thoroughly review each error with full context of what the application and feature is trying to achieve could perhaps uncover different insights, or cover edge cases more smoothly.

No one is debating this. The debate is that an agent performing validation work via its own chrome plugin and internal reasoning will always cost more than performing that same validation work with deterministic tools.

▲pu_pe an hour ago | parent | next [-]

It isn't the same work, though, so the question is whether the added cost is worth it. Yes it costs more, but the price of tokens has fallen 100x in the past two years and it will continue to drop.

▲chrisweekly an hour ago | parent | prev [-]

"non-deterministic" -> "deterministic"

right?

▲castis an hour ago | parent [-]

yuppp, thank you :D

▲jgalt212 2 hours ago | parent | prev | next [-]

As far as timelines go both property based tests and fuzzers predate this token maxing strategy.

▲bayindirh an hour ago | parent [-]

But they are old, well tested, and not AI based, hence bland.

In this age of AI, anything well understood and effective while being efficient is out. These tokens must be used, atmosphere must be heated.

It's November. Winter is coming.

▲skydhash 2 hours ago | parent | prev [-]

> you try to predict everything that can go wrong, then design deterministic tests that can catch those things and a decision tree to figure out what to do in each case

No real software engineer think that way, as things that can go wrong are near infinite. Instead you design the right thing and write tests that verify that it’s indeed the right thing and add some more that are the wrong thing, especially at the edge of definition. Then you rely on the dev to not make very obvious errors like coding something that is not in the requirements or is completely opposite.