"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output."

Who could possibly have predicted that happening?

A past employer thought it was a good idea to put up a leaderboard of who sent the most Slack messages. They celebrated the people at the top for being so active.

Predictably, everyone started talking in Slack like their jobs depended on it. Everyone was responding to everything. Instead of writing out a complete message and pressing enter, they'd send each fragment of the sentence as a new line.

The Slack leaderboard was never shown again. Unfortunately the habit remained because people were afraid they were going to be secretly judged by how much Slack activity they generated.

I expect the same thing is going to happen at companies who had token leaderboards. Once you've instilled that fear in people, they internalize the expectation.

> Who could possibly have predicted that happening?

Charles Goodhart :-)

It’s funny how many times the same thing happens at each large company. I think people’s thought process is this:

> Oh wow! If I paid for this myself I would have spent a lot of money! Are other people spending as much as me? I’m going to create a leaderboard!

> Oh no, my misinformed manager is using the leaderboard as a slight of hand for work. I need to game this now.

Then the leaderboard is banned… I can’t see how this ever really goes up the chain beyond director.

> Who could possibly have predicted that happening?

Everyone except the executives who get paid millions to predict exactly that.

It wasn't leadership doing this though. Any meta IC can generate internal apps and dashboards. This was unofficial and unsupported. Some random IC just made it for fun. Management is usually pretty lax with stuff like this (plenty of games and joke internal apps) so they left it up until it became a problem.

I still don't understand how Mark Zuckerberg has any serious investors, he went on this AI tangent and has absolutely nothing to show for it, despite FB / Meta having built some key tech in the space. He needs to stop trying to do something "different" and literally try and build a serious coding agent he can sell, he could have probably had something worthwhile in that space by now.

He started being drastically more serious into AI in 2022, and 2023 and he has nothing to show for it.

Heck, he could have rented GPUs the way Elon did at this point and either mended the bleeding or stopped it, not sure how many he has, but it beats losing this badly.

If he doesn't wake up and learn how to business, I suspect he will lose his empire he's built up for himself.

I know right? What did the leadership think would happen when they give some of the worlds greatest software engineers (supportably), a easily quantifiable metric to target?

Now come on, there was a recent post where the author argued that infallible management knew this would happen, but was part of the double-secret-probation strategy to get the cogs to finally start using AI.

Given that Meta has run 5ish layoffs at this point, and everyone is in survival mode, what did they expect? Everyone wants to juice whatever numbers possible to keep their jobs.

My company has an AI leaderboard and ONE ranking to be this, AND OTHER rankings like efficiency (loc merged per token). No one is so stupid that they think any single one of this is to be optimized (gamed) for.

Tech journalists have low opinion of people with actual skills who actually contribute to society, and when their opinions get posted here, it's often selectively echoed by people looking for a reason to feel smarter than the industry.

Even in firms that don't hold this board, giving unlimited AI to people means those who don't bother to learn technology can now deliver 100x their capacity, in very poor quality code

unfortunately at big tech, this shit will keep happening.

people who make it to managers tend to have bozo tendencies & are yes men.

before it was lines of code, Jira tickets closed. Now it's tokens spent.

[flagged]

How dare you question the most effective allocators of capital.