> The blue team is more obviously necessary to create the desired product; but the red team is just as essential, given the damage that can result from deploying insecure systems.

> Many of the proposed use cases for AI tools try to place such tools in the "blue team" category, such as creating code...

> However, in view of the unreliability and opacity of such tools, it may be better to put them to work on the "red team", critiquing the output of blue team human experts but not directly replacing that output...

The red team is only essential if you're a coward who isn't willing to take a few risks for increased profit. Why bother testing and securing when you can boost your quarterly bonus by just... not doing that?

I suspect that Terence Tao's experience leans heavily towards high-profile risk-averse institutions. People don't call one of the greatest living mathematicians to check your work when they're just trying to duct taping a new interface on top of a line-of-business app that hasn't seen much real investment since the late 90s. Conversely, the people who are writing cutting-edge algorithms for new network protocols and filesystems are hopefully not trying to churn out code as fast and cheap as possible by copy-pasting snippets to and from random chatbots.

There are a lot of people who are already cutting corners on programmer salaries, accruing invisible tech debt minute by minute. They're not trying to add AI tools to create a missing red team, they're trying to reduce headcount on the only team they have, which is the blue team (which is actually just one overworked IT guy in over his head).

▲

nostrademons 5 days ago | parent [-]

Tao is talking about systems, which are self-sustaining dynamic networks that function independently of who the individual actors and organizations within the system are. You can break up the monopoly at the heart of the blue team system (as the U.S. did with Standard Oil and AT&T) and it will just reform through mergers over generations (as it largely has with Exxon Mobil and Verizon). You can fire or kill all the people involved and they will just be replaced by other people filling the same roles. The details may change, but the overall dynamics remain the same.

In this case, all the companies who are doing what you describe are themselves the red team. They are the unreliable, additive, distributed players in an ecosystem where the companies themselves are disposable. The blue team is the blue team by virtue of incentives: they are the organization where proper functioning of their role requires that all the parts are reliable and work well together, and if the individual people fulfilling those roles do not have those qualities, they will fail and be replaced by people who do.

▲

kibwen 5 days ago | parent [-]

> and it will just reform through mergers over generations

You say "just" as though this is a failure of the system, but this is the system working as designed. Economies of scale are half the reason to bother with large-scale enterprise, so they inevitably consolidate to the point of monopoly, so disrupting that monopoly by force to keep the market aligned is an ongoing and never-ending process that you should expect to need to do on a regular basis.

▲

nostrademons 5 days ago | parent [-]

I'm not saying this is a failure of the system, only that it is the system. My overall point is that systems take the form they do based on available technology, efficiencies of production, lines of communication, and incentives, and that the individual firms involved are disposable actors that are forced by the factors above into economically-rational actions. If the natural form of an industry is monopoly (as most "blue team" industries are), that's what we'll get, and government action can at best delay it.

▲

kibwen 4 days ago | parent [-]

Sure, but this is making the common mistake of viewing government intervention as being somehow separate or outside of the market system, rather than being inside the system. Corporations in a competitive market consolidate to the point of monopoly, they use that monopoly to abuse customers, customers demand their governments intervene, some measure of competitiveness is restored, goto start. This is the system.

	▲	nostrademons 4 days ago \| parent [-]
		That's a fair way of looking at it. Bringing it back to the article's point, the government is part of the "blue team" portion of the system. In that if they don't do their job, and their job includes complex regulations that balance multiple competing factors, then large portions of the system...well, "collapse" is a judgy term, but "function in significantly different ways" gets the point across. Inaction or ineffectiveness of the government effectively creates new "blue team" industries, and distributes power in different ways across the economy.