Look up Amdahl's Law and Universal Scalability Law.

The math doesn't care whether the nodes are people, CPUs or language models. If agent A's next action depends on what agent B decided, you've introduced a sequential dependency.

Note that Amdahl's rule doesn't capture the practical situation.

1) The purpose of algorithms is ultimately to create value, not compute some fixed value X. This is important as it gives flexibility to choose different value producing tasks where parallelism dominates over serial tasks, whenever the the latter becomes a bottleneck.

2) In terms of producing value, perfect accuracy or the best possible solutions are not always necessary. Many serial tasks can become very parallel tasks when accuracy or certainty do not have to be complete.

3) Solutions that are reusable changes the math further. No matter how serial a calculation is, if something is calculated that can be reused, that serial part becomes effectively order O(1), after calculation if reused exactly, but as neural network demonstrate, many serial tasks become very parallelized after training a model that can be reused for now a wide class of specific problems. Resulting in very amortized serial computing costs.

It doesn't matter how many steps something takes, if those steps are now in the past and the value is "forever" reusable.

4) The economics of serial and parallel computation are not static, but improve relative to economic value achieved. Meaning that demand for cheaper serial time and currency costs result in improved scaled up hardware that delivers cheaper serial costs. This may have less impact than the previous points, but over years makes a tremendous difference on top of all those points.

This can go on.

The point being Amdahl's law certainly applies to specific algorithms, but is not the dominant determinant of computing in general, and not useful application of computing to a significant degree, where problems can be strategically chosen, strategically weakened or altered, and can be strategically fashioned to create O(V) of value - to balance any O(S) cost of serial computing, via direct reuse and generalization.

The point is that we don't need an equivalent number of nodes (agents) as we needed people.

The computer flattened the coordination dependencies of that room full of people by doing all the calculations by itself. As they get smarter, you can theoretically assume 1 agent could eventually run the entire US federal government.

In the historical [human] computer example; if 15,000 calculations needed to be done, a CPU doesn't need to wait on Bob to come back from lunch to do the next 20 calculations...and doesn't need to wait on Alice to combine his work with the 20 calculations done by Jane...and doesn't need Bill to wait for everybody to be done to double check Jane's work.

The CPU does all 15,000 calculations instantly, by itself. This will be similar with AI agents.