It depends on the type of task. There was an interview on Dwarkesh with an OpenAI researcher and he said that massive multi-agent swarms can increase capability on some problems that are very parallelizable, but there are diminishing returns. But not all tasks can be parallelized like that.

Where it gets scary is the fact that efficiency is going up 10x per year, so the cost of running 1000 agents will cost the same as running 1 agent in 3 years if the trend holds.