If you need fewer workers due to the increased productivity, then the number of white collar workers (the TAM) won't be as large as anticipated.

That’s never really how efficiency gains tend to work except in places where entire industries ceased to exist (printers, weavers, clothes washers, etc).