As this becomes more common it makes me wonder where the LLM ends and the harness begins.

The underlying model may still effectively be a stochastic parrot, but used properly that can do impressive things and the various harnesses have been getting better and better at automating the use of said parrot.