By that logic we should also consider the case of cutting the number of layers in half because that would also reduce hidden state between token generation.

In your generalized example I think the concern is when the additional evaluation effectively becomes a replacement for CoT, where something like the coconut research could replace it completely.

However, I don’t think we’re anywhere close to that with Astra.