This is such an elegant paper. At the risk of minimizing the thought that went into it, I love how it boils down to cleverly introducing zero, twice (to decompose the hard MTP gradient and to show the effect of soft MTP on the gradient's bias and variance).

I wish I could come up with this kind of stuff, haha. Clearly came from spending some time looking carefully at and thinking carefully about MTP heads. Kudos and thanks for an interesting paper!

thank you for this comment! it was really a serendipitous idea—i was playing around with amortizing the KL computation in our previous Next-Latent Prediction work (https://arxiv.org/abs/2511.05963) and this Soft-MTP idea trickled out from it. i sincerely appreciate your kind words, it means a lot to me!