Not an alternative transformer like you asked for, but OptiLLM looks interesting for squeezing more juice out of existing LLMs.