You can see all the development directly from them: https://github.com/BlinkDL/RWKV-LM

Last week version 7 was released and every time they make significant improvements.