This sounds like we are trying to add an LSTM into a transformer

Sepp would like a word