I'd add on that the models have a very poor sense of time and the complex changes in world state that occur as time passes.

Training with memory is an interesting idea...