The memory is model-readable but not model-writable, so you still need to train via backprop to get the memory to store useful data.