I like the idea, just that the examples are reproduced from the training data set.

How does it handle unknown queries?

It mostly doesn't, at 9M it has very limited capacity. The whole idea of this project is to demonstrate how Language Models work.