We've got quite a list of history-only LLMs brewing on the Models Table.

https://lifearchitect.ai/models-table/

This one is easiest to talk to in a HF space:

https://huggingface.co/spaces/tventurella/mr_chatterbox

These are more like Small Language Models since the amount of textual data from the past is extremely limited, and most of what's out there hasn't even been digitized.