I feel like this is related to these issues (with somebody attempting this approach for real):
https://github.com/chardet/chardet/issues/327
https://github.com/chardet/chardet/issues/331
That's worth its own submission and discussion.
It has been submitted last week, happy reading:
https://news.ycombinator.com/item?id=47259177