Agree to all of this but I would make a minor correction: the interpretability is true under certain conditions, e.g., you use something like n-grams and train an SVM with a linear kernel. You could change the representation back then too, e.g., LDA (topic models), Latent Semantic Indexing, and of course, you could change the SVMs kernel as well, e.g., use a sequence kernel [1]. Some of these were much less interpretable. But the common pipeline back then was indeed n-grams + Linear SVM. Today it is easy to modernize the pipeline by just changing the representation step to use an embedder like ModernBERT. I have a "one page" classifier if you want to try it out - [2].

[1] I went down the rabbit hole of using sequence kernels quite a bit back then, see https://arxiv.org/abs/1910.13800, but the timing was terrible. RNNs and variants like LSTMs were beginning to become popular, and everyone (including me) switched.

[2] https://pocket-text-lab.abhishek-ghose-82.workers.dev/