While you are correct at a higher level, comparing bigrams/trigrams would be less compute not more because there’s fewer of them in a given text

▲

measurablefunc 2 hours ago | parent [-]

I'm correct on the technical level as well: https://chatgpt.com/s/t_698293481e308191838b4131c1b605f1

▲

refulgentis 2 hours ago | parent [-]

That math is for comparing all n-grams for all n <= N simultaneously, which isn't what was being discussed.

For any fixed n-gram size, the complexity is still O(N^2), same as standard attention.

▲

measurablefunc 40 minutes ago | parent [-]

I was talking about all n-gram comparisons.

▲

refulgentis 18 minutes ago | parent | next [-]

Thanks for clarifying. I was hoping to clarify the disconnect between you two, looked like on on "bigrams, trigrams, & so on." It reads idiomatically as enumerating fixed-n cases. Parsing "& so on" as "their simultaneous union" asks quite a bit of those two words. Either way, as ChatGPT showed you and you shared, all-ngram comparison brings us to O(N^3), still several exponents short of N^10 that started this thread.

	▲	measurablefunc 13 minutes ago \| parent [-]
		This is getting tiresome. I can make the operations as complicated as necessary by comparing all possible permutations of the input string w/ every other permutation & that will not be reducible to standard attention comparisons. The n-gram was a simple example anyone should be able to understand. You can ask your favorite chatbot to compute the complexity for the permutation version.

▲

24 minutes ago | parent | prev [-]

[deleted]