The use case isn't highlighting a huge amount of code, but local re-entrant highlighting.

You generally never need to highlight a whole file, only the part you're looking at. Which by definition must be fuzzy since you don't have the full source, the part you see will not parse as a complete program so you need to guess. This is actually one of the nastier parts of writing a highlighter, and it must be done for every language.

The lesson is that if you're going to be fuzzy you might as well be learned too and the results are pretty good.

This is more useful than it appears at first glance. It would be just as useful running on the CPU, which I'm sure it can.

▲

umpalumpaaa 5 hours ago | parent | next [-]

Is that really the use case for the related packages mentioned on their website? highlightjs and prismjs are typically used to highlight code snippets on websites… which are at most a few hundred lines long. And are usually highlighted as a whole

	▲	tough 5 hours ago \| parent [-]
		you have to bundle and ship per language syntax dictionaries tho

▲

silon42 2 hours ago | parent | prev | next [-]

No need to guess... For basic system highlighting, one can store the lexer context periodically (maybe per line) and invalidate it if edits are made before it. For more advanced stuff, you need a more complete parser anyway and do it asynchronously (LSP...).

	▲	vidarh an hour ago \| parent [-]
		That is a good optimization, but you still need to figure out the starting state at a given location, and for quite a lot of languages that is not solvable in the local case. E.g. if you're looking at a page of Ruby, you can not in the general case know if it's inside or outside a quoted string, as the quote character can be any arbitrary character (and I really mean any. "% x " excluding the double-quotes is a valid Ruby quoted string where the quote character is space but you can e.g. pick a unicode codepoint you're not going to use anywhere else in the file) So unless you parse from the start of the file, you're left with fuzzy matching, and that can be made good enough the vast majority of time.

▲

normie3000 5 hours ago | parent | prev [-]

> the part you see will not parse as a complete program so you need to guess

Presumably this could also be a useful trait when live highlighting of files when editing, as in-progress typing is likely to be unparseable sometimes.