> LineReader splits input into lines, handles \n and \r\n, and trims the stray trailing \r that malformed input likes to leave behind

Is there a common source of extra \r in malformed inputs, beyond those existing as part of \r\n? Or is this just a dig at Windows-style line endings? If there's something weird going on I think I'd rather fail loudly.

> Bounding the inner scanner to a single line makes “run past the end of a malformed line” unrepresentable rather than merely unlikely.

I don't really see what makes it "unrepresentable", and this reads more like "if you used the right scanning logic, you can't have used the wrong scanning logic".

▲

dspillett 20 hours ago | parent | next [-]

> Is there a common source of extra \r in malformed inputs, beyond those existing as part of \r\n?

Old Macs and some other systems use \r as their EOL, I still sometimes see that with string values in CSV files y code has to deal with (though I don't think I've seen it as an EOL marker in the format itself for a _long_ time).

Sometimes incorrect cleaning steps can leave them behind, such as replacing \r\n with \n but that replacement not being global: it tests fine on strings with zero or one \r\n but subsequent ones will retain their \r. Also code splitting on \n assuming it will always see just that as EOLs will leave trailing \r characters in place. Also, code cleaning EOLs from strings that are supposed to be one-line-only may replace \n (or \n or \r\n, ignoring the possibility of just \r) with a space or a comma and a space, that could be where the \r characters in certain string values I see in files from clients are coming from.

I suspect that off-by-one errors caused by character counting bugs in UTF8/UTF16 handling may cause splitting on EOLs to be a bit off in some cases, though here you will probably be seeing other data corruption at the same time and an errant \r is one of your smaller problems.

▲

inigyou a day ago | parent | prev | next [-]

Sure, start with \r\n, split on \n, now you have a stray \r at the end of every input.

▲

Retr0id a day ago | parent | next [-]

But the preceding clause says it handles \r\n. If you're already handling \r\n, what remaining sources of \r are there, that you'd actually want to silently ignore?

▲

inigyou 21 hours ago | parent [-]

Someone else (possibly you) already split on \n.

	▲	Retr0id 20 hours ago \| parent [-]
		Fair point. I think if something is getting mangled like that I'd rather fail loudly, but it depends on the use case I suppose.

▲

HackerThemAll 21 hours ago | parent | prev [-]

\r\n?|\n

handles all EOL sequences without backtracking. Or write a non-regex equivalent of that.

▲

inigyou 20 hours ago | parent [-]

I'm sure every time you split something on newlines you remember to use a regex.

	▲	HackerThemAll 18 hours ago \| parent [-]
		I tend not to, because it's an overkill, but this regex nicely sums what needs to be done.

▲

thesz a day ago | parent | prev [-]

End of line on Classic Mac is \r.