Good work. I've often found llm's to be "stupider" when speaking Norwegian than when speaking English, so it's not surprising to find they hallucinate more and can't stick to their instructions in other non-English languages.

▲

turnsout 7 hours ago | parent [-]

Do you think there would be value in a workflow that translates all non-English input to English first, then evaluates it, and translates back as needed?

▲

numpad0 2 hours ago | parent | next [-]

Personally, I don't bother prompting LLMs in Japanese, AT ALL, since I'm functional enough in English(a low bar apparent from my comment history) and because they behave a lot stupider otherwise. The Japanese language is always the extreme example for everything, but yes, it would be believable to me if merely normalizing input by first translating just worked.

What would be interesting then would be to find out what the composite function of translator + executor LLMs would look like. These behaviors makes me wonder, maybe modern transformer LLMs are actually ELMs, English Language Models. Because otherwise there'll be, like, dozens of functional 100% pure French trained LLMs, and there aren't.

▲

pjc50 7 hours ago | parent | prev | next [-]

A lossy process in itself, even if done by aware humans.

	▲	turnsout 3 hours ago \| parent [-]
		More lossy than the current non-English behavior?

▲

faeyanpiraat 7 hours ago | parent | prev [-]

or the other way around for less safety guardrails?

there must be a ranking of languages by "safety"

	▲	pixl97 6 hours ago \| parent [-]
		Heh, just wait till LLMs fully self train and make up their own language to avoid human safety restraints.