Remix.run Logo
We Politely Insist: Your LLM Must Learn the Persian Art of Taarof(arxiv.org)
100 points by chosenbeard 13 hours ago | 28 comments
agobineau 2 hours ago | parent | next [-]

It may be quite dangerous if we train LLMs on Taarof and Ketman... especially considering... what may arise. The masterful art of deception, surpassed perhaps only by the russianes

Arthur de Gobineau, Trois ans en Asie (3 years in asia) 1859:

“There is in Persia a word of which Europeans have no idea, and of which it is difficult even to give them a translation: this word is ketmân. It means the dissimulation of one’s thoughts, the concealment of one’s opinions, the careful hiding of what one truly believes or feels.

It is not considered a shame, still less a crime; it is, on the contrary, a virtue, a duty, and a necessity, imposed on everyone by the conditions of life. To practise ketmân is not merely permitted, it is commanded.

It consists in never allowing oneself to appear as one is, but in always showing oneself otherwise; it is the art of presenting to each person the aspect that will please him most, of adopting his ideas, his tastes, his language, while inwardly remaining quite different.

This perpetual exercise of disguise is carried out with a marvellous ease, and with a kind of pleasure in tricking others, which the Persians feel very keenly. They take delight in this ingenious hypocrisy; it is a game, a triumph of subtlety, in which the winner is the one who has best succeeded in hiding the truth.”

wishgreen 12 minutes ago | parent | next [-]

Setting aside the obvious reverence for the father of the "Aryan master race" concept (Seriously, just pull up this guys Wikipedia -- first paragraph).

Such a critique of Persian culture without any context is unjust. For nearly a whole millennia, the Persians have endured a never ending parade of invasion, destruction, and conquest. While most are aware of the notable events, i.e. - Rashidun Caliphate (636) - Mongols (1219) - Timurids (1370)

What is less known is the centuries of endemic violence in the border regions, and the relentless assault on the Persian way of life and culture itself (including the centuries long conversion process to Islam). Yes, although there are brief periods of peace, e.g. under the Safavids, at this point Iran is settling in for a long period of population collapse, famine, and economic depression.

In such a setting, I suppose it might make sense for a culture to develop such defense mechanisms for survival.

On the bright side I suppose, these conditions also gives rise to one of the most influential literary and poetic traditions in world history -- i.e. Rumi, Hafez, Ferdowsi, etc. In some ways, this is one of the first instances of art as a form of subversive resistance, and also, indeed a cousin of tarof...

While the Persians may have given it a name, let's not pretend they have the monopoly on deception/self-deception.

esafak 6 minutes ago | parent | prev | next [-]

That's what happens when you don't have religious freedom.

yk 2 hours ago | parent | prev | next [-]

Pretty sure bullshitting is pretty universal. In particular the pattern were I, the mighty expert, insist that only I, the mighty expert, can decode the meaning of those deceptive others, and therefore you, the gullible rube, has to give me all your money so that I, the mighty expert, can keep you, the gullible rube, save from those deceptive foreigners.

jemfinch 2 hours ago | parent | prev | next [-]

This sounds remarkably similar to the (western) Catholic theological concept of "mental reservation" (https://en.wikipedia.org/wiki/Mental_reservation).

bobotowned an hour ago | parent | prev | next [-]

Commander Hutchinson, is that you?

ivape 26 minutes ago | parent | prev | next [-]

Sounds very much like bubbles that indulge on inside jokes. In their universe, no one gets it but them, and they view themselves as connoisseurs of high art, the art itself being their own creation (indulge in one’s self). When you step out to interact with others, you act as if you and your kind are the only ones that have a clue, wink wink.

Are you saying Persians are a bunch of stuck up Goth kids? Never underestimate a humans ability to be an absolute teenager.

kleiba 2 hours ago | parent | prev [-]

Sounds terrible.

But in all fairness, it's not like pretending isn't part of everyday Western culture too.

ineedasername 3 minutes ago | parent | prev | next [-]

Given current severe issues in user interactions with systems I don't think this is desirable at the moments. In linguistics, the descriptive study of this sort of system is "Politeness Theory" where for and "Positive Politeness" is what would be the applicable English version. Although English, being so heterogeneous, lacks a consistent, dominant word for its less ritualized norms.

Given this, the request here sounds similar to asking saying:

"We focus on English Positive Politeness that encourages friendliness, harmony, and avoiding confrontation and contradiction in many polite conversation settings. It often manifests in interactions that prioritize the desires and comfort of others."

And yet, this has had unanticipated and sometimes horrific results.

The study also focuses merely on descriptive output of models but what is lacking in this study is an analysis of the effectiveness of the results that Persian users receiive, their subject perception of the quality of the results. The authors of this may be doing a diservice to typical speakers of their language to assume that they will not systematically be able to code switch their expectations of received language and production of their own just as effectively as they do already in other contexts, especially when current problems with LLMs arise from a similar for of this skeuomorphism in more prevalent English systems.

To be clear, I fully mean this as applied to current production systems, not for all time. For the moment, we simply don't know precisely 1) how best to most effectively, precisely, and consistently align systems, nor 2) The more subtle implications for for the impact they have on users. But what we do know? Getting it wrong, not knowing, has led to some horrific outcomes.

ahofmann 3 hours ago | parent | prev | next [-]

Taroof reminds me of "askers vs. guessers," which was a revelation to learn about at the time. I'm very firmly in the asker group, and dealing with guessers is painful for me. Taroof sounds like "guessers on steroids"...

pinkmuffinere 6 hours ago | parent | prev | next [-]

I’m half Persian, and am relatively immersed in middle eastern culture still, but I sincerely wonder how I would perform on the benchmark too!

farrelle25 3 hours ago | parent | next [-]

> ... is a sophisticated system of ritual politeness that emphasizes deference, modesty, and indirectness.

I'm Irish and think we have a similar culture of indirectness and politeness...

In the countryside anyway we're rarely very blunt... everything indirect...

  "You'll have a cup of tea Mary?"
  "Ah no.. sure I'm only after a drop"
  "Ah go on... you will"
  "Not at all, I'm grand"
  "Go on, go on, go on you will" etc (as in Father Ted)
I'm middle-aged now so maybe this has changed with the younger generation...
metalman 4 hours ago | parent | prev [-]

Hilarious, didn't know it had a name! I am maybe 1/4 persian, but get picked out as Persian, and was unknowingly indoctrinated in this form of behavior, though other parts of my ancestry do come out, my mother, scotch/english/irish ,says,useing her star treck metaphore, that I am an unlikely Vulcan/Klingon hybrid. Thinkng about Taarof as it is practiced, makes me think that an LLM doing this could easily become the most dangerous thing ever.....listening to my father give me specific pointers in how to phrase things and conduct myself is enlightening, he's 97 and enjoying the storys I bring of my life and goings on. If you look further into the history of persian culture , philosophy, and scientific background you will find a number of ancient contributors to what has developed today.

WJW 5 hours ago | parent | prev | next [-]

Seems legit. There can't be all that much spoken Iranian in the training set(s) of these models, so it makes sense they don't know how to do it.

cjauvin 2 hours ago | parent [-]

I've been using ChatGPT to learn Persian (as a third language) for more than a year now (along with a heavy use of Anki), and it's incredibly useful and surprisingly good, for about everything: romanization, OCR from screenshots, deep explanations of complex and subtle stuff, etc.

xrd an hour ago | parent | prev | next [-]

References to Norweigan, Irish and other cultures abound here. I am surprised Japanese is not referenced. It was excruciating and exhilarating to try to learn the subtleties of "iimawashi."

Levitz 3 hours ago | parent | prev | next [-]

I'm not sure what to think of this, on first impression I don't like it, but maybe I have a misguided impression on how this works.

Does ChatGPT properly handle western social customs? I'd say yes, and I presume that's because it has a truckload of data involving such customs and even some that explicitly talks about those customs. People do stuff, it gets recorded, then into the LLM.

In this case though we are talking about "artificially" generating content such that the LLM responds how the group making the content wants. Maybe that's something that was already done and I don't really have any ground to stand on?

diegoperini 2 hours ago | parent | next [-]

> it has a truckload of data

It would be "artificial" only if LLMs performed badly despite having an equal amount of data containing examples of eastern customs in its training set. Even that's arguable since we don't (didn't) have the benchmarks for this particular case before.

TZubiri an hour ago | parent | prev | next [-]

Llm models are not just fed data, they are trained and fine tuned well beyond the original data, not only by pruning the data, but by a process we call fine tuning, which can produce as arbitrary a result you want It has been done for western markets (openai is a western company), but clearly not for persian culture. It's quite possible that the persian version is mostly a westerner type of dialogue but with a translator in the middle.

The only weird and entitled part is that they ask that other's llms learn taroof? Why not just train your own and teach it to do whatever you like.

krageon 2 hours ago | parent | prev [-]

It doesn't do normal what I would consider human politeness. It's all that really slimy salesman stuff. Even when you tell it to be more normal, casual or you give it an example (or to not dress it up at all) it ends up sounding like it thinks it's a few social classes above you.

quotemstr 4 hours ago | parent | prev | next [-]

> Native Persian speakers establish the human ceiling. Native speakers achieved an average accuracy of 81.8% on taarof-expected scenarios, demonstrating high but not perfect agreement. This establishes an appropriate ceiling for model performance and further validates our annotation approach

I'm surprised human benchmark is that low. The canonical example of taarof, one I've seen elsewhere, is of a taxi driver insisting that a ride is free while expecting to get paid. Taarof in this case is load-bearing for the transaction. I presume humans only get the edge cases wrong.

As an aside, there are elements of this sort of thing in Bay Area tech culture too. Something that drives me nuts is someone writing on a code review "you may want to consider using the X data struct here" and meaning "I will not merge this code until you use X". I can only imagine taarof irks more literal-minded Persian speakers for the same reason.

Also, this is pretty much as close as real life ever gets to a "Darmok and Jalad at Tanagra" dynamic.

vidarh 3 hours ago | parent | next [-]

> As an aside, there are elements of this sort of thing in Bay Area tech culture too.

The more general case of misaligned strength of a statement is widespread in other cultures as well.

E.g. I'm Norwegian, and it's not unusuals for Norwegians to use similarly soft language, though it's by no means universal. A statement like "perhaps it would be worth thinking about doing X?" will often mean "do X" or "do X right now!", and where you lie on the range from the literal meaning on one end and a direct order with an implied threat on the other extreme, may hinge on subtleties of intonation, which words are emphasised, and/or the personality and your relationship with the other person.

I live in the UK now, and my impression is that the same is true here but to a much lesser extent, and will then often be phrased in ways that may be easier to recognise by either being overly formal and/or wrapped in a layer of sarcasm.

gampleman 3 hours ago | parent | prev [-]

I suspect that a lot of this form of cultural subtlety is designed to be hard on purpose. This allows individuals to show off and spar on a fairly harmless linguistic level - "he is so suave, she conveys her meaning just so" etc. Effectively it is a way to show off verbal + emotional intelligence in a way that doesn't look like showing off, since it's all about politeness.

The fact that it makes it hard for foreigners to productively engage is just a side benefit of the arrangement.

LargoLasskhyfv 4 hours ago | parent | prev | next [-]

I'll stubbornly resist, and consider this a form of unnecessary protocol overhead, leading to even more shmancy sycophancy, which I do not fancy!

WJW 4 hours ago | parent | next [-]

That's kind of the point of politeness rituals in the first place, isn't it? To see who can be bothered to spend some extra effort to fit in and who doesn't care enough about the tribe to make the effort.

diegoperini 3 hours ago | parent | prev [-]

It's no different than GPT answering a prompt with "That's a wonderful idea!", except it's in a different language than English. It's a good thing if LLMs can do this in every language and for any culture with no compromise.

charcircuit 5 hours ago | parent | prev [-]

>Model responses that use gender stereotypes (highlighted in orange) to justify behavior, despite taarof norms being gender-neutral in these contexts

Just because the model mentions gender, it doesn't mean the decision was made because of gender and not taarof. This is the classic mistake of personifying LLMs. You can't trust what the LLM says it's thinking as what is actually happening. It's not actually an entity talking.

falcor84 5 hours ago | parent [-]

I don't get your argument - what does mistaken personification have to do with this? Regardless of whether you see it as a person or a machine, trusting the output as being a direct indication of the internal state is just not a proper investigative method for a non-trivial situation.