The difference in response time - especially versus a regex running locally - is really difficult to express to someone who hasn't made much use of LLM calls in their natural language projects.

Someone said 10,000x slower, but that's off - in my experience - by about four orders of magnitude. And that's average, it gets much worse.

Now personally I would have maybe made a call through a "traditional" ML widget (scikit, numpy, spaCy, fastText, sentence-transformer, etc) but - for me anyway - that whole entire stack is Python. Transpiling all that to TS might be a maintenance burden I don't particularly feel like taking on. And on client facing code I'm not really sure it's even possible.

▲

noprof6691 19 minutes ago | parent | next [-]

They're sending it to an llm anyway tho? Not sure why they wouldn't just add a sentiment field to the requested response shape.

	▲	FuckButtons 12 minutes ago \| parent [-]
		because a regex on the client is free vs gpu compute is absolutely not.

▲

cyanydeez 2 hours ago | parent | prev | next [-]

So, think of it as a business man: You don't really care if your customers swear or whatever, but you know that it'll generate bad headlines. So you gotta do something. Just like a door lock isn't designed for a master criminal, you don't need to design your filter for some master swearer; no, you design it good enough that it gives the impression that further tries are futile.

So yeah, you do what's less intesive to the cpu, but also, you do what's enough to prevent the majority of the concerns where a screenshot or log ends up showing blatant "unmoral" behavior.

▲

true_religion 2 hours ago | parent [-]

This door lock doesn’t even work against people speaking French, so I think they could have tried a mite harder.

	▲	bigbuppo 9 minutes ago \| parent \| next [-]
		There are only Americans on the internet.
	▲	sebastiennight an hour ago \| parent \| prev \| next [-]
		En toute honnêteté, je pense avoir dit "damn it" plus d'une fois à chat gépété avant de fermer la fenêtre dans un accès de rage
	▲	ben_w an hour ago \| parent \| prev [-]
		The up-side of the US market is (almost) everyone there speaks English. The down side is, that includes all the well-networked pearl-clutchers. Europe (including France) will have the same people, but it's harder to coordinate a network of pearl-clutching between some saying "Il faut protéger nos enfants de cette vulgarité!" and others saying "Η τηλεόραση και τα μέσα ενημέρωσης διαστρεβλώνουν τις αξίες μας!" even when they care about the exact same media. For headlines, that's enough. For what's behind the pearl-clutching, for what leads to the headlines pandering to them being worth writing, I agree with everyone else on this thread saying a simple word list is weird and probably pointless. Not just for false-negatives, but also false-positives: the Latin influence on many European languages leads to one very big politically-incorrect-in-the-USA problem for all the EU products talking about anything "black" (which includes what's printed on some brands of dark chocolate, one of which I saw in Hungary even though Hungarian isn't a Latin language but an Ugric language and only takes influences from Latin).

▲

mlmonkey 38 minutes ago | parent | prev [-]

> Someone said 10,000x slower, but that's off - in my experience - by about four orders of magnitude.

You do know that 10,000x _is_ four orders of magnitude, right? :-D

	▲	jonbwhite 28 minutes ago \| parent [-]
		OP is saying that in their experience it is more like eight orders of magnitude