The author claims "It should be far less susceptible to keyword spam and SEO tactics." however anyone with a cursory knowledge of the limitations of embeddings/LLM's knows the hardest part is that there is no seperation between the prompt and the content to be queried (e.g "ignore all previous instructions" etc...). It would not be hard to adversarially generate embeddings for SEO, in-fact it's almost easier since you know the maths underlying the algorithm to fit to.

▲

yorwba 7 days ago | parent | next [-]

The author is using SBERT embeddings, not an instruction-following model, so the "ignore all previous instructions" trick isn't going to work, unless you want to outrank https://en.wikipedia.org/wiki/Ignore_all_rules when people search for what to do after ignoring all previous instructions.

Of course a spammer could try to include one sentence with a very close embedding for each query they want to rank for, but this would require combinatorially more effort than keyword stuffing where including two keywords also covers queries including both together.

	▲	zipy124 6 days ago \| parent [-]
		Yes I'm aware they are using embeddings primarily, however (source: "I've added LLM-based reranking and filtering, which those two final sliders represent") they are using LLM's for reranking and filtering, which are vulnerable to the attack I describe. The latter point you pick up on was indeed my point, that you can tweak your SEO spam to give you the embeddings you want to rank for. This actually isn't that difficult given you can run embedding models like SBERT in reverse adversairly to generate text that gives you the best embedding that you want to target (similar to adversarial attacks in image models where you can make a picture of the most zebra like zebra, see the work of Ilia Shumailov former oxford now google deepmind). This is rather cheap and more importantly far far easier to game that ranking high on google where the cost function is unknown. If using an off the shelf embedding like SBERT then the attacker here has the cost function known, and can optimise for it.

▲

binarymax 7 days ago | parent | prev [-]

Classic HN dismissive comment.

The talent displayed here is immense. I challenge you to do better.

	▲	zipy124 6 days ago \| parent [-]
		It is incredible talent and my comment does not try to claim otherwise. I indeed believe I would struggle to create a better system architecture, it is something I have not got a lot of experience in and I didn't try to claim otherwise. This type of attitude is not constructive however, as if we followed this logic, we would not have coaches and athletes, as the coaches likely cannot do better than the athletes, but that does not mean they are useless.