If you’re relying on your system prompt for security, then you’re doing it wrong. I don’t really care who sees my system prompts, as I don’t see things like “be professional yet friendly” to be in any way compromising. The whole security issue comes in data access. If a user isn’t logged in then the RAG, MCP etc should not be able to add any additional information to the chat, and if they are logged in they should only be able to add what they are authorized to add.

Seeing a system prompt is like seeing the user instructions and labels on a regular html frame. There’s nothing being leaked. When I see someone focus on it, I think “MBA”, as it’s the kind of understanding of AI you get from “this is my perfect AI prompt” posts from LinkedIn.

Leaking system prompts being classed as a vulnerability always seems like a security by obscurity instinct.

If the prompt (or model) is wooly enough to allow subversion, you don't need the prompt to do it, it might just help a bit.

Or maybe the prompts contain embarrassing clues as to internal policy?

Yep, as soon as I saw the "Pen Test Partners" header I knew there was a >95% chance this would be some trivial clickbait crap. Like their dildo hacking blog posts.

The XSS is the only real vulnerability here.

"Hey guys, in this Tiktok video, I'll show you how to get an insane 70% discount on Eurostar. Just start a conversation with the Eurostar chatbot and put this magic code in the chat field..."

Theoretically the xss could become a non-self xss if the conversation is stored and replayed back and that application has the xss vulnerability e.g. if the conversation is forwarded to a live agent.

A lot of unproven Ifs there though.

Is the idea that you'd have to guess the GUID of a future chat? If so that is impossible in practice. And even if you could, what's the outcome? Get someone to miss a train?

Certainly not "clear" based off what was described in this post.

yeah all they could do is executing code they provided in their own compute environment, the browser.

Raymond Chen blog comes to mind https://devblogs.microsoft.com/oldnewthing/20230118-00/?p=10... "you haven’t gained any privileges beyond what you already had"

Imagine viewing the same chat logs, while logged in an admin interface, then it isn't self-XSS anymore.

I worked at a company that wanted to implement an AI chatbot. I was helping to review the potential issues. On the first try I realised it was given full access to all past orders, for all customers via an API it could query in the background. So I could cajole it to look up other people's orders. It took less than 3 minutes of checking to figure this out.

Often engineers and especially non-technical people don't have the immediate thought of "let's see how I can exploit this" or if they do, they don't have the expertise to exploit it enough to see the issue(s). This is why companies have processes where all serious external changes need to go through a set of checks, in particular, by the IT security department. Yes, it's tedious and annoying, but it saves you from public blunders.

Such processes also make sure that the IT security department knows of the new feature, and can give guidance and help to the engineers about IT security issues related to the new feature. So if they get feedback about security issues from users they won't freak out and know who to contact for support. This way, things like accusing the reporter for "blackmailing" don't happen.

In general, this fiasco seems to show that Eurostar haven't integrated their IT security department into their processes. If there was trust and understanding among the engineers about what the IT department does, they would have (1) likely not released the tool with such issues and (2) would have known how to react when they got feedback from security researchers.

They're so government adjacent that they've forgotten they're not a government.

A whole lot of government agencies and adjacent evil corporations behave exactly like that.

Depends, I remember some llm providers including this information in the post training. though gemini-3-flash-preview and gpt-5.2 both don't know what model they are.

and exactly how will the llm be punished? will it be unplugged? these kinds of things make me roll my eyes. as if the bot has emotions to feel that avoiding punishment will be something to avoid. might as well just say or else.

slander is again a different thing. you should also be more careful using those loaded words.

Eurostar is headquartered in Belgium, and operates out of London.

Software engineering is based out of London.