| ▲ | BrissyCoder 7 hours ago |
| This reads like internet fiction to me. Very vague and short. |
|
| ▲ | yawniek 7 hours ago | parent | next [-] |
| fwiw i know tobias and its very very unlikely he made this up.
my guess its intentionally vague to not leak any information about the culprit which i guess is fair. |
| |
| ▲ | rausr 7 hours ago | parent | next [-] | | heh, I know that username. I came to the same conclusion.
(I hope all is well with you, Yannick) | |
| ▲ | BrissyCoder 7 hours ago | parent | prev | next [-] | | Okay. If it's real I apologize. But in any case it's so lacking in detail and so brief as to make it so uninteresting that it might as well be fake. > Somebody "vibecodes" medical app/system. The app was insecure. Personal info leaked. Okay cool. | | |
| ▲ | kuboble 7 hours ago | parent [-] | | Is really weird to me that this is your reception. It's a rarely updated personal blog, not a daily tabloid story. | | |
| ▲ | BrissyCoder 6 hours ago | parent [-] | | It's pure bs. If you read that blog post and think "this definitely happened", let alone "wow - this is interesting" then I have a monorail to sell you. > Technical Background > The entire application was a single HTML file with all JavaScript, CSS, and structure written inline. The backend was a managed database service with zero access control configured, no row-level security, nothing. All "access control" logic lived in the JavaScript on the client side, meaning the data was literally one curl command away from anyone who looked. > All audio recordings were sent directly to external AI APIs for transcription and summarization. > There was more, but this is already enough to get the idea. Hmmmm... interesting, now that I have the "Technical Background" I for sure know that this medical app was 100% vibe coded by a Medical Practice in the Real World and exists! (TM) | | |
|
| |
| ▲ | spacebacon 7 hours ago | parent | prev [-] | | It’s unlikely any LLM tasked with a prompt involving medical records did not automatically address separation of concerns. The type of data involved is worst case scenario. One JS file is also worst case scenario. This is why it may feel manufactured. If it is true, they truly deserve to be put on blast. | | |
| ▲ | moooo99 7 hours ago | parent [-] | | I can 100% imagine prompts that would even feel natural that would never hint at any medical background of the data being processed. Could be as simple as using customer instead of patient. |
|
|
|
| ▲ | abrookewood 7 hours ago | parent | prev | next [-] |
| Given the subject matter, it would be highly unethical to reveal the name of the company before verifying it was indeed fixed. I'd be wary of getting sued. |
| |
| ▲ | mijoharas 5 hours ago | parent | next [-] | | The first time I stumbled onto a big security vulnerability (exposed stripe/aws/play store keys. I was poking around an API a web app was using, and instead of hitting /api/v1, if you just hit /api it served them. I wasn't trying to do anything malicious), the very first thing I did was contacted a security researcher friend to ask about covering my ass while performing responsible disclosure. You hear too much about people being persecuted for trying to point out security vulnerabilities. (Guess they haven't heard about "don't shoot the messenger"). (It turned out fine after finally managing to speak with someone. Had to ring up customer service and say "look, here are the last digits of your stripe private key. Please speak with an engineer". Figuring out how to talk with someone was the difficult thing) | |
| ▲ | croemer 6 hours ago | parent | prev [-] | | Company should just take down the whole thing. One vuln might be fixed but how many others might be there. |
|
|
| ▲ | samuel 7 hours ago | parent | prev | next [-] |
| I don't think it's the case, but it would be very funny that this would end being AI generated clickbait. |
|
| ▲ | sixhobbits 7 hours ago | parent | prev | next [-] |
| yeah keeping it vague makes sense to protect the place if it's still online but the whole thing doesn't really make sense? The timelines mentioned are weird - he spoke to them before they built it? Or after? It's not that clear, he mentions they mentioned watching a video. > The entire application was a single HTML file with all JavaScript, CSS, and structure written inline. This is not my experience of how agents tend to build at all. I often _ask_ them to do that, but their tendency is to use a lot of files and structure > They even added a feature to record conversations during appointments So they have the front-desk laptop in the doctor's room? Or they were recording conversations anyway and now they for feed them into the system afterwards? > All "access control" logic lived in the JavaScript on the client side, meaning the data was literally one curl command away from anyone who looked. Also definitely not the normal way an agent would build something - security flaws yes, but this sounds more like someone who just learnt coding or the most upvoted post of all time on r/programmerhorror, not really AI. Overall I'm skeptical of the claims made in this article until I see stronger evidence (not that I'm supporting using slop for a medical system in general). |
| |
| ▲ | mewpmewp2 7 hours ago | parent | next [-] | | I don't know what to make of the article. First I thought it seems like a made up LinkedIn story, it seems too crazy while talking about it in such a casual manner. Ultimately I don't know, maybe it was vague for a specific reason. I guess one thing I'd find odd is that whoever developed it, that they didn't run and get stuck with CORS issues, if everything was done client side to those services and that they managed to get API keys, subscription stuff everywhere while still making mistakes like this. And no mention of leaked api keys and creds which UI side there must have been, right? > Everything that could go wrong, did go wrong. Then this claim seems a bit too much, since what could have gone more wrong is malicious actors discovering it, right? Did they? Maybe I have trouble believing that a medical professional could be that careless and naive in such a way, but anything could happen. I guess another thought is... If they built it why would they share the URL to the author? Was author like "Ooh cool, let me check that out", and they just gave the url without auth? Because if it worked as it was supposed to it should have just shown a login screen right? That's the weirdest part to me, I suppose. | |
| ▲ | fzzzy 7 hours ago | parent | prev | next [-] | | The single file thing makes perfect sense if it was built as an artifact in one of the big provider's webui. | |
| ▲ | furyofantares 6 hours ago | parent | prev | next [-] | | > The timelines mentioned are weird - he spoke to them before they built it? Or after? It's not that clear, he mentions they mentioned watching a video. I took that all to mean she had explained the history of it to the author, but it had already been written and deployed. It is worded a little weird. It's also translated from german, I don't know if that is a factor or not. | |
| ▲ | lesostep 5 hours ago | parent | prev | next [-] | | I could bet 5 dollars that they used chat and not an agent, and that's also the reason why it's a single html file. Copypasted and than dropped into hosting folder, sweet web 1.0 style | |
| ▲ | BrissyCoder 7 hours ago | parent | prev | next [-] | | > The timelines mentioned are weird - he spoke to them before they built it? Or after? It's not that clear, he mentions they mentioned watching a video. Yeah although I didn't comment I found this weird as well. Chronology was vague and ill-defined. He went to a doctors office and the receptionist mentioned vibe coding their patient records system unprompted? > A few days later, I started poking around the application. What!? How... was there even a web-facing component to this system? Did the medical practice grant you access for some reason? Yeah I'm back to calling bullshit. What a load of crap. Whole post probably written by an LLM. | |
| ▲ | viraptor 5 hours ago | parent | prev | next [-] | | Having experience working with medical software, I call BS on this article as presented, unless it was some minimal support app. When you deal with patient records, there's so much of local law, communication, billing rules and other things baked in that you CANNOT vibe code an app to handle even 1% of that. Your staff would rebel and your records would completely fall apart. Even basic things like appointment bookings have a HISTORY and it's a full blown room scheduling system that multiple people with different roles have to deal with (reception and providers). It takes serious time to even reverse engineer the database of existing apps, and you first have to know how to access the database itself. Then you'll see many magic IDs and will have to reverse engineer what they mean. (yes, LLMs are good at reverse engineering too, but you need some reference data and you can't easily automate that) I have decompiled database updaters to get the root password for the local SQL Server instance with extremely restricted access rules. (can't tell you which one...) I have also written many applications auto-clicking through medical apps, because there's no other way to achieve some batch changes in reasonable time. I have a lot of collateral knowledge in this area. Now for the "unless it was some minimal support app" - you'll see lots of them and they existed before LLMs as well. They're definitely not protecting patient data as much as other systems. If the story is true in any way, it's probably this kind of helper that solves one specific usecase that other systems cannot. For example I'm working on an app which handles some large vaccination events and runs on a side of the main clinic management application. But accidentally putting that online, accessible to everyone, and having actual patient data imported would be hard-to-impossible to achieve for a non-dev. For the recording and transcription, there are many companies doing that at the moment and it would be so much easier to go with any of them. They're really good quality these days. | |
| ▲ | suddenlybananas 7 hours ago | parent | prev [-] | | I don't think you read the article very carefully, the timeline is that he met a person, and that person told him that they made vibe-coded an app after having seen a video. He then investigated the app. | | |
| ▲ | BrissyCoder 7 hours ago | parent [-] | | Yeah because every medical practice I go to, I'm always able to investigate all of their systems. | | |
| ▲ | kuboble 6 hours ago | parent | next [-] | | I live in Switzerland. Sounds pretty normal. There are plenty of small medical practices with 1-2 doctors and a front desk. On my last visit i actually casually discussed their IT system with a doctor. | | |
| ▲ | BrissyCoder 6 hours ago | parent [-] | | > On my last visit i actually casually discussed their IT system with a doctor. Oh right, cool. Did it have a public-facing web-portal that you were able to "investigate" and that "Thirty minutes in, I had full read and write access to all patient data". The level of credulity in these comments is immense. | | |
| ▲ | kuboble 6 hours ago | parent [-] | | No, they were complaining about using expensive, overly complicated third-party system that they need like only basic features like keeping text records about visits, and prescriptions and sending invoices to health insurers.
And in some practices you get direct access to your data as a patient. I mean the story might be fake obviously, but is definitely plausible. | | |
| ▲ | BrissyCoder 5 hours ago | parent [-] | | Yeah sure, as a matter of rule, every time I visit any health provider I am always discussing with the medical receptionist: the software they use, the challenges the business as a whole faces, the tensions between insurers and third parties. Things that absolutely 100% happen everytime I - a tech guy - experiences when I go to the doctor/phyiso-therapist etc... etc... These are discussions that are happening. | | |
| ▲ | kuboble 5 hours ago | parent [-] | | Ok. So can I conclude that your point boils down to: "Your claimed experience is different than my experience so you are lying"? | | |
| ▲ | BrissyCoder 4 hours ago | parent [-] | | No I'm just straight up saying this post is fake. Don't try to bring social justice warrior talk onto a tech forum please. | | |
| ▲ | kuboble 4 hours ago | parent [-] | | What kind of nonsense are you talking about? What social justice?
What kind of argument is that? you also claim that I am lying. Are you willing to put money to be proven wrong? that it's normal to have a tech discussion with your doctor in Switzerland? | | |
| ▲ | BrissyCoder 3 hours ago | parent [-] | | Yes in good faith I will one hundred percent no-lie wire you 100AUD if you can prove beyond doubt that you had a discussion like that with a Swiss health provider. Not one that you contrive now after the fact - one that happened before this wager. Spoiler: you didn't mate and you are full of shit. |
|
|
|
|
|
|
| |
| ▲ | EliRivers 6 hours ago | parent | prev [-] | | Isn't that the whole point? That one should NOT be able to investigate all of their systems? |
|
|
|
|
| ▲ | drkiz75 7 hours ago | parent | prev | next [-] |
| Agreed. It’s right there at plausibly deniable just short of falsifying facts you could look up. |
|
| ▲ | watwut 7 hours ago | parent | prev | next [-] |
| Short writing is just a good writing. Also, it was not vague, it just omitted identifying information. |
|
| ▲ | rubzah 7 hours ago | parent | prev | next [-] |
| I assure you that these kinds of things are happening right now. |
|
| ▲ | camillomiller 6 hours ago | parent | prev | next [-] |
| It’s Germany, you want to be as generic as you can because libel, privacy and similar laws are pretty strong here |
|
| ▲ | no_shadowban_9 6 hours ago | parent | prev | next [-] |
| [dead] |
|
| ▲ | spacebacon 7 hours ago | parent | prev [-] |
| [flagged] |