Remix.run Logo
Remix clone Hacker News
new | show | ask | jobsGithub
▲ Muse exposes users' personal and business secrets when they share conversations(arstechnica.com)
173 points by gumby 9 hours ago | 69 comments
▲lmf4lol 2 hours ago | parent | next [-]

We are building a Personal AI Assistant platform in Europe. Oh man. It is FREAKING HARD! We have now 20 paying users and it is amazing what those people do lol. You really can't make it up. The complexity of building such a business is something I've really underestimated. Especially if you allow the harness to evolve. For our 20 users, no sandbox is still the same anymore. Everyone customized it, has crons running, built apps or automations. And then jsut to think that Meta Muse went to a million users in literally no-time. I am not surprised at all. On the other hand. Zuck has literally infinite money to spend. You'd expect better than, right?

EDIT: Because someone downstream rightfully asked if we look at user's sandboxes: Yes! IF they ask us. Our users aren't engineers and often get stuck and even the LLM can't figure it out. THEN - AND ONLY THEN - we jack into the sandbox and help them.

▲m4rtink an hour ago | parent | next [-]

So dump all my secrets, then have people running the service causually spy on me & brag about it on the Internet ? Marvelous, 10 out of 10!

▲jonathanstrange 2 hours ago | parent | prev | next [-]

Good to know how much you're "looking" closely after everything your customers are doing. The name of your company is in your profile, right?

▲lmf4lol 2 hours ago | parent [-]

Yes. We are Bluehouse. The homepage still sucks but we are working on a nice one right now without AI Slop :-)

▲yellow_lead 2 hours ago | parent [-]

Crazy to hear. So you look in user's sandboxes, does this comply with GDPR and other regulations?

▲timthelion an hour ago | parent | next [-]

Actually, for the purpose of providing the service its fine to look. They just aren't allowed to use what they see anywhere else. Just like your doctor hears about your health problems but can't sell your medical records. Whats with the nasty toxicity?

▲lmf4lol an hour ago | parent | prev | next [-]

We only look IF they ask us. Our users aren't engineers and often get stuck and even the LLM can't figure it out. Then - AND ONLY THEN - we jack into the sandbox and help them.

For instance, one customer has 15(!) cronjobs. And they all used LLM calls to do stuff. So they ran out of tokens super quick and asked us for help because they didn' tknow what to do. Its not an engineering company, the owner has a sales background. So we went in and wrote scripts for them so that the cron calls don't need to use the LLM anymore.

▲sksksk an hour ago | parent | next [-]

That’s interesting, so in that scenario, you fixed it for that customer.

How do you then ensure it’s fixed for customers in the future? Do you add instructions to agents to write scripts where possible?

▲lmf4lol an hour ago | parent [-]

Yes. Or we make a skill of it and offer it via our skills marketplace. Depends really on what the fix was. Sometimes its really a one-off issue, and sometimes it makes sense to roll the new "insights" out to everyone.

▲sksksk 39 minutes ago | parent [-]

Thanks!

Another question I have, is from a product perspective, how do you decide how much of the internals you show to users (like token usage).

I always thought that seems like (from a non technical users POV), like a implementation detail; and quite hard to map to actual usage. For example, one request might use lots of tokens, and another will barely use any; but it can be difficult for a non technical user to undertand the difference.

▲lmf4lol 18 minutes ago | parent [-]

intuition and a lot of talking with our customers :-)

▲m4rtink an hour ago | parent | prev [-]

Even just the possibility anyone other than the user can "take a look" makes a service like this unfit for purpose.

▲weird-eye-issue an hour ago | parent [-]

You do realize basically every service does this right?

▲not-kinsale-joe an hour ago | parent | prev [-]

Can IT support in EU remote connect to view a users PC? or does that violate GDPR and other regulations?

▲strofocles 2 hours ago | parent | prev [-]

literally?

▲close04 an hour ago | parent [-]

It's a topic that comes up once in a while, usually when it's less relevant to the point being made. The short story [0] is that it can mean both literally or figuratively depending on if you use it literally or figuratively.

> We define literally in two senses:

> 1) in a literal sense or manner : actually

> 2) in effect : virtually

[0] https://www.merriam-webster.com/grammar/misuse-of-literally

▲ryandrake 5 hours ago | parent | prev | next [-]

I couldn't be true. You're telling me that Meta, of all companies, would be careless with user data and privacy? Totally unbelievable!

▲llljk 5 hours ago | parent | next [-]

The ceo who infamously said this

    Zuck: Yeah so if you ever need info about anyone at Harvard
    Zuck: Just ask.
    Zuck: I have over 4,000 emails, pictures, addresses, SNS
    [Redacted Friend's Name]: What? How'd you manage that one?
    Zuck: People just submitted it.
    Zuck: I don't know why.
    Zuck: They "trust me"
    Zuck: Dumb fucks.
and suspended the instagram account of protest organizer during its peak? https://www.reuters.com/technology/meta-apologizes-after-sus...

anyone surprised?

▲nomilk 5 hours ago | parent | next [-]

To be fair, this is every tech company, ever, with extremely rare exceptions.

▲educar an hour ago | parent | next [-]

I suspect you are projecting. This is not every tech company, not even remotely close. Maybe the big Silicon Valley companies have misled you into thinking that other software companies don't exist.

▲nomilk 27 minutes ago | parent [-]

How many companies can you name whose staff certainly cannot access your data?

I thought I could name one (GitHub), but after checking, they can (e.g. if requested by law enforcement etc).

Therefore, the total I can name is 0.

▲bryanrasmussen 2 hours ago | parent | prev | next [-]

generally the guy with this attitude and willingness to just share with some buddy when asked for it is a mid term hire.

▲alex1138 4 hours ago | parent | prev | next [-]

To be "fair", Zuckerberg is a mendacious little prick with a history

Did the Google guys ever get caught saying that? Or using their power over the company to unsend messages, oh I guess you can't do that with email, but with Messenger you can...

▲zaptheimpaler 4 hours ago | parent | prev [-]

It's not, there are levels of scumbaggery and Meta is one of the absolute worst. Rigging elections all over the world, privacy violations everywhere, actively scheming to get kids addicted to slop media, privacy destroying recording glasses, the list goes on.

▲close04 an hour ago | parent | prev [-]

> anyone surprised?

A lot of people actually, even techies. I commented something similar on a different tech website (Ars Technica) highlighting the trustworthiness of the company behind the tool, and that by now people should not be surprised the face eating leopard eats faces. I was met with a lot of implicit defense of Meta and redirecting the attention to "whatabout" Anthropic, OpenAI, etc. (much like sibling reply here makes it an "every tech company" problem). The general agreement in that forum was that the company is not the problem, it's just what AI tooling does.

So either the internet is full of shills, or people are genuinely surprised because they chose ignorance for decades. Or maybe both can be true.

▲tonyhart7 5 hours ago | parent | prev [-]

the worse part its by design for targeted ads

its not incompetent, its designed that way

▲JumpCrisscross 3 hours ago | parent | prev | next [-]

> But a central design decision for constructing URLs has made the links easy to discover. As a result, researchers from security firm Claroc said they were able to enumerate 6,800 links Muse users created for sharing conversations with others.

To be clear, did the users intended for these conversations to be shared with someone? Meta just created a guessable capability URL?

Or were all Muse users' conversations by default public, just behind an obscure URL?

▲skohan 7 minutes ago | parent | next [-]

I remember back when Facebook was still .edu email addresses only (2004 or 2005 maybe), and my university got access, we discovered you could just grab the user id of another user from their public profile, and paste it in place of your own user id on your own profile, and get access to their profile as if it was your own.

▲cloudbonsai 2 hours ago | parent | prev | next [-]

It seems that Meta assigns a unique hostname to each shared conversation. For example:

> fiscal-review-b4a5ff.cf.metaaiusercontent.com

So when someone opens the link, the DNS resolution exposes the unique hostname. The original bug reporter found dozens of these URLs in passive DNS logs.

▲venzaspa 40 minutes ago | parent [-]

That is remarkably amateur. I'm not a software engineer by trade nor have worked at a billion dollar company, but knew that to be a bad design within seconds.

▲fxwin 2 hours ago | parent | prev [-]

Sounds like it's the former: URLs are guessable/easily enumerable but their contents protected by auth unless they're "published" using the share feature

▲omoikane 5 hours ago | parent | prev | next [-]

Lesson here is that the subdomain names can be queried due to passive DNS records, and putting stuff that is not supposed to be guessable in subdomain names is a bad idea.

▲kristopolous 40 minutes ago | parent | prev | next [-]

This is news? Didn't we all just presume that was what was happening? I mean they're a data mining company.

▲hamburglar an hour ago | parent | prev | next [-]

I am baffled by all these people giving muse their personal info. I tried muse out. Didn’t install anything locally and didn’t give it any personal info beyond a fresh meta identity. I just poked at the free vm for a while. There’s no way in hell I’m giving it access to anything sensitive.

▲adishiktensward 13 minutes ago | parent | prev | next [-]

Zuck never beating the allegations

▲motbus3 an hour ago | parent | prev | next [-]

My honest view is that it is taking too long for them to be made legally responsible for these problems.

There is no point on saying "there is no guarantee software" as it is supposed to be "maybe it does not work" and not "it might make your life terrible"

▲piterrro 3 hours ago | parent | prev | next [-]

I dont get why they bother with DNS and subdomains when they could generate a unique url. This would eliminate the possibility of easy reverse dns lookup.

▲ulfw 3 hours ago | parent | prev | next [-]

For the life of me I will never understand why people are so eager to use these things

▲sksksk 2 hours ago | parent | next [-]

I run an personal AI agent locally (hermes), and use a combination of local models and whatever model I can get for a reasonable price (usually deepseek).

After using it for a couple months, I'm converted. They're so good, the more you use it, the more useful it is; you internalise more of what it could be used for. It runs so many boring parts of my life, leaving me with more time to spend with my family.

I liken it to release of the first iPhone; it didn't invent any new technology, but packaged different bits of tech together in a way that was consumer friendly. I think we'll look back at OpenClaw/Hermes/Muse/Instinct in the same way.

I do worry about using purely cloud based offerings like Muse, I reckon there is a market for someone to package an consumer AI agent that lives on your home network.

▲skohan 3 minutes ago | parent | next [-]

I think the promise of personal AI agents is incredible, and they can already be super useful. But we haven't even begun to figure out the safety side for a mass audience.

Even as an experienced developer, sandboxing an agent is not trivial. The idea of my aunt, who has her passwords on a post-it-note on her monitor and forwards me scam emails using these things is terrifying.

▲ad_fontes 2 hours ago | parent | prev | next [-]

Whenever this comes up, all I ever see are hand-wavy responses like this. "It does everything", "it changes your life." But how exactly? Because I can only see this being true for the terminally online.

▲sksksk 2 hours ago | parent [-]

That's fair, I'll give a couple of example of things that I've used it for...

I built an app (in codex), that goes through my emails, pulls out any receipts, and adds the purchases to a sqlite database. I do all my grocery shopping online (through my supermarket), so our weekly shop is stored in the database.

Hermes uses that to predict what items will need restocking (based on how often I buy it, and when I last bought it).

Every week, it logs into the supermarket website on my behalf, books a delivery slot, and adds all the items that need restocking to the shop.

Throughout the week, if I run out of something, I take a photo of the barcode and send it to hermes, it adds it to the shop for me.

I built a similar app for travel; if I have any upcoming travel, hermes will send me updates (delays, platform changes etc...). I do a lot of train travel for work, if my train is delayed, it will automatically apply for the compensation on my behalf.

It reads my emails, and pulls any actionable items out, and adds it to an Asana list. I can either do the tasks myself, or assign it to Hermes and it will do it for me. As an example, I had an task to book swimming lessons for my son, hermes monitors the swimming lesson availability and automatically booked it when a suitable one becomes available.

Shopping is a good use case, I have a clock that needed a new battery, it wasn't a standard battery, and required a screwdriver to open the compartment. I took a photo of the clock, sent it to hermes, and it found out what battery took and ordered it of Amazon for me.

I wanted to buy some smart thermostats for our radiators, but wanted to be compatiable with our current smart heating system. Hermes is connected to my home assistant, so it figured out what heating system we have and found a compatiable set of eBay (this is slightly more involved than it sounds, as there is a new version of the heating system with a new set of completely incompatiable accesories).

I don't think "it does everything" or that it has "changed my life", but all of those examples are chores, things that I don't particularly enjoy doing; and if something can take it off my plate, then it's useful for me.

▲convict 16 minutes ago | parent | next [-]

When i run out of something to buy from the store, i type it in the notes app on my phone

▲sksksk 8 minutes ago | parent [-]

When I need to change the channel, I just get up and walk over to the television.

▲gherkinnn an hour ago | parent | prev [-]

Your thermostat and non-standard battery are examples of LLMs being helpful.

As for the rest, that is a lot of tokens burned to keep a fridge stocked.

▲sksksk an hour ago | parent [-]

It doesn't burn that many tokens, as most of the code is deterministic.

Receipt emails are in a predictable format, so codex built scripts to parse the emails. If there is a receipt email it doesn't recognise, it will use an LLM to build a parser for that kind of email once.

The agent also "self improves", so after asking once to get items to restock, it wrote a skill with the query it used in, so it doesn't burn tokens figuring it out again.

I was impressed that after it logged into my supermarket account, it inspected the network traffic and discovered a GraphQL API, and used that (instead of screen scraping); much more economic.

I've averaged 2.6 cents a day in token costs over the last month.

▲ctolsen 2 hours ago | parent | prev | next [-]

I am the opposite. I work with LLMs all the time but don’t use them much at all personally. Not for lack of trying, I’ve tried several times with different frameworks and I just can’t find a use case that is remotely worth it.

▲sreejithr 2 hours ago | parent | prev | next [-]

That's pretty much what Openpod.live is doing

▲ulfw 2 hours ago | parent | prev [-]

Oh I convinced now!

I'll use personal agents for my laundry now. Thanks mate for the samples given

▲sksksk 2 hours ago | parent [-]

no need to be rude, see my reply to a sibling comment for examples

▲educar an hour ago | parent | prev | next [-]

You cannot understand it because you are possibly thinking of it from a utility perspective? People just do things because it's cool or keep themselves entertained. Mostly to keep themselves entertained.

▲sourcecodeplz 2 hours ago | parent | prev | next [-]

i understand the adoption for these things is off the charts.

usage-time growing fast, with tasks and cron jobs.

in the last spacex earnings report (few hours ago), Elon said he thinks their grok-bot will next year make up most of their AI inference usage across their datacenters.

basically he tried to imply these bots/personal-agents get lots of usage once users start customizing them. and it is bringing in a lot of money in API usage.

▲prasadjoglekar 3 hours ago | parent | prev [-]

For the same reason people buy shiny new cars, bikes, keyboards....you name it. The novelty and sometimes bragging rights in the circles they inhabit.

▲ulfw 2 hours ago | parent [-]

Yea I think that's about as far as the usefulness goes for personal use

▲eviks 4 hours ago | parent | prev | next [-]

> Muse, which Meta CEO Mark Zuckerberg said last month was “built from the ground up for privacy and security,”

> reported the findings through Meta’s bug bounty program on October 8 and also tried reaching people at Meta on LinkedIn, but got no reply either way.

Checks out

> The ease in discovering share links further justifies asking if Muse is worthy of the access it requires.

That's all you could justify?

▲giorgiogamba 2 hours ago | parent | prev | next [-]

There was a feature requirement for this

▲crorella 2 hours ago | parent | prev | next [-]

oops! This is totally unexpected coming from Meta.

▲johnnienaked an hour ago | parent | prev | next [-]

Comedy of errors

▲postatic 4 hours ago | parent | prev | next [-]

This is a feature no?

▲diegoborgh 3 hours ago | parent | prev | next [-]

Shocking!

▲vektormemory 3 hours ago | parent | prev | next [-]

If only there was a company that didn't use ads, protected people's privacy, didn't sell their data, and had local data sovereignty?

▲viccis 4 hours ago | parent | prev | next [-]

Muse is the most obviously doomed claw-style bot because you already need to segment those things from ANY personal info, and Meta rolls into the room as the journal that will tell everyone what you write in it and says "trust me"

Part of a huge problem with FAANG companies. You can't be Meta and just ratfuck your users for decades and then ask them to trust you, just like you can't be Amazon and turn your storefront into Temu-lite and expect people to want to order products sight unseen with your Echo shit.

▲Razengan 5 hours ago | parent | prev [-]

I mean, not to victim blame, but anyone willingly using any Facebook product at this point should know what they're signing up for.

▲touristtam 3 hours ago | parent | next [-]

Sure but for most people they will not understand the consequences until it is hammered by constant notifications. And those will not happened until such law are mandating for the users to be notification left, right and center. After all Facebook is one of the original promoter of dark patterns in UX/UI.

▲wayvey 4 hours ago | parent | prev | next [-]

Unfortunately people are still not that smart or aware of the risks. They should be, but that's not the reality yet.

▲pferde 3 hours ago | parent | next [-]

It's worse. Even if you talk to them, explain in detail why and how Meta and their products are evil, they will just nod, frown, go "yeah, that's serious, someone should do something", and then they will turn around, check their Whatsapp messages and post a story on Instagram about their lunch.

▲Razengan 2 hours ago | parent | prev [-]

Can some post ads on Facebook telling people to get off Facebook?

▲gherkinnn an hour ago | parent [-]

Watch those ads get pulled. And then see actual scams being promoted.

▲alex1138 4 hours ago | parent | prev [-]

Network effects matter and no you can't just 'email/call them'. Not everyone automatically has those. You can get it, but something like FB, loose connections, regaining contact (you literally don't have their email/phone numbers and don't tell me 'oh but you lost touch for a reason'), is a nice thing if its CEO wasn't who he is