| ▲ | eterm 7 hours ago |
| This seems surprising to me. I think live presence is an essential part of real time communication, but also something that should be well understood how to scale? |
|
| ▲ | jareklupinski 7 hours ago | parent | next [-] |
| i could see how after 10k users, the logarithmically? growing time/resource pressure of checking more users against every list, without properly distributing the data / sharding, starts to creep in |
| |
| ▲ | eterm 6 hours ago | parent [-] | | Perhaps I'm not understanding the Matrix model, is there no central broker that is holding user contact relations and fanning out and directing statuses / messages? Because, otherwise, even if you went crazy and did the most naive implementation: Storing for every User
User -> List <User Contacts>
And the reverse:
User -> List<Users who have you as contact>And even then if you go full quadratic and have literally everyone have everyone else as a contact, and store and update the online status for each pair, and do that for each direction, you still shouldn't have scaling issues at 50k users, as that just isn't that much data? By 500k Users you might be starting to feel it with a naive approach, but 50k? It shouldn't be a bottleneck unless status updates are causing cascade effects into other systems. And that's just an analysis of the most naive design I could think of! | | |
| ▲ | everforward 3 hours ago | parent | next [-] | | I don’t believe there’s a central coordinator (it wouldn’t be distributed then). I think you’re missing that rooms are attached to servers, and you can join rooms that aren’t on your server. If I go join a room with 50k people on another server, those 50k people need to pull an online status from my server and my server needs to pull all of theirs. Multiply that by all the rooms 50k people have joined and you get a _lot_ of messages bouncing around. It’s sort of hard to optimize without either a central and implicitly trusted cache, or giving up on trust entirely. Central cache is sort of obvious. You could go peer to peer but then you have to deal with “what if somebody modified their server to issue forged updates”. You could maybe sign them but I’m not sure how anyone would validate that without getting a pubkey from your server and now we’re back to scaling problems and have a new “storing a million pubkeys” storage issue. | |
| ▲ | JadedBlueEyes 3 hours ago | parent | prev | next [-] | | Based on live metrics from my current server, there are three order of magnitudes more presence updates processed than any other form of event (typing, message, encryption, etc). This is after the majority of people have turned it off. It's a huge amount of work relative to everything else in the protocol. | |
| ▲ | jareklupinski 6 hours ago | parent | prev [-] | | based on experience with another tool i saw, each lookup itself must be authorized by checking membership in at least two other lists on three other servers :P |
|
|
|
| ▲ | techpression 6 hours ago | parent | prev | next [-] |
| Indeed, this seems like a really low number for when to run into issues, unless they're running it on a shared VPS and/or it's just not very well built. |
|
| ▲ | btreecat 6 hours ago | parent | prev | next [-] |
| It's an asynchronous protocol, the need for someone to be visibly online is your own. Send me a message, I'll get back to it when I'm online. |
| |
| ▲ | afavour 5 hours ago | parent | next [-] | | > Send me a message, I'll get back to it when I'm online. Which isn't acceptable in a lot of workplaces. I'd say these chat systems have an expectation of only semi-asynchronicity. If you're truly async, you use e-mail. If you send a chat message you're expecting an at least somewhat timely response, an online indicator helps let you know how soon you might get it. | |
| ▲ | 5 hours ago | parent | prev [-] | | [deleted] |
|
|
| ▲ | esseph 6 hours ago | parent | prev [-] |
| I mean, if the user names and the messages themselves are cached and displayed, why can't user location be done the same way every ~5-10min or so, or ~1m with an active direct user-to-user (with approval) lookup? it's an exchange of two ~6-10 digit grids and an elevation, I'll give allowance for a timestamp. |