Your intuition is correct, Nostr objects are stored on relays and can be retrieved by any client with the proper private keypart. In my test runs of trying Nostr proper for a month or two, I had problems deleting objects (clients lack support for the NIP API calls was common) - and you can only delete your end of a DM, not the whole thing. (another frustration - your "chat" or "inbox" can never be cleaned or deleted of the other person's messages).

Effectively nostr objects lay around on relays until they reach some server-side expiry policy and sometimes forever if one can't figure out how to delete them. Nostr clients (web and mobile) are the wild west of good luck with which features and NIPs they support (XMPP all over again). My experience here led to dissatisfaction as well. Relays are another wild west - it's choice paralysis and a helping of good luck with that.

The real problem for IM: Nostr does not ensure all relays sync, instead a user chooses their preferred relays (some you pay for in crypto). It is entirely realistic (I experienced it) that you're on relays other people aren't and you can't share content. This is death for an IM app and just trying to use Nostr became frustrating. (not to mention it's flooded with porn and crypto shills)

Edit: a nostr client has to keep open network connections to all these relays, as objects can be stored on multiple relays and it's up to the client - not the relays - to query all relays and then de-dupe the JSON responses. There are combobouncers in use (who will de-dupe) but they're not the default and tend to be read-only (because you can't choose which relays to post to when using a combobouncer).