True, but imagine this use case:

A messenger file with hardcoded settings and a hardcoded PGP key, stored on a USB stick.

You send a message.

Then you physically destroy the USB stick.

The client, the key, and the configuration are gone.

At some point the joke starts looking suspiciously like a dead-drop communication protocol.

How do you like that, FBI?