These days banking is one of the things for which a phone is required for. It is used as the primary banking device for most people, and for the rest it is required for two factor authentication when logging in on a PC or to verify online transactions.

Maybe some bank would allow you to use some third party two factor authentication device to log in sometimes, but most (if not all) would require you to use their "app".