I've had this same issue with BECU (Boeing Employee's Federal Credit Union). They're a really good financial institution, but like many, they suffer from nearsightedness. They know that they're "the good guys", so they feel that it's unnecessary for them to properly authenticate themselves to you. So it's asymmetrical security and asymmetrical trust.

The worst part of this (for BECU) is that they've been warning their customers about phishing attacks from entities claiming to be BECU.