I agree with your first part but your edit is a logic fallacy. I don't need to be able to hack something to say that it is insecure.

For example: I don't know how to exploit SAML but I know it is a terrible standard dur to making all of the XML parser an attack surface. I am not a security researcher so I dont know how to find exploits in XML parsers but I know having a huge attack surface is bad.