How are we sure that it is really end-to-end encrypted?

Moxie Marlinspike (founder of Signal) [0]implemented the same E2EE algorithm as Signal (Signal Protocol) into WhatsApp, but that was 10 years ago, so who knows if things have changed since then.

[0] https://en.wikipedia.org/wiki/Moxie_Marlinspike

Practically speaking, it isn't secure; no closed app can be. It receives regular compulsory updates (old versions refuse to work) and there's nothing at all stopping Zuck from sneaking in backdoors targeted at you personally.