Most have internal batteries and are still "on" to a certain extent unless the battery is completely discharged.

Use one of the few phones with hardware kill switches or removable batteries.

If you cover your phone with an antielectrostatic bag it can't communicate; that is a Faraday cage.

Since people around you will think you are also wearing a tinfoil hard, you had better stick to the phones with hardware switches as sibling comment mentions