Steel or aluminum cans might be better? At least they rust away and don't float.

I mean you could have a screw cap on a can too if that is important. Dunno why 33cl/50cl and whatever the oz. sizes are, are metal and plastic respectivly.

Using one time cannisters for water feels fundamentally unsubstainable.

I think “one time” is the key word here. Multiple use glass and metal containers were just fine before plastic; they can be the norm again.