If they're stopping production of the penny because it costs more to mint each one than each is worth, that's a stupid reason. Why does everything have to be about making money? Can't we just have useful things without worrying about how profitable they are?

We do lots of things that are unprofitable, and we should probably do more than we do. But they should be things that bring non-monetary value into the world. Pennies do not bring non-monetary value into the world.