A GoPro fits all of those requirements and has easily replaceable batteries. Now, I understand that the shape and sizes are different. But I wouldn't mind some extra mm of thickness (I already get a pretty big camera bump anyway) if that means I can replace a battery faster.

YOU would not mind, many others would.