That’s the first part, but will the charts work if I override the image name with a non-Bitnami one (e.g. docker.io/library/redis for redis)? Or do they bake in special stuff in their images that their charts rely on?

You need the images that go with the charts. They have their own config system, which usually involves elaborate shell scripts in the images that receives parameters from the chart.

It is hard to tell if it will work or not. Just need to compare the image and test the override out.