The fix is to not let users download the credentials. In fact, ideally the web server wouldn't have access to files containing credentials, it would handle serving and caching static content and offloading requests for dynamic content to the web application's code.

Disabling auto-indexing just makes it harder to spot the issue. (To clarify, also not a bad idea in principle, just not _the_ solution.) If the file is still there and can be downloaded, that's strictly something which should not be possible in the first place.