At that point just use whitelist-based approach. I already used gitignore like that. It also easier to avoid committing temp file and secret through whitelist because those file are never in gitignore in the first place.