This is on the engineers. Asking for time to refactor is like a painter asking for time to prime. It’s part of the job so just do it. When brought out in discussion, it sounds as if it is optional and its value dubious. That’s where the feeling of “wasting time” comes from.

Additionally, a lot of refactoring is about predicting the future rather than adapting the code to present expectations while keeping things easy to change. The only metric that your future self cares about is time to feedback. Refactor to adapt to present expectations while keeping the code easy to change in the future, and don’t ask for permission to do this- it is your job.