Is infinite scroll really the problem or is it really the whole malicious toolbox and intention of "maximizing engagement"?

I agree, I don't think an "infinite refresh" like if YouTube had a limited homepage and changed on each refresh, would be much better. But infinite scroll is likely the most addictive.