For how much Infuse gets praised I was disappointed in the UI/UX. Playback is great, but the browsing UI is terrible and offline downloads are really buggy (Just like with Plex for a very long time)

Yep, total agreement here. But they really nail the "it just works" of playback, so all things considered I stick with it.