I got sick of it, unplugged it, and left it for a few months, and ... apparently it fixed itself? Now it's working fine, as fine as any HomePod does.