Huh, I have matter devices working here and IPv6 is off on my router and DHCP. And on home assistant too which does the matter router. Does it use link local or something?