I have a similar dim memory, but (at least according to this article) invertebrate bilaterians don't have that swap at all, so it can't be too strong a constraint.