CardDAV is an extension of vCard. vCard can handle an arbitrary amount of contacts. But who knows, you definitely have more experience in using address book protocols than me.

I don't claim to know much. But with multiple contact lists and many groups in each list, CardDAV is a must.