the multi-vector duplication thing makes sense, copying every attribute once per vector explodes quickly. what's the new primary index?

An automatically generated internal ID: (segment ID, doc ID). The user-provided primary key (the field called `id` in the document) turns into a secondary index at the storage layer.