It does not, actually. Signal only gets the contact number if you click on the button.

In my case I allowed Signal to access my contacts and these options only appear for people who use it, don't know what happens if you deny the access.