For the most sensitive fields (names, addresses, phone numbers) it's quite simple. For names, you get a list of the (say) 1,000 most common names, and pick randomly from the list. For phone numbers, you generate random numbers with valid formatting (not all area codes are valid, etc.). For addresses, you pick randomly from a database of real addresses. Etc. No LLM-style generation needed.