Why not use a text embedder for the unstructured data and concatenate with the structured data?

For example you could freeze most of the layers of the embedder but let the final ones learn. Then you wouldn’t need to do either feature or prompt engineering?