As the sibling comment already said "kai" (pronounced ke like in keg) just means "and". So it literally means 4 and 10 sides in greek. But I have often seen it written as τετραδεκάεδρο (tetradecahedron) in greek as well, so without the kai part. I'm not sure why it is 4 and 10 instead of 14 though. It would be more natural in greek that way (δεκατετράεδρο - decatetrahedron). Maybe it is for putting the distinctive part (4) first, or maybe it sounded more "poetic" like that to someone and then it stuck.

I think the three-and-ten, four-and-ten way of expressing numbers is primarily an ancient Greek thing. The modern numbers are expressed differently (δεκατέσσερα / dekatessara for fourteen, for example). In a lot of older European languages 11 and 12 behaved irregularly. You could argue that they do in English too (we don't have oneteen and twoteen).

I haven't read of any particular reason for this, but I'd posit that numbers up to twelve were more commonly used in everyday life, so shorter, irregular forms were easier to use and remember. Much like many of the irregular verb forms in spoken language happened because they were so commonly used.

The ancient Greek system also gave us triskaidekaphobia - the fear of the number 13.

Four and twenty blackbirds, baked in a pie.

https://en.wikipedia.org/wiki/Sing_a_Song_of_Sixpence

Nobody would be so weird as to put the "four" first when pronouncing "14".

:-)