I looked into my policy once, and it said that if I let someone else drive my car, the coverage from my policy would revert to the state minimum, around $15k.

If the driver has insurance, that would probably get tapped first, but if they don't have enough it could end up hitting your policy.