I actually strongly recommend Mammoth Caves National Park. They also have nice trails on the outside (within the park).

The caves are grand. But just keep in mind: You will not see stalactites/stalagmites there. For that, there's a cave tour right outside of Mammoth Caves (I forgot the name) - it has plenty of them.