You're not alone and I'll even say this is evolutionary.

Men haven't evolved to be overly attached to babies or small kids like mothers did - that'd have been a weakness in the survival race. Men skills were completely different, usually related to being away from the kids most of the time.

Translate to modern day, that is men not really wanting to have kids (sometimes just going along with the wife's biological clock), and pretty much "feeling nothing" when the kids are born. There's a million documented examples of that.

However, once kids leave the "little puppy" phase and grow beyond the basic needs care that mothers provide, it's when the fathers start to really relate to them.

If your older kid is maybe 5-6 you're about to start that new phase. That's what happened to me, and I find (maybe not so extreme) examples of that on almost every guy I talk to.

Give it a second chance, maybe having kids is just not for you. But don't assume it's already the case, you may miss out whole new world.