It's got an effect but agreed it's not the biggest effect given what else is going on. I think time might be the bigger factor here when simply discussing biology. If you have kids every ~3 years and don't start until you're 35, you have maybe 1.75 years of kids left in you before it starts getting tenuous. (ie, before the woman is over 40) That same math works out differently if your first kid is at 20.

None of this touches on industrialization and higher education, which seem to be the more universal effects, even if one of their bigger effects is merely to delay motherhood.