I agree that the apologies tend to make one a little uncomfortable. This is because people do not simply say, "I am sorry I did not have time to write sooner. Here is my response", but instead say, "I would have written but my child was sick, etc.," so you feel the need to respond to that, and feel bad for having bothered them.

Almost nobody writes, "I am sorry I was scrolling Twitter and Hacker News while ignoring my e-mail. Fortunately, I have stopped and now can respond!"

I write no more than either, "Apologies for the delayed response," or "Apologies for the delayed response, I've been out of the office unexpectedly." Very business-like. Tone neutral. Easy to digest and move on.