Technically you're right, a gpu is just a bunch of ALU and FPU in a way, we just defer computation to a massive array of them so the cpu can do other things, what you do on these gpu cores is up to you.

That said, historically, software rendering meant non-accelerated passes that could be carried out by a cpu at decent enough FPS.

ps: some could also say that a FPU is not software rendering since early x86 didn't have it in the cpu.