The libraries, yes, but the code you write is portable (at least until you get into squeezing the last few percent and switch to Triton / Helion in case of GPU, and even those are decently portable).

There's also Halide, where you write the algo but the framework gets you the scheduling and SIMD.