Pointers are sort of needed for high performance memory management for HPC targets for existing design patterns, maybe we can think of better solutions down the line but it's hard for me to say anything I just use/abuse CUDA pointers as well.

Julia has pretty good design heritage for how to deal with this sort of thing. you build the right abstractions and everything works (the main key is making sure the compiler elides bounds checks)