At risk of being called out for my ignorance (I am still new to GPU development and have only limited experience with CUDA), it seems to come down to how appropriate the execution model is to the work e.g. SIMT vs SIMD here.