Remix.run Logo
adrian_b 19 hours ago

The so-called "SIMT programming model" is a term from the obfuscated jargon that NVIDIA has introduced for CUDA, where they have renamed almost all traditional terms used for decades in the computing literature.

The "SIMT programming model" is the same thing that in 1963 was called "Parallel DO" (from the name of the loop statement in FORTRAN), and later it was more frequently called "Parallel FOR", like in the C/C++ version of OpenMP. In the influential research paper on which later the programming language Occam was based, C.A.R. Hoare referred to the same thing as "an array of processes".

The "SIMT programming model" just means that you write iterative structures where the programmer guarantees that the iterations are independent (unless specified otherwise), so that their order of execution does not matter.

In CUDA it is slightly less obvious than in OpenMP that the so-called "CUDA kernel" is the body of a loop, because the header of the loop is not adjacent, but it is placed elsewhere in the file.

The essential difference between the "SIMT programming model" and writing a "for" loop in C is that the compiler is certain that the iterations are independent. For auto-vectorization, the compiler must prove that they are independent, which can be difficult.

Compiling for a VLIW CPU is in general a much more difficult problem than auto-vectorization (which implements data parallel processing with multiple cores and/or SIMD cores).

A VLIW CPU is able to execute in parallel distinct instructions, not only the same kind of operation like a SIMD CPU, but in most cases there are complex restrictions about what kind of instructions may be combined.

It can be too difficult for a compiler find a schedule for the instructions in such a way that this would allow a maximum number of instructions executed per clock cycle.

Probably the worst part is that it is unlikely to be able to keep busy all execution units without speculative execution of the instructions that are beyond conditional jumps. For these, it is pretty much impossible to guess an optimal schedule at compile-time, because it would change during execution when the same code (in a function or in a loop body) is executed again.

A CPU with dynamic instruction scheduling a.k.a. with out-of-order execution, will change the instruction schedule depending on the predicted branches, with much better chances of keeping busy the execution units.

In HPC applications, where Itanium worked best, it is possible to predict the branches at compile-time quite well, so a compiler has a chance to find an acceptable instruction schedule, unlike in more general-purpose applications, which are hard to predict.

A VLIW CPU also has speculative execution and a branch predictor, because these are needed in any pipelined CPU.

But unlike in an OoOE CPU, the speculative execution handles only complete bundles of instructions that are executed in a given clock cycle. A bundle of instructions may be executed speculatively or not, but the CPU cannot extract speculatively individual instructions from a set of bundles and combine them into a bundle that would occupy all execution units, like in an OoOE CPU.