So a simplex is a hypertriangle? The minimal shape that can be made with planes in any dimension?

Understanding that 10 years ago might have made it easier for me to understand collision detection algorithms for game physics

Exactly! It's n+1 points in n dimensions (when finite). Another way to think about it (the way that I know because it extends into general Banach spaces and not just n dimensional spaces) is that each point inside is the unique weighted average of the extreme points (corners). So in 2d, if you have a square you can get that middle point by averaging all the corners, or averaging two opposing corners, so it's not a simplex.