Are they? Or do they just have superior recall? Or maybe lack test-taking anxiety? Or write or type quicker or...?

Lots of reasons a slow student can be just as smart or smarter than a fast one.

Those are all proxies for "is smarter". They have better memories, perform better under pressure, etc. Universities are meant to prepare students for the real world where these things matter.