IIRC the largest speed difference was caused by Active Desktop. Windows 98 burned a "lot" of memory and clock cycles in order to view dynamic content inside Explorer.

Back in the day there used to be custom builds of Windows 98 that had Internet Explorer completely stripped out. Those were much closer to Win95's performance.