| ▲ | Tiberium 6 days ago | |
It'd be curious to see how those AI generated kernels compare to kernels generated by https://github.com/tinygrad/tinygrad | ||
| ▲ | xiphias2 6 days ago | parent [-] | |
As they wrote most of the wins are because of fusion and TimyGrad started to have fusion optimizations in the last few weeks. GeoHot didn't want to make it only FlashAttention specific, he worked on FlashAttenrion being automatically generated by the optimizer. It's going well | ||