| ▲ | FergusArgyll 3 hours ago | |
Interesting that he's attributing so much to the base model. I thought the majority of the gains are due to post training | ||
| ▲ | Ohentis 2 hours ago | parent [-] | |
If the majority of gains were due to post training, they just wouldn't do pre training. Many of the tasks people are trying to do with these llms (math and coding) don't actually benefit from natural language skills. | ||