| ▲ | TinyLoRA – Learning to Reason in 13 Parameters(arxiv.org) |
| 68 points by sorenjan 5 days ago | 6 comments |
| |
|
| ▲ | a-t-c-g 23 minutes ago | parent | next [-] |
| The quality of custom models trained with proper reasoning datasets[0] even with small parameters (3-7B is sweet spot) is incredible now [0]: cartesien.io or Salesforce's WebscaleRL |
|
| ▲ | measurablefunc an hour ago | parent | prev [-] |
| With four parameters I can fit an elephant, and with five I can make him wiggle his trunk so there is still room for improvement. |
| |
| ▲ | esafak an hour ago | parent [-] | | Except learning to reason is a far cry from curve fitting. Our brains have more than five parameters. | | |
| ▲ | voxelghost 12 minutes ago | parent | next [-] | | After a quick content browse, my understanding is this is more like with a very compressed diff vector, applied to a multi billion parameter model, the models could be 'retrained' to reason (score) better on a specific topic , e.g. math was used in the paper | |
| ▲ | ekuck 14 minutes ago | parent | prev | next [-] | | speak for yourself! | |
| ▲ | est 30 minutes ago | parent | prev [-] | | reasoning capability might just be some specific combinations of mirror neurons. even some advanced math usually evolves applying patterns found elsewhere into new topics |
|
|
