| ▲ | croemer 2 hours ago |
| Yes, obviously. |
|
| ▲ | swiftcoder 2 hours ago | parent | next [-] |
| Even though none of us could actually afford to train it? |
| |
| ▲ | croemer 2 hours ago | parent | next [-] | | Some of us might be rich, or get funding, or in the future it could be useful when training is cheaper. Open source has benefits even if you can run yourself. You can read the code for understanding/insights. Other labs could replicate/build on it. | | |
| ▲ | gpugreg 2 hours ago | parent [-] | | > Some of us might be rich I sure wish I had a few 100M of disposable income to train a frontier model. > or in the future it could be useful when training is cheaper. I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then. | | |
| ▲ | Muromec 2 hours ago | parent [-] | | If you would tell people at the start of 20th century how much energy we consume, they may not believe you or think it is wasteful. | | |
|
| |
| ▲ | eptcyka 2 hours ago | parent | prev [-] | | Maybe not now, but what about 10 years down the line? |
|
|
| ▲ | WithinReason an hour ago | parent | prev [-] |
| As an example, Grok 4 took $388M to train |
| |
| ▲ | HPsquared 26 minutes ago | parent [-] | | Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights. The weights are literally a binary blob. | | |
|
