|
| ▲ | simonw 4 hours ago | parent | next [-] |
| It's part of the tradition. |
| |
|
| ▲ | pwython 31 minutes ago | parent | prev | next [-] |
| My comment on GLM-5 five months ago: "How many pelican riding bicycle SVGs were there before this test existed? What if the training data is being polluted with all these wonky results..." https://news.ycombinator.com/item?id=46974853 |
|
| ▲ | Sol- 2 hours ago | parent | prev | next [-] |
| I wouldn't be surprised if models were optimizing for pelican-related comment chains at this point |
| |
| ▲ | lambda 2 hours ago | parent [-] | | You can always ask them to draw something else, as a way to avoid any possible pelican related data contamination; given how popular the pelican test is, I'm sure there's some pelican SVG drawing in the training sets of at least some of these models by now. For instance, you could ask for an SVG drawing of a cyborg bear riding a rocket powered unicycle. It's a silly fun little benchmark, and because Simon's been doing it for so long, you have a lot of examples over the years to compare. But you can always come up with and run your own test with other drawings. | | |
|
|
| ▲ | SalariedSlave 3 hours ago | parent | prev [-] |
| we should automate this |
| |
| ▲ | edanm 3 hours ago | parent | next [-] | | Based on the amount of output, I'm fairly sure simonw has replaced himself with ai years ago :) | |
| ▲ | gilfaethwy 2 hours ago | parent | prev [-] | | Claude, automate this thread, make no mistakes. |
|
