| ▲ | measurablefunc 3 hours ago | ||||||||||||||||||||||||||||||||||||||||||||||||||||
Generate instructions for their simulator to compute some numbers (hashes) in whatever is considered the memory of their "machine"¹. I didn't see any places where they actually disallow cheating b/c it says they only check the final state of the memory² so seems like if you know the final state you could just "load" the final state into memory. The cycle count is supposedly the LLM figuring out the fewest number of instructions to compute the final state but again, it's not clear what they're actually measuring b/c if you know the final state you can cheat & there is no way to tell how they're prompting the LLM to avoid the answers leaking into the prompt. ¹https://github.com/anthropics/original_performance_takehome/... ²https://github.com/anthropics/original_performance_takehome/... | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| ▲ | saagarjha 3 hours ago | parent [-] | ||||||||||||||||||||||||||||||||||||||||||||||||||||
Well, they read your code in the actual hiring loop. | |||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||