I always thought of the Pelican more of like a gimmicky quick test. There are people who took it as a serious benchmark for overall model performance?

▲

NitpickLawyer 4 hours ago | parent [-]

"Draw a pelican on a bicycle" is not a serious benchmark.

"Draw an animation of this long ass scene from a movie, and only call me when everything works e2e" can be.

	▲	ActionHank an hour ago \| parent [-]
		I feel like we are going to look back on this era of llm usage like we do at the period of time when we thought radiation was magic. Consuming radium and using uranium glass, that’s what we’re doing.