| ▲ | satvikpendem a day ago |
| Can't I just parallelize my LLM calls myself for each question? |
|
| ▲ | orbital-decay a day ago | parent | next [-] |
| You can. It will be expensive, slow, and less reliable than a specialized model. |
|
| ▲ | zwily a day ago | parent | prev [-] |
| Anything you can do in Jev can be done with an LLM at much greater cost and latency. |
| |
| ▲ | mmnfrdmcx a day ago | parent [-] | | Agree, except the probabilities for outcomes in the structured output. I don't think you can get those for most frontier LLMs (logprobas). You can get it for open source models but not frontier LLMs. | | |
| ▲ | esafak a day ago | parent [-] | | That number is a big deal, assuming it is well calibrated. Did they talk about calibration? | | |
| ▲ | Matticus_Rex a day ago | parent [-] | | I've seen them talk about it a bit on Twitter -- it seems to be fairly well-calibrated in general, but obviously you need to test it on your use case and dial it in comparison with known data for best results. |
|
|
|
