| > But with no benchmarks or price comparison, which likely means it doesn't compare that well.
Doubt that's the case; this was only the announcement and the API isn't even released yet. As I noted, my own testing using a batching strategy (nothing more than "read 20 lines from this file instead of reading 1 at a time") yielded ~the same accuracy on an 800 row dataset we have with ~1.6x the throughput (at only batch_size=20) and 1.2x the cost. Didn't tune to find accuracy dropoff by batch size, but it's easy to see that the decision capability was there (understood that it's not the same without the probabilities, but even without it, the classifications were nearly identical). |
