Google's research showed some of the speed benefits tended to drop off at larger batch sizes on an H100. Are there published batch-size vs tokens/s figures for Mercury 3?

▲

volodia 2 hours ago | parent [-]

We've been happy with how performance has scaled with batch size. The model runs in our API at reasonable costs, which is also what allows us to serve the model with competitive prices ($0.10 per 1M input and $0.35 per 1M output at launch).

	▲	LarsDu88 an hour ago \| parent [-]
		Pricing can always be set at a loss, however, particularly for a private company. Are there solid published numbers for a specific hardware spec?