Our codex on AWS Bedrock read / write cache ratio was less than 5%. Cache writes are very expensive and they were never being used. This results in codex on Bedrock causing ~10x what it should due to no caching and massive writes.

The workaround in issue resolved for me: web_search = "disabled"

▲

otterley 6 hours ago | parent | next [-]

If you’ve got a workaround, I’d suggest updating the issue description to have it up top there so similarly impacted users can spot it quickly and benefit.

▲

the_duke 5 hours ago | parent [-]

It's already mentioned in the issue...

	▲	otterley 5 hours ago \| parent [-]
		It’s in a comment halfway down the page. It’s not in the issue description. If enough comments are added to the discussion, it might end up being collapsed.

▲

chrisweekly 5 hours ago | parent | prev | next [-]

"causing" -> "costing", right?

	▲	vorticalbox an hour ago \| parent [-]
		In this case yeah. If it’s not reading the cache then it has to compute all the context window again and not just the newest tokens.

▲

yablak 6 hours ago | parent | prev [-]

Way to bury the lede..