Exactly my thoughts, can not really understand how delayed alerts are acceptable... Have you managed to settle the cost with Google, what was the outcome?

▲

sillysaurusx 11 hours ago | parent [-]

Back in 2020 I had a similar situation. Ended up charging $500 due to an overnight TPU training run using egress bandwidth across zones.

Google support was surprisingly understanding, after I explained the issue. They asked some clarifying questions. Then they said that they can offer a one time refund for this case.

Since then I was paranoid not to accidentally do it again. I don't know whether GCP would refund a second time.

▲

genxy 11 hours ago | parent [-]

GCP charging for interzone traffic is an interesting financial choice. They own all the infra and in many cases this is literally moving from building to building.

▲

sillysaurusx 11 hours ago | parent [-]

There's cross-region, and cross-zone. If both boxes are located within the same zone (e.g. us-east1) then the bandwidth is free, since it's intrazone traffic. Cross-zone egress traffic (e.g. us-east1 to us-central1) is billed at a certain rate, and cross-region egress traffic (e.g. us-east1 to europe-west8) is billed at a significantly higher rate.

Amusingly enough, ingress traffic seems to always be free. So you can upload as much data as you want into their cloud, but good luck if you need to get it out.

▲

genxy 11 hours ago | parent [-]

I am referring to cross-zone within in the same region, so like us-central1-a to us-central1-b. These are building to building and often never cross public land.

▲

sillysaurusx 11 hours ago | parent [-]

Oh, yes! I forgot entirely about that case. You're right, egress traffic is charged there too.

Are the datacenters really located so close together? I assumed they weren't within walking distance of each other.

	▲	coredog64 10 hours ago \| parent [-]
		Correct, they're close in the sense of country-scale geography but physically spaced to avoid specific issues like location on a flood plain.