Remix.run Logo
nklmilojevic 3 days ago

Sure thing!

Ingested logs 24h: 428 Mil Ingested bytes 24h: 625GB Inser req/s: 6k/s

8vCPU, 16GB mem. Running standard-rwo PVC on GCP.

We have a couple of projects like this with similar usage and similar machine sizing.

Still running vmlogs-single, and we will until we see a need to move to vmlogs-cluster version.

camel_gopher 3 days ago | parent | next [-]

That sounds like a lot of resources provisioned for 6k/s. 625GB/24hr is a small footprint.

CubsFan1060 3 days ago | parent [-]

I would have said it sounded pretty good. What technologies are you comparing it against, out of curiosity.

camel_gopher 3 days ago | parent [-]

ClickHouse

CubsFan1060 3 days ago | parent [-]

Can you share any additional details? What kind of ingestion do you have, with what dimensions of a clickhouse cluster?

I'm also curious how it handles structured vs unstructured logs.

Thanks!

CubsFan1060 3 days ago | parent | prev [-]

That seems pretty good. Do you have any sort of HA solution?