Remix.run Logo
gigatexal a day ago

You people can’t be serious, right?

Ok so I build my data lake on s3 using all open tech. I’m still paying for S3 for puts and reads and lists.

Ok I put it on my own hardware. In my own colo. you’re still paying electricity and other things. Everything is lock in.

On top of that you’re beholden to an entire community of people and volunteers to make your tech work. Need a feature? Sponsor it. Or write it and fight to upstream it. On top of that if you do this at scale at a company what about the highly paid team of engineers you have to have to maintain all this?

With snowflake I alone could provide an entire production ready bi stack to a company. And I can do so and sleep well at night knowing it’s managed and taken care of and if it fails entire teams of people are working to fix it.

Are you going to build your own roads, your own power grid, your own police force?

Again my point remains. The vast majority of times people build on a vendor as a partner and then go on to build useful things.

Apple using cloud vendors for iCloud storage. You think they couldn’t do it themselves? They couldn’t find and pay and support all the tech their own? Of course they could. But they have better things to do than to reinvent the wheel I.e building value on top of dumb compute and that’s iCloud.

thejosh a day ago | parent [-]

After running Snowflake in production for 5+ years I would rather have my data on something like Parquet/Iceberg (which Snowflake fully supports...) than in the table format Snowflake has.

It's not that deep

gigatexal a day ago | parent [-]

Ok. And this flexibility is only really possible since they did a lot of work to make external and internal tables roughly equivalent in performance.

thejosh a day ago | parent | next [-]

Yeah, performance depends.

I think a hybrid approach works best (store on Snowflake native and iceberg/tables where needed), and allows you the benefit of Snowflake without paying the cost for certain workloads (which really adds up).

We're going to see more of this (either open or closed source), since Snowflake has acquired Crunchydata, and the last major bastion is "traditional" database <> Snowflake.

gigatexal a day ago | parent | next [-]

I had no idea they did. This pg lake announcement dropped that nugget and i was surprised.

gigatexal a day ago | parent | prev [-]

Agreed btw.

nxm a day ago | parent | prev [-]

They didn't do it out of good will. They realized that's where the market was going and if their query engine didn't perform as well as others on top of iceberg, then they'd be another Oracle in the long-term.