new | show | ask | jobs Github

robhawkes 7 hours ago

Tech stack is a mix of things, but the big-ticket items:

- Dagster for data pipeline

- Parquet files for data format

- Cloudflare R2 for data storage

- DuckDB for data processing

- NodeJS for API

- NextJS + React for the app

- MapLibre for the base map functionality

- Deck.gl for the complex map layer (eg. wind particles)

- WeatherLayers (+ GFS) for the wind particles

- Cloudflare as a caching layer

The main difficulty was learning about the underlying data and industry nuances. I've been working on this map the past few years and I'm still learning new things that force me to change my approach. It's an incredibly complex domain and it's part of the reason I made this, to try and make this complex data a bit more digestible.

▲

jacquesm 6 hours ago | parent [-]

It's a beautiful piece of work, thank you very much.

It would be amazing if this was extensible and wind farms and solar farms outside of GB could contribute their data as well.

▲

robhawkes 4 hours ago | parent [-]

Thanks! I would love to make this something that works beyond GB though right now that would require a fundamental change to the way things are set up. It's on my list though, so one day.

	▲	toomuchtodo 4 hours ago \| parent [-]
		I do some volunteer work with the https://electricitymaps.com/ folks, highly recommend getting in touch if their API(s) would be of use. I can provide a contact if needed.