I actually got it working with cloud storage on hetzner. Wasn't supersnappy, but it worked. I borked the build though and am planning to run it on my home-server

I'm currently using Nextcloud Memories connected to a Wasabi bucket for photos and being "not supersnappy" is a real dealbreaker. When I scroll through hundreds or thousands of photos and have to wait five or ten seconds for each new page of thumbnails to load, then the same when I open a photos, I quickly go back to Google photos.