I understand wanting to run a local LLM for privacy, but on something more powerful. Yes it's costly but what use is an LLM on a cheap board?

Depends entirely on the model you want to run.

There are some absolutely useful things you can do with TTS/STT/Diarization/etc on even really minimal specs.

Some of those will run fine on RPis even without this new hat.

The extra ram probably opens the door to a large number of vision/image models, which typically want a minimum of 16Gb, but do better with 24/32.

There are just a HUGE number of case specific models that do just fine on hardware at the RPi level, assuming you have the ram to load them.

I was mostly responding to

> [...] if I want some low power linux PC replacement with display output, for the price of the latest RPi 5, I can buy on the used market a ~2018 laptop

I guess. I don't care about the AI hat at all.