I think the timeline is roughly: SGI (90s), Nvidia gaming (with ATi and then AMD) eating that cake. Then cryptocurrency took off at the end '00s / start '10s, but if we are honest things like hashcat were also already happening. After that AI (LLMs) took off during the pandemic.

During the cryptocurrency hype, GPUs were already going for insane prices and together with low energy prices or surplus (which solar can cause, but nuclear should too) allows even governments to make cheap money (and for hashcat cracking, too). If I was North Korea I'd know my target. Turns out, they did, but in a different way. That was around 2014. Add on top of this Stadia and GeForce Now as examples of renting GPU for gaming (there are more, and Stadia flopped).

I didn't mention LLMs since that has been the most recent development.

All in all, it turns out GPUs are more valuable than what they were sold for if your goal isn't personal computer gaming. Hence the price gone up.

Now, if you want to thoroughly investigate this market you need to figure what large foreign forces (governments, businesses, and criminal enterprises) use these GPUs for. US government is aware for long time of above; hence export restrictions on GPUs. Which are meant as slowing opponent down to catch up. The opponent is the non-free world (China, North Korea, Russia, Iran, ...), though current administration is acting insane.

You're right, high demand certainly plays a role. But it's one thing for the second-hand market to dictate the price of used hardware, and another for new hardware to steadily get more expensive while its objective capabilities only see marginal improvements. At a certain point it becomes blatant price gouging.

NVIDIA is also taking consumers for a ride by marketing performance based on frame generation, while trying to downplay and straight up silence anyone who points out that their flagship cards still struggle to deliver a steady 4K@60 without it. Their attempts to control the narrative of media outlets like Gamers Nexus should be illegal, and fined appropriately. Why we haven't seen class-action lawsuits for this in multiple jurisdictions is beyond me.