>There's no reason to believe that the average painting would continually go up in value.

If I had to guess, on average works of art go down in value, simply because of how much is produced.

>Some random pieces blowing up in value if the artist becomes famous later, but most would go nowhere.

That's exactly where speculation comes from. It's the same as any form of gambling. On average you're losing money, but every once in a while one lucky guy makes off like a bandit.