Two things here:

I have no data either way but I can imagine that there are many more people who are wealthy and quietly having a great time with it. Most of the retired people I've known, early or not, also enjoyed it. Some have definitely taken up work-like pursuits on their own terms.

Secondly the wealth being the means to achieve this is itself a confounding variable. I don't think it's good for your mind or soul to "have everything," no. Life isn't and shouldn't be merely a series of your own preferences. That doesn't indicate to me that lacking confidence in your mere survival is necessary for human thriving. As far as I know research indicates the opposite.