> I would like an audio device which can play mp3, podcasts, internet radio.

Get a second-hand Apple iPod Touch, remove all apps you don't need.

For just mp3 and podcasts: get an iPod Classic (or Video) and install Rockbox.

Rockbox is amazing.