Sounds like the types of projects I can make with a Pico will be rather limited if I am a beginner. Thanks!

The Pi Pico W 2 can run micropython, which is a reasonably limited and fun version of python that is accessible for beginners. If the types of projects you have in mind will involve reading simple sensors, turning on lights / speakers / beepers, or responding to buttons then you'll have a good time!

That's not necessarily a bad thing. If you want to build a blinking LED thing, a USB controller, or similar small IoT thing, I find microcontrollers to be easier and more natural than running an entire Linux single-board-computer to do it.

I might break it down as: if you know C/C++, I'd start with a microcontroller [Arduino, ESP32, RP2040/2350, etc]; if you know Python, there are still micros that run it, but a full Pi might be an easier starting point for you. (Either way, if you end up doing this for more than a year, you're going to eventually do both...)

Don't think of the microcontroller storage specs in relation to a modern PC. I use an EV CAN diagnostics tool that runs on an ATmega328 (32K of program storage and 2K of RAM).

Your knowledge will be the limiting factor. For the same price though you can get an Adafruit Trinket (https://www.adafruit.com/product/1501) which has a lot of potential.