It really depends on what you want to do. Start with a list of projects you want to build. Are they really robotics or just home automation? If you have no experience with embedded programming, I'd start with home automation or other simple hardware stuff first. Learn how to read sensors and operate lights or motors before you get any deeper into the craft.

There are three reasons why "real" robotics is harder. First, to navigate unconstrained environments by means other than wheels, you need to learn a fair amount of kinematics and control theory. Second, your robot will need to build a map of the operating environment, which often requires complex sensing (lidars, cameras) and a lot of compute power. Even something as basic as a Roomba is no small feat, and these things get stuck a lot. Third, it's just hard to come up with practical applications for simple robots outside of industrial production lines.

Once you have that figured out, decide how much you want to learn. Do you want to do it the hard way, which would mean moving more slowly and working your way up from first principles and probably 3D printing or machining your own parts... or do you want to get results as soon as possible, in which case, you probably buy a Linux single-board computer and put it on top of an existing robot kit? The trade-off with the first approach is that it will take you months or years to get where you want to be. The downside of the second is that you're limited to premade kits and software components, and once you exhaust the possibilities there, you kinda hit a wall.