If you are a programmer then one of the synthesis and composition DSLs would probably be your easiest entry, CSound and SuperCollider are a good place to start, Csound has better non-realtime capabilities, SuperCollider has better realtime, realtime meaning live performance; they overlap a great deal and neither falls completely into realtime or not. PureData is a good option for the less accomplished programmer but still comfortable with programming. There are a great many other DSLs but these three have been around the longest and are well established so a very good place to get started. The Computer Music Tutorial is a great book and the best geberal reference for these languages going over all the techniques in a generic way applicable to most of these DSLs.

For the non-programmer who does not want to get into programming, something g like LMMS is a good place to start, gives you everything you need and a simple interface that can be quickly learned.

Or just do like we all used to do, buy a synth and dive in.