I wrote a software synth myself with the intention of running it on Raspbery Pi 3 / Zero 2. Those are actually quite capable processers; sound synthesis requires very little RAM, both code and for maintaining state, so everything fits in the rather tiny cache. But at the same time, while these Pis use "little" cores, the maximum throughput of NEON instructions is actually the same as for the corresponding "big" cores like the Cortex-A72. With four cores, you can do in the order of ~10 GFLOPS 32-bit FMA instructions. With a sample rate of 96 kHz and 32-note polyphony, you theoretically have a few thousand FMA instructions per note to spend.

To put this into perspective, five FMAs per sample is enough to implement a biquad filter. This is a very common DSP building block that can implement an idealized version of any of the 2nd-order filters that were ubiquitous in analog synths, e.g. high pass/low pass/band pass/notch/all pass. See the famous Audio EQ Cookbook for examples:

https://www.w3.org/TR/audio-eq-cookbook/

By chaining various combinations of EQ and non-linear distortion (lots of ways to implement this, probably involving more FMAs) and you can build very good simulations of common analog synth signal paths.

Note that gsliepen's example sample rate of 96 kHz is perfectly reasonable in this context; it's more than you need to exceed the limits of human hearing, but it's common to oversample your signal for processing to avoid problems with aliasing.