That is a tricky question to answer. Farms need small towns scattered all over - that is where many of the teachers, accountants, mechanics, hired hands, other services, and owners of the stores that serve all of the above live. Often small towns have factories that are not farm related and those employees live someplace. Do you count those small towns as rural? Many of the above have also realized that they can buy some build a house on marginal farmland cheap and so live rural but they are working a small town job - they may have a few goats or something but it isn't how they earn their money - hard they farmers? There are also people who retire to the country, hunting cabins (not residents), camp grounds (the owner lives there), and other non-farmers living in rural areas. Parents generally transfer the farm to the kid who will inherit it over decades, and part of that is the parents move to a small house off the farm but still rural - are they living on a farm?

Depending on how you count the above you can say that most people in rural areas are not living on farms. Even if you don't count small towns residents, there are a lot of people who are not farmers living out there.