I struggle to think of a way of keeping an entire house or apartment warm without power.

Is there an actual solution recommended by the respective governments for this or was the problem reformulated into keeping one or more persons warm instead?

You can't keep it warm but you can conserve heat as much as possible. You're supposed to add extra insulation by covering the windows with blankets, pick a space in the house where everyone will be and additionally insulate that main space from the rest of the house, use candles as it's safe to.

Our houses are generally quite well insulated due to the climate. Government preparedness info says a modern house (~15 years old) can go four full days in -20C weather before inside temperatures drop to 5C. A typical 70s row house would drop to that after 48 hours but a 70s brick house can drop that low after just 24 hours. So depending on your house and location, the specific plan for a 3-day outage can range from "you'll be fine as long as you have blankets" to "you need a fireplace or other heater with fuel for at least a few hours a day".

Home design helps. I'm in a very old earth bermed home and even if the temperature outside is -25F (-31.3C) the coldest it gets if I don't have the heat on is about 42F (5.5C) and that's just because my windows are really old and the door weather strip is decomposing. If I throw a few logs in the wood stove it can get too hot. I doubt there are many earth bermed apartments however. I would probably just put on my snow pants, jacket, hat. Snow rated clothing make a big difference. That is what I would stock up on along with other layers underneath. If I had to sleep outside I would add a tarp, a lot of rocks and some paracord. Not saying it would be fun.

I figure we'd all just go to the country side, where people still have houses with chimneys that use wood for heat, and hope we fit.

We have winter sleeping bags for the whole family. Although we don’t live in Northern Europe it is good to have them at home.