The problem isn't that people complain instead of flagging. The problem is that flagging provides no visible reason or feedback loop. That's what my own experience tells me.

I share that thought, but wonder if is the flagger who needs the comment more than the admins. Like the scene in Raiders of the Lost Ark when Harrison Ford just shoots the man with the big sword, I find just down voting or flagging feels like an act of boredom.