Totally agree that the lack of a common base of evaluation is terrible for the discussion, and benchmaxing only contributes to this.

The only way to get a sense for these systems is to use them on things you know well, and everyone knows different things at different levels.

People also tend to underestimate how fast this is moving and base their take on dated and subpar systems for a variety of reasons, a key one being that the firehose is too big for any one person to have a proper focus on all of it.