> Measure. Measure. Measure.

The problem is knowing what to measure. There's another saying

"When a measure becomes a target, it ceases to be a good measure."

As an example from memory, there was a game dev company that celebrated they had maxed out the cores on the PS3. That didn't mean anything though, anyone can max out the cores by filing them with bad code. But hey, their "measurement" told them they had maxed out the machine

▲

tialaramex a day ago | parent | next [-]

> The problem is knowing what to measure.

This can be a problem, but much less so because so often we're doing "easy mode" where we don't need a proxy. The "it ceases to be a good measure" is because you're measuring a proxy. You wanted to deliver happiness, you measured wealth because it was easier to measure but seemed correlated and now you've got rich miserable people, oops. But software engineers can often measure the actual thing they want to improve directly, not a proxy and so it cannot cease to be a good measure.

▲

ahartmetz a day ago | parent [-]

With a few caveats, though they do tend to get fixed over time. For example, frame rate. Higher frame rate is better, the end, right? Wellll...

Latency and hitching are annoying to measure, so for a long time, they were pretty much ignored. That has improved and hopefully will continue to improve. There is the bufferbloat initiative, gaming magazines take frame time histograms now, input devices and screens commonly have their latency measured. But latency is still under-measured - for GPUs / GPU drivers, for all software, for games and particularly for websites.

	▲	tialaramex 17 hours ago \| parent [-]
		Yeah, frame stutter is definitely one of the cases where somebody may end up targeting a proxy and that saying about targets happens as expected. A game which typically hits 50-60fps but has occasional half second stutters may not fix the stutters if your proxy was "time to render 100 frames" and you throw out outliers - since every stutter will be an outlier.

▲

groundzeros2015 19 hours ago | parent | prev [-]

That quote mostly applies to human organizations where the connection between outcome and metric is questionable. Or where incentives unexpectedly change behavior.

If your goal is fast software it can be measured quantively and you’re likely to improve the actual thing using those metrics.

	▲	socalgal2 9 hours ago \| parent [-]
		The goal above was to write fast software. The developers made the mistake of seeing that they had saturated all the cores as "We did a good job of using the entire machine" = "our code is fast". They had a measurement and misinterpreted what it meant.