I can't speak to anyone else's sneers and disdain, but mine come from all the motivated reasoning surrounding them. One example:

'The Effective Ventures Foundation (EVF), which defines effective altruism as “using evidence and reason to figure out how to benefit others as much as possible”, decided £14.9m of its cash would be best spent buying Wytham Abbey, a 15th-century Grade-I listed manor house near Oxford.'

There were a lot of explanations about why that was the most effective use of funds, but every single one reads like motivated reasoning to me.

Examples like this abound.