>Adding new backends should be trivial.

Sounds like famous last words :-P

And I don't really know about faster once you start to handle all the edge cases that invariably crop up.

Point in case: gcc

It's the classic pattern where you redefine the task as only 80% of the original.