The REILs are part of ARFF training. Pilot training on it is also clear. The system is automated. It plots the direction and speed of anything approaching the runway and predicts a conflict. If the REILs are red it is HIGHLY likely there is a conflict that is missed by human error and you should not proceed without confirming. Don't just confirm cleared to cross, explicitly tell the controller "XYZ tower we have red runway entrance lights. Please confirm runway XX is clear".
The system is smart enough that if you get red bars to cross for an airplane departing once it passes your position the red clears because it knows the airplane is already past you. It is not dumb - it was deliberately designed to minimize false positives so everyone would trust it otherwise they might ignore it when it really counts. (AFAIK it very accurate in fact so the firetrucks weren't crossing because they distrusted the red lights).
This is just like all aviation incidents and indeed most incidents of any kind: the holes in the swiss cheese lined up.
The emergency aircraft couldn't find a free gate, creating a massive distraction for ATC, airport, et al. This is probably the primary domino that started the sequence. Had a gate been free this incident would not have happened. One big hole lined up.
Normally the aircraft would visually see the truck or the truck would visually see the airplane. But it was dark and rainy. Another hole lined up.
Everyone involved was rushing because noise abatement requires the airport to close at a certain hour. Thus everyone wanted to take-off or land before that shutdown. Another hole.
Normally the controller wouldn't issue the clearance to cross or their supervisor monitoring behind them would notice the error and override. But the controller and/or supervisor were distracted by the emergency. Another hole lined up.
The controller realized the error and issued a stop command but the fire truck proceeded anyway; they may or may not have heard the transmission. Another hole lined up.
Then someone else decided to jump on frequency during this busy time (we don't know who just yet) which may have prevented the controller's stop and/or go-around commands from being heard (another hole lined up).
The ARFF crew did not obey the REILs, accepting the clearance. Perhaps they thought the red lights were due to aircraft on short final and they still had time to cross? Perhaps it was some other misunderstanding of how that system works. Another hole lines up.
And the Air Canada jet was not paying attention to the chaos on frequency. There's a reason runway crossings are typically done on tower frequency: so aircraft can hear what is going on. But it was late at night and their brains probably didn't process what was happening. Or they were too close to touching down to have the bandwidth. Another hole lined up.
