> Could this be causing me to run out of heap memory faster

Yes, definitely.

> I thought I would be able to compute the graph with 64GB of ram but it kept crushing before CH and LM stage.

For normal GraphHopper and just the EU the 64GB should be more than sufficient.