It's at 80~90% the max perf it can achieve on this hardware... Which is DDR3 speeds. I'm working on getting FPGA's with DDR4 and HBM2 next.

Why is there any point in trying with faster memory if you are only at 80% perf on DDR3? You'll just go to 40% perf on the next iteration?

You should only upgrade when the bottleneck becomes memory, right now the bottleneck is still on compute until you are at 100% max perf