I've been testing HPL and mpirun a little, not yet with this new RDMA capability (it seems like Ring is currently the supported method)... but it was a little rough around the edges.

See: https://ml-explore.github.io/mlx/build/html/usage/distribute...