FWIW, "lay monasteries" and hermitages exist, and there are some religious ones which are not strict about who wants to join, the Bose monastic community[0] comes to mind talking of christianity, but there's plenty for other religions.

[0] https://www.monasterodibose.it/en/