Web version with links, etc:

https://rlhfbook.com/

Thanks! We've switched to that above from https://arxiv.org/abs/2504.12501, and put the latter in the toptext.