> Someone should make the millennium problems for AI alignment so the frontier labs will actually try to solve that problem.

Rather: Someone should make the millennium problems for human alignment so the top researchers will actually try to solve that problem. :-)