I am now not commenting on these specific prompts or participating in discussion about them, as I have not investigated how this project works in general, and whether their approach is legitimate in the larger context.

Specifically, I am not advocating for anything criminal and crimes against children are something that really bothers me personally, as a father.

However, in general terms, our thinking appears to be often limited by our current world view. A coherent world view is absolutely necessary for our survival. Without it, we would just wonder what is this thing in front of us (food), instead of just eating it.

However, given that we have a constant world view, how do we incorporate new information? People often believe that they will incorporate new information when provided with evidence. But evidence suggests that this not always necessarily so in reality. We sometimes invent rationalizations to maintain our world view.

Intellectual people appear to be even more suspect to inventing new rationalizations to maintain their world view. The rationalizations they make are often more complex and logically more coherent, thus making it harder to detect fallacies in them.

When we meet evidence that contradicts core beliefs in our world view, we experience a "gut reaction", we feel disgusted. That disgust can obviously be legitimate, like when somebody is defending crimes against children, for example. In such cases, those ideas are universally wrong.

But it can also be that our world view has some false core belief that we hold so dear that we are unable to question it or even see that we oppose the evidence because our core belief has been violated.

We cannot distinguish between these just by our emotional reaction to the subject, because we are often unaware of our emotional reaction. In fact, our emotional reaction appears to be stronger the more false our core belief is.

If you go deeply enough to almost any subject, and you compare it to the common understanding of it in general population, for example how newspapers write about it, there is usually a very huge gap. You can generalize this to any subject.

Most of this is due to just limited understanding in the general population. This can be solved by learning more about it. But it is not unreasonable to think that there may also be some ideas that challenge some basic assumptions people have about the subject. Hence the saying "if you like sausage, you should not learn how it is made".

What you appear to be suggesting is that as you cannot think of any subject that you believe the general population (or you specifically) has false non-trivial core beliefs bout, then such false core beliefs do not and can not exist, and people should not be morally or legally allowed to make a project like this.

You are asking for evidence of a core belief that you have a wrong belief about. But based on the above, if you would be presented with such an example, you would feel gut reaction and invent rationalizations why this example is not valid.

However, I will give you an example: this comment.

If you think the analysis in my comment is wrong, try to sense what is your emotional reaction to it.

While I agree with your your gut reaction to the prompts, it seems to me that you are rationalizing your gut reaction.

Your reasoning does not appear to be rational under more a careful scrutiny: even if you cannot invent anything bad actors could use LLM for (lets say a terrorist in designing a plot), that does not mean it could not potentially be used for such purposes.