LLMs are next token predictors. They predict the next most likely token given the previous context window of N tokens.

This means not giving an answer is not a technically possible option. Best you can do is force it to output a magic "stop speaking" token, but this is a vastly different training problem than getting it to not know something.

People naively expect LLM outputs to have some sort of confidence value when predicting, but the technology just doesn't work that way.