you would have to train it three times for two colors.

each by itself, they with both interactions.

2!

The models are already massively over trained. Perhaps you could do something like initialise the 2 new token sets based on the shared data, then use existing chat logs to train it to understand the difference between input and output content? That's only a single extra phase.

You should be able to first train it on generic text once, then duplicate the input layer and fine-tune on conversation.