Sounds like you're fighting the weights. What would it take to align the setup with what the LLM expects?