Have you tried allowing the LLM to decide the use of knowledge retrieval, through tool use or a direct query?