Couldn't you instruct your LLM to make the starting dir configurable?