Autopilot based on a LLM would guarantee I’d never fly again

That would be scary, thankfully I don't think anyone would seriously consider it. But I could see other systems based on similar models being useful. Obstacle avoidance, emergency decision-making, etc. There are many places where a private solo pilot can get overwhelmed and make poor decisions or ignore important information.