Check out the approach here: https://github.com/allenporter/home-assistant-datasets and the reports/ directory has a leaderboard for function calling. I'm curious to see how well this model does.

Oh nice, I thought about doing something Ng similar with a docker image and fake devices but never got around to it