But "write a fizzbuzz fn" has one important assumption / limitation: the LLM should have seen a ton of fizbuzz implementations already to be able to respond.

Hence, LLMs can be helpful to produce boilerplate / glue code, the kind that has already been written in many variations, but cannot be directly reused. Anything novel you should rather outline at a more detailed level.