Not a dev but I’ve had similar experiences testing AI to help me craft various documents.

I spend ages crafting the perfect prompt only to cross my fingers and hope that the output is what I want it to be when I could have just spent all that time actually crafting the document exactly as I want it to be in the first place. Often I then need to spend more time editing and tweaking the output from the AI anyway.

I’m honestly starting to feel a little crazy because I’m the only one at my workplace that sees it this way.

I've found AI to be a poor fit for writing when the task is getting an idea out of my head and on to the page. Works well enough when the task is taking some collection of already written information and creating writing based on it though.