But who gives it the intial prompt as to what to do?

Someone has to type in "create the next billboard top 1 single" ?

If you take Daniel Schmachtenberger's thesis, these models are the latest extension of an ongoing misaligned moloch-like entity, using our institutions/economy/corporations, etc. as its organelles. All powered, most recently, by fossil fuel.

the product owner/manager

The ownership class, the capital class - whatever people prefer to call it, will still call all the shots as they always have, and always will.

So whether that's a CEO at the top instructing a human manager that then instructs AI systems, which then put into motion plans that include the harnessed humans getting prompted, or however it's decided to best configure the top operational tiers - that's all fairly irrelevant ultimately. All human labor, and all AI labor, will be in harnesses. Realistically all human labor has always been in a harness.

The people in charge will be the people that control the capital. They will instruct the AI to their bidding, they will control the data centers the AI lives in, they will own the robots doing the labor in factories, they will own the factories, they will control the grid, they will control the electricity generation, et al.

It's true now, it'll be true tomorrow. Nothing fundamental will shift at that tier: don't be labor, own.

If you're a software developer, you're increasingly depreciating labor. Every unit of improvement the AI systems see, the average human software developer's labor value diminishes. It's diminishing extraordinarily fast, kind of like a confrontation between increasingly productive machines and physical human labor during the industrial revolution. You might as well picture a human with a hand plow taking on a modern industrial tractor.

"Maximize my share of the economy", from one of the leaders of a frontier lab, should probably be enough.

It'll be Sam Altman "make people love me, fan out humans".