LLMs might be the least interesting of the statistical models for creative purposes, they're kind of nasty to work with. Creating low rank adaptations is slow and expensive because the models are so fat, and meddling with the inference flow is a much more explosive game of cat and mouse. You can tell Flux where colors, shapes, textures, etc. should live on a canvas. Trying to wrangle an LLM into "spatially/temporally" arranging text in accordance with a writing style is a nightmare and a half. It's complicated, I can't really explain it well. Probably because I still haven't been able to put in the time to figure out the "grain" of pure text transformers. I can tell you that they're very hard to work with, though.

The simple fact of the matter is that these things are best viewed as a computer implementing a continuous form of computation. If you can understand the following words: "the function between function 1 and function 2 is function 1.5" and can imagine that this is a process with "infinite" descent, where you can continue to pull functions halfway between other functions trivially, that's a pretty great mental model to have. To that end, using them is like operating a big, complicated radio setup. Or a huge collection of synthesizers and filters. You're essentially tuning in to a structure. It can't help you figure out what structure you should reach for, or what you should do with that structure.

The creativity doesn't come from the tool itself. If you were only capable of having a mediocre body of work before, they aren't going to help you after. There is no removing human brilliance from the equation. AI is immensely exciting for what it promises, there is something genuinely new and interesting here, but it's not a crutch for the untalented. That's how it's getting sold, and it's having a massive disservice done to it. What we have is a genuinely new space to pioneer in. The interesting stuff and compelling art isn't going to be found at the end of a however-many-word positive prompt, but a symphony of total control over the model itself. Drawing 50 original works for the purpose of fine-tuning a diffusion model for a single project, and only as one particular component of the fully customized inference workflow, which will be scrapped when the project is completed. That's what artistic usage of statistical models looks like. The construction of a specific program for a specific purpose. Anything else is fantasy. The model can't give you that purpose.