Remix.run Logo
sxp 21 hours ago

Yeah. Ironically, if he used an LLM to proofread his work, it would have told him that GAN's are generative not "generalized". That LLMs are primarily built on Transformers rather than GANs. And that they're more famous for image generation rather than image recognition.

bonoboTP 21 hours ago | parent | next [-]

Transformers are an architecture and GANs are a training method for the architecture. There are GAN Transformers.

andy99 19 hours ago | parent | next [-]

Do you believe it was this edge case he was referring to in making his point?

bonoboTP 18 hours ago | parent [-]

No, I simply think the person is not familiar with these concepts, which is not a crime, it's okay to be wrong, I didn't want to banish that person from here, just corrected the statement. But might have been also a brain fart. Happens. Just wanted to correct it in the interest of beginner reading this.

Once Musk said he believes in Transformers instead of diffusion (or the other way around). When actually diffusion transformers are very popular and mainstream. What he meant was autoregressive inference vs diffusion. People like to use buzzwords while not knowing them. Old issue, it was the same decades ago.

To make it even more confusing, there are also diffusion LLMs, which typically but not necessary, use transformers also.

And independently of diffusion and transformer and llm or image generator, you can optionally put a GAN discriminator adversarial loss on any of them.

nullstyle 20 hours ago | parent | prev [-]

Examples for us less educated folk?

bonoboTP 19 hours ago | parent | next [-]

Analogy "it's not GAN, it's transformer!" - - "it's not a recursive implementation, it's object oriented!" if you are more familiar with CS.

Or "it's not 4-wheel-drive, it's diesel!", if familiar with cars.

Alpha3031 19 hours ago | parent | prev [-]

Jiang et al. (2021) TransGAN: https://dl.acm.org/doi/10.5555/3540261.3541391

In general probably not much of a stretch to get rid of convolutions or recurrence by replacing with attention and see if it works hence the title of the original transformers paper.

FeepingCreature 21 hours ago | parent | prev [-]

And that GANs haven't been relevant in image generation since 2021.