Do LLMs still use backprop?

samsartor 4 days ago | parent | next [-]

Yes. Pretraining and fine-tuning use standard Adam optimizers (usually with weight-decay). Reinforcement learning has been the odd-man out historically, but these days almost all RL algorithms also use backprop and gradient descent.

▲

ForceBru 5 days ago | parent | prev | next [-]

Are LLMs still trained by (variants of) stochastic GRADIENT descent? AFAIK what used to be called "backprop" is nowadays known as "automatic differentiation". It's widely used in PyTorch, JAX etc

	▲	imtringued 5 days ago \| parent [-]
		Gradient descent doesn't matter here. Second order and higher methods still use lower order derivatives. Back propagation is reverse mode auto differentiation. They are the same thing. And for those who don't understand what back propagation is, it is just an efficient method to calculate the gradient for all parameters.

▲

4 days ago | parent | prev [-]

[deleted]