So the movie quote says the character was a pumpkin, but definitely not a jerk. Isn't that movie from the late 70s? So the shift took place by then. Or did the movie somehow redefine it?

The article that TFA quotes has a much more thorough description:

In 1979, kids may have been using it to mean "asshole" but the writers were in their 30s and definitely meant it to mean "idiot"

By the end of the '80s, the "idiot" usage seems to be completely gone.