That matches my experience, too. I passed JLPT N1—then called 1-kyū—back in 1985 (!).

I did spend a lot of time memorizing vocabulary with flashcards, but I spent even more time on extensive reading—novels, newspapers, magazines, anything I was interested in, even if at first I understood little. The repeated exposure to vocabulary in real-world contexts really made a difference.