More things in memory also leads to making more connections / recognizing more types of patterns more easily.

There are things that you just see/know/recognize that are not easily searchable/promptable because the fact that you should be searching/prompting would not occur if you had not seen the pattern before.

One could argue "experience" is just the accumulation of memory such that you can recognize more patterns more rapidly, short circuiting bad decisions before they are made. With LLMs its even more important to have a well tuned "what should be getting built" mental framework if that becomes our primary job.