Interesting article to read but by defining memory as a prediction about future needs they are side stepping the real problem that these models cannot learn continuously i.e. update their weights. Opus 5.5s real memory , experience and knowledge is set in stone no amount of markdown tricks will change that.

This is a better research direction I feel

https://research.google/blog/the-sleep-framework-a-new-way-f...

▲

ffsm8 a day ago | parent | next [-]

Continuously updating weights will never be viable for a company like anthropic that's providing cloud inference.

So if you're hoping for that you should probably just look at the models that can be ran locally... And then you prolly realize that that's still several years into the future.

▲

DriverDaily a day ago | parent | next [-]

Without updating weights, AI will always be the main character in Memento with a hyper-optimized filing cabinet.

AI that remembers through continuous experience instead of documents would be significant and lucrative enough to change the business model.

	▲	RussianBot9580 an hour ago \| parent [-]
		You likely don't actually want an AI that learned only from your business documents. Imagine the equivalent of this in a person: someone who only consumes your wiki, no news, books, TV shows, games, conversations... Such a person would get "overfit" to the wiki, lose the ability to think rationally, and probably end up in psychiatric care. We assume that weights need to be continuously updated because that's how lay people imagine the human mind to work. But even the human mind (most likely) doesn't learn continuously: there is a consolidation process that happens during sleep. So in some sense AI learning is already not that different: latest data from short term is consolidated into long term memory. It's just that for a person this happens nightly and for an AI - quarterly. And the short term memory is that person's experience vs the entire Internet. And whether something is lucrative or not is decided by utility and price, not how the technology is put together.

▲

reacharavindh 7 hours ago | parent | prev | next [-]

My guess would have been that every tenant gets their own “handler/driver” model that updates its weights based on the user intent, problem etc.. but it has “experts” to call on demand that are pre-trained set weights. I thought this is how MOE models worked. Then a user gets access to large weights where it matters..

	▲	ffsm8 6 hours ago \| parent [-]
		I don't work for anthropic, so I don't know how they've implemented their MOE... But at least with the ones I know you cannot easily "attach" another expert to the model like that. Maybe they do have a special sauce that let's them do so. If so, it may be viable but it's effect would be a lot less effective then the full version this discussion was started on.

▲

tombot 21 hours ago | parent | prev [-]

Could you say more about why it’s not viable for cloud inference? I’d love to understand this

▲

ffsm8 8 hours ago | parent | next [-]

Think about it from an ops perspective: for running inference, a model is essentially a gigantic binary blob that needs to be loaded into ram to process the request.

Updating weights means your changing this model.

To facilitate this feature, the inference provider would need to have your personalized model loaded into ram to process your request in a timely manner. Now multiply that by the number of users they're serving.

You can try to optimize it a bit by essentially gluing the changing weights as a "overlay" to be applied to the original model, but that comes with it's own trade-offs (aside from the fact it hasn't been implemented, just theoretically doable). And it would be seriously hard to implement well.

The amount of resources this would bind makes this infeasable for cloud inference like anthropic provides.

▲

stevenhuang 4 hours ago | parent [-]

Doesn't sound like you've been following the space. They exist, it's called LoRas. The Unsloth project is huge and whole purpose is to make it easy to train to train LLM LoRas. The reason you don't see LoRas on their own is because people who fine tune models merge them back into the base and ship the full model weights instead

Also see LoRAX, it's an inference engine that exactly does this, it can serve many LoRas off a single base model

▲

ffsm8 2 hours ago | parent [-]

I see, so you're off the opinion anthropic could provide a loras adaptation per user ready for use to enable timely response for every subscriber.

It is an interesting thought to explore. I admit I'm not a specialist whatsoever on the topic, from my perspective that sounds like an incredible challenging ops problem.... But again, I don't work for anthropic and maybe they're up to the challenge as you seem to think.

	▲	e12e 2 hours ago \| parent [-]
		If users get a fixed set/numbers of Paramus for LoRa - say 1-5% - it doesn't sound that crazy. The crazy part is probably loading, running a model pr users - but if the untrained (unspecialized?) model can somehow stay resident - might be possible to load adjusted weights on demand? I suppose each run now needs a different context pr user/customer anyway, for cache hit rate, if nothing else?

▲

21 hours ago | parent | prev | next [-]

[deleted]

▲

nullsanity 14 hours ago | parent | prev [-]

[dead]

▲

ford a day ago | parent | prev | next [-]

I think the end state of any productive LLM includes both: - a real-time knowledge/context graph that keeps track of state - continuously updating weights

Long term I think it's probably more interesting to explore continuously updating weights. But (and this may be yet another opinion that falls victim to the bitter lesson) even if it's _theoretically_ possible to encode everything in weights alone, I'm not sure it's not the most optimal. Today it's too expensive & finnicky to get better on your own tasks without overfitting (ie [0])

Practically (like next few months) getting the right instructions & context would be a huge improvement in capabilities.

[0] https://thinkingmachines.ai/blog/on-policy-distillation/#tra...

▲

zbyforgotp 20 hours ago | parent | prev | next [-]

Why real memory is weights? Why the learning entity must be the model and not a composite system?

	▲	ghoblin 16 hours ago \| parent [-]
		Because of generalization. Updating weights allows the model to internalize patterns and apply what it learned to novel situations rather than just retrieving and reusing stored information

▲

nullc 15 hours ago | parent | prev [-]

Linear attention like Qwen's GDN is essentially equivalent to dynamic weight update.

I think for many case 'memory' not tied to immediate context is problematic, particularly if its illegible. It's common for agents to make an error and then fixate on them-- e.g. has a wrong conclusion about a root cause, states it emphatically, and then it keeps coming back to haunt you.

With memory consisting of files on disk you can hunt out and destroy the source of such confusion. If it gets buried in weights you may be SOL.