I looked into this "GRAM" stuff a sibling comment links further to, and just to say:

- this gets reinvented/rediscovered constantly under different names

- it cant be trained very well (right now, will change)

- massive theoretical improvements over current models (log_2(vocabsize)=17, residual stream dim is thousands of dimensions, recursivity means more information bandwidth by ~3 OoM)

- BUT it cant be interpreted or aligned <- this is why no one uses it and no one talks about it. the idea is 100% obvious to all the frontier labs and there is a good reason why it isn't used

I follow this stuff closely, I think I know what I'm talking about (edited for formating)

▲

onlyrealcuzzo 2 hours ago | parent | next [-]

> - this gets reinvented/rediscovered constantly under different names

What are the different names? I haven't seen this before.

> - it cant be trained very well (right now, will change)

If you're sure it will change, then why are you certain that it hasn't yet, and if it's proven a 5000x boost in reasoning... why aren't they exploring this path more aggressively?

> the idea is 100% obvious to all the frontier labs and there is a good reason why it isn't used

Surely someone is willing to take a 5000x boost in reasoning on a small research model... None of them have even tried anything resembling this AFAIK. It does not seem like something 100% obvious to them.

▲

everforward an hour ago | parent [-]

> Surely someone is willing to take a 5000x boost in reasoning on a small research model... None of them have even tried anything resembling this AFAIK. It does not seem like something 100% obvious to them.

Without knowing anything about the technology at all, if it can't be aligned I could see no one pursuing it. As far as I know, alignment is where the "don't tell the user how to make meth or generate CP" instructions end up and the last I saw eliding all the unsavory training data made materially worse LLMs.

It could maybe be post-evaluated by a non-GRAM LLM? Not being aligned is probably a fatal flaw or at least a very short runway into Congress.

▲

jjmarr an hour ago | parent [-]

Many open-source models prioritize alignment less than American frontier ones and respond to those instructions. Why haven't they adopted GRAM?

	▲	everforward a few seconds ago \| parent [-]
		Which ones are you thinking of? It feels to me like all the open source models I've seen lately are still pushed by corporate entities who don't want the legal blowback. I can't really think of a new open source model that's "by the people, for the people" in the sense of a crowd-funded/trained model.

▲

l674 4 hours ago | parent | prev [-]

Could you explain how/why GRAM cannot be interpreted or aligned how current LLMs are? Not very familiar how it works

▲

kmavm 4 hours ago | parent | next [-]

Crudely? Because you can't grep a sequence of latent states for variants of "If I kill all the puny humans, I can <achieve my current goal>."

▲

onlyrealcuzzo 2 hours ago | parent | next [-]

Why do you need to grep latent space?

As long as it's giving the right outputs, who cares what's in latent space?

If the model thinks in latent space: "God I wish these people would die," and constantly does the right thing, who cares?

Additionally, if one of it's latent spaces that it never explores is a psychopath -> who cares? The path never gets taken...

That's a lot of harmless people walking around with crazy thoughts...

	▲	noddybear 2 hours ago \| parent \| next [-]
		Thinking ‘God I wish these people would die’ could increase its propensity to kill all people, even if that propensity is still vanishingly small almost all of the time. A lot of people are walking around with crazy thoughts. Some of them harm.
	▲	czl 2 hours ago \| parent \| prev [-]
		[dead]

▲

czl 2 hours ago | parent | prev [-]

[flagged]

▲

sometimelurker 3 hours ago | parent | prev [-]

sibling comment got to the main points before me, but to add on kmavm's reply, the attack surface for gradient decent to get the system to exchange "bad information is much higher in latent reasoning models (like GRAM). You get ~3 OoM more bits (~17 bits per token in a standard CoT vs the whole residual stream of the model @ f16 = a few kb) per forward pass of the system coming back to itself, and even if you could sift through all that for signs of misalignment, you just can't put a blockade on all of the bad things that leak through.

	▲	haldujai 2 hours ago \| parent \| next [-]
		I think you’re overstating the impact of interpretability here. Your earlier point that latent reasoning models can’t be trained very well and that discretization may be load bearing rather than a readability tax in addition to significant inference infra hurdles (e.g. batching, speculative decoding) have limited any serious attempts and reduced the theoretical advantage over CoT at least in the near term.
	▲	ACCount37 2 hours ago \| parent \| prev [-]
		Most alignment methods nowadays don't rely on interpretability. And neither do all LLM vendors care about alignment much - especially not in China. Those things being untrainable at scale is why they aren't around. Alignment is an afterthought.