Reverse-engineered Jev-like model

Remix clone Hacker News

new | show | ask | jobs Github

▲ Reverse-engineered Jev-like model(github.com)

81 points by rochansinha 7 hours ago | 12 comments

▲ steeve 5 hours ago | parent | next [-]

https://x.com/harshagundal/status/2100044305536889015?s=20

> They were building in stealth for 2 years, I was building in stealth for 2 hours…

> Happy to open source Qwen-2.5-1B-RLCD, 5x faster on-device inference for JSON workloads that need to be type-safe.

▲

flockonus 4 hours ago | parent [-]

No question OSS is amazing, but this video is a satire at best. It doesn't take much attention to see the results on right vs. left side are significantly different.

Jev is not interesting if it's not "smart", a 1B param model is most definitely not smart.

	▲	nullbio 16 minutes ago \| parent [-]
		Jev has a 32k context window. I doubt it's a large model.

▲ mmastrac 2 hours ago | parent | prev | next [-]

Any diffusion model is potentially a Jev in disguise: https://github.com/vllm-project/vllm/pull/57250

Runs ~0.2s per decision on my DGX Spark.

  10/10 programming language detection
  9/10 human language detection
  10/12 unit magnitude comparison

All incorrect answers are marked with low-P.

It (DiffusionGemma with the Jev mode) can also solve an ASCII maze.

▲ vrc an hour ago | parent | prev | next [-]

Out of curiosity and semi unrelated — why do so many of these projects with customized encoder-decoder setups use earlier Qwen versions like 2.5 and 3 and not the smallest 3.5? Purely the few 100m params, or something else in the latter’s arch or pretraining?

	▲	augment_me 41 minutes ago \| parent \| next [-]
		In my experience if you tell Claude to port LLM-like stuff without explicit steering for versioning, it will default to the most popular thing for this in its training window to reduce errors. 3.5 is outside its training data.
	▲	FuckButtons 32 minutes ago \| parent \| prev [-]
		My guess would be qwen 2.5 predates linear attention which would be more complex to use.

▲ rochansinha 7 hours ago | parent | prev | next [-]

Can play Doom too - https://x.com/vinnylarouge/status/2100281651930513460

	▲	DavCreator 6 hours ago \| parent [-]
		https://xxcancel.com/vinnylarouge/status/2100281651930513460

▲ tomrod 3 hours ago | parent | prev | next [-]

I like it! I suspect Jev may have more going on under the hood, but I like the idea of efficient universal transformers

▲ _superposition_ 5 hours ago | parent | prev [-]

That was super quick.

	▲	looksjjhg 3 hours ago \| parent [-]
		Insane