The memory limit for Safari on iOS is less than 400 MB so it's probably because of that

Laya is just ModernBert fine-tuned. It's still a language model (just a bidirectional one). Let's call it what it is instead of this weird Decision Model mysticism.

That sounds like a lot of fun, love the creativity. Do you have a repo you can share? It would be nice to run it, and if you wanted, maybe I could add it to the demos on https://playground.jeffyclassify.com

Cool project too. If you are looking for a 500MB instead of gigabytes, with evals on Jevbench that you can run fast on CPU check out gutsy.

https://github.com/kouhxp/gutsy

Take ModernBert, add a fully connected layer with 255 outputs, take a bunch of classification datasets from Huggingface, write the code to have the datasets fit the jev format on these 255 outputs, do supervised fine tuning on the datasets with that format. then use a confidence loss of some kind

use it to classify a bunch of research papers (by feeding it section by section, or summaries of sections if too long)

works like a charm

(not a product, so not much to share)

I see Jev as a major step towards commoditizing current LLM paradigm. One thing would be to further optimize this particular route to work purely on CPU. This will grant an option to embed this feature into any application, from MS Office to games. The other is integrating this into agent workflow to vastly minimize token consumption.

Not in a serious manner but I created a testing harness for a Nintendo 3DS game I'm making that uses the OpenAI Decisions API. The main advantage is the speed (~2-300ms per input) which I really need for this purpose.

Also like to see a "layman harness" like LM Studio integrate an open decision model in its workflow.

Not with Jev, but you can use classifiers for a lot of use cases, here’s a few: https://playground.jeffyclassify.com/

I use Jev in a Claude code hooks to detect dangerous commands.

if it works it works! as long as the test set is reasonably large and diverse its better than nothing. You could characterize how robust it is by throwing dozens of different types of work at it and see how much the confidence varies

I think part of what made Jev catch on is that the API is like an if(...) or switch statement

People are just so used to the chat style APIs that they didn't even consider doing things like sending a bunch of emojis to a chat model and then asking for the optimal one in this context etc. Also chat models are pricier for the same behavior and can also output something random like a refusal

But yeah ironically I think in the initial breakthrough LLM paper on GPT-3 in 2020 some of the multiple choice questions were answered by comparing token probabilities of specific continuations rather than fill in the blank

> But then again plain LLMs have been perfectly cheap and serviceable as zero-shot classifiers for quite some time now, so again I'm back to my internal screaming.

It's the scale of "perfectly cheap", jev (specifically) is so dirt cheap and fast that you can throw it at things that should not be justifiable in the past and you barely have to do any work other then quick testing.

on the bright side, maybe this zero-shot classifier wave could be the back bone for more specialized classifiers (with more mindful selection of data and training)

Jev is classification for normies. Look ma, no training.

I want to share the rage. Can you expound on what makes you scream?

> But then again plain LLMs have been perfectly cheap and serviceable as zero-shot classifiers for quite some time now

I'm no prompt engineer, but I've found them to be too slow any time I wanted to use them that way. I have never tried Jev, but apparently it is supposed to be fast, so it seems like, according to the marketing, it could become usable where LLMs haven't been.

Maybe instead of screaming you can take this as a chance to level up your engineering.

Good engineers don't treat approach as A == B or even A like B, when extremely integral parts of their applications differ.

Zero-shot isn't just "more convenient", in a low data regime: it's the only workable solution, and 100x so if your plan involves the acornym "BERT" (because even the largest of those models has the world knowledge of a fart to draw priors from)

Better ergonomics while being faster and cheaper as the existing things really is enough to justify callling what you've done a new thing, in a world of finite resources and time. It's actually making me scream how many people don't get that.

> As an MLE

Yea but those models you were building were lame and inaccessible to play with for common devs.

Just because they have the same api doesnt mean you were building the same thing.

it doesnt generate text though, only probabilities