Anyone who has worked in ML for 10+ years would already know that the usage of LLMs for everything is lazy, wasteful and a high degree of marketing on it.

Anyone who has designed circuits will consider CPUs wasteful compared to ASICs. This new FPGA technology is just a less efficient ASIC.

That’s roughly what I’m hearing.

The fact that general purpose intelligent classifiers can be dynamically hacked together by an LLM in real time to allow them to build evolving labeled and understandable networks that perform substantially faster than the LLM, and can act as an intermediate sorting and organizing layer for caching context or handling simple tasks, and a complete layman like me can assemble a teachable layer of these in a few days from an inexpensive service…

That’s wild!

And then you can identify where an expert system needs a more specific ML technique for efficiency within this network that overlays the SOTA model. Or manually adjust the stored context in each secondary “neuron”. And paths forward can run programs or take actions at relative high speed.

And you can share these with others and improve them as a group.

You could insert this at the datacenters at scale with a local supervising expert to prune and encourage proper growth. You could identify specific gaps in capability that need more training, and patch over them temporarily.

Then you train those corrections back into the general purpose model, or you identify highly efficient subsystems for specific purposes.

And this is just one way to use it. High speed intelligent workflows can live in this. There’s a spot for a local LLM to learn on the fly.

Maybe I’m way off base, but for the non-experts Jev seems extremely valuable.

I would rate using LLM for tasks more specific ML can handle as a lot like using one's smartphone to snap photos, listen to music, set alarms, and play video games in preference to carrying around a fun cam, ipod, watch, and switch 2 everywhere.

For those who need to dive really deep into each specific avenue and squeeze maximal quality out, the photographers will be packing DSLRs and intense gamers will wait til they get home to strap into a PS5 or a gaming rig or VR or whatever.

But "can get 90% of anyone's needs met in this field, and can do the same in dozens or hundreds of other fields simultaneously" will remain the killer solution for anyone with lots needs that each have bounded depth.

I have, LLMs are less fragile, that’s why I like them. The ability to generalize isn’t just about being general purpose, it’s super robust, and so assuming the budget is there (I agree they are inefficient) end up performing better on many classical tasks that have ood inputs. Before LLMs / foundation models we all struggled with generalization and at least in the work I was doing people were independently converging to using bigger more general models for tasks anyway as compute got cheaper. LLMs are just the most popular version of this.

I thought one core result that led to LLMs was the realization that a specialized model is not necessarily better at a task than a general one.

I wouldn’t say lazy, LLMs are fast to use and much more cost effective especially if you factor the cost and time of training (data preparation, data cleaning, … etc).

It’s hard to justify several months to business when there is something off-shelf ready to use and doesn’t require domain specialists to run.

why would you waste your time messing around with a team of expensive ml engineers and data scientists that produce vastly inferior to a llm.

We ripped out custom homegrown ml models that were developed in last 10 yrs and put an llm in its place. Its the opposite of wasteful. Even local gemma models are vastly superior.