Personally I think Apple should have acquired them. if you could burn a gemma4 class model into an iphone and actually get extremely low latency and low battery usage it would feel like the future IMO. even if it means you wont get frontier intelligence, there might actually be incentive to buy a new mobile device every year again.

▲

Melatonic 37 minutes ago | parent | next [-]

The Taalas chips are not physically small. And part of their secret (if you look at the design) is just locating a bunch of memory soldered on the edges ( I belive higher amounts of SRAM ? )

▲

adgjlsfhk1 an hour ago | parent | prev | next [-]

I don't think this works out from a cost/silicon perspective. Small models already run pretty well in software (since the weights fit in cache) and big models require silicon area proportional to the size of weights. On a mobile device putting a chip like this is competing directly in BOM and power against a whole lot more l3 cache, and the l3 cache makes everything faster

▲

bastawhiz 32 minutes ago | parent | next [-]

The weights might fit in cache, if you're using a small model. If you wanted to have a 20B+ parameter model, that's just going in RAM. You could put more RAM in the device and pay the perf cost or have a dedicated chip. Most devices already have a dedicated chip, this just changes which silicon you're spending the money on.

▲

teaearlgraycold 35 minutes ago | parent | prev [-]

My question is what changes about LLM use cases when you’re getting 1000 tok/s? Models in silicon might dramatically change how we think about them.

	▲	RussianCow 27 minutes ago \| parent [-]
		That likely isn't as relevant for on-device iPhone usage as it is for Real Work™. I won't notice the difference between 50tps and 1000tps when asking Siri a question.

▲

bsaul an hour ago | parent | prev | next [-]

That's actually a really good point... There's currently zero incentive to buying more hardware, and that's one very good reason do have a new one.

▲

superb_dev an hour ago | parent | prev | next [-]

From what I remember, these chips are not mobile size yet

▲

bradfa an hour ago | parent [-]

A small model would be. I think that’s more the point. It’s definitely not SOTA but it’s fast and energy efficient and local.

▲

mdp2021 44 minutes ago | parent | next [-]

> A small model would be [mobile size]

A ~30mm side for the HC1 tech for an 8b model (still unclear the planned HC2)?

	▲	teaearlgraycold 33 minutes ago \| parent [-]
		Is that analogue or are they baking floating points into the silicon?

▲

wmf 40 minutes ago | parent | prev [-]

Nope, a small model would be larger than the whole iPhone SoC.

▲

whatsThisBtn4 37 minutes ago | parent | prev [-]

Apple is somewhere between fashion company and second rate tech company.

They could have 9 year old AI and still post profits.

Not sure if it's my pixel or android, but I made a randos jaw drop with what the crappy AI on android can do.

When are we getting android OpenClaw?