Probably! But not viable yet; the chips would be about a year behind SOTA. Note the ~16 months that the article quotes as being insanely fast to get this chip to tape-out (read: start producing). We'll have to bootstrap our way there: AI is actively being used to get us closer to viable lead times for this.

Unfortunately, there's some real physical constraints: IIRC, manufacturing a wafer takes on the order of a month, start to finish, for the physical processing.

Maybe once LLM improvements asymptote further?

▲

kurthr 4 hours ago | parent | next [-]

The metal masked ROM is basically only 2 metal/contact layers. It's not a full new design and tapeout. You could roll a new set of parameters every ~2-3months. It's not an architectural change. See statements below.

https://www.eetimes.com/taalas-specializes-to-extremes-for-e...

https://www.turingpost.com/p/taalas

https://cambrian-ai.com/taalas-launches-hardcore-chip-with-i...

Part of the key is that by moving even from 6nm to 3-4nm one could embed a 20-30B model as part of a MoE (or only a subset of activated layers) on a single reticle die (note B300s are already multi-reticle), with a separate predictive/dispatch model controlling them each on a separate chip. This is without even stacking CiM ROM die. Moving the layer activations (and KV cache etc) between die requires relatively high speeds (and low latency), but distributed with multiple die in parallel might well be doable even with standard multilane PCIe. Of course KV cache prefill could also be handled by external GPUs. I'm sure AMD will make some reasonable choices.

▲

vineyardmike 5 hours ago | parent | prev | next [-]

How much of that 16mo is design versus just production? If there was a “plug and play” chip where you just BYO weights, how long would it take?

The bigger issue seems to be that these chips can’t hold that many weights at the moment.

(I’m curious if chips with large weights in them would be more tolerant or less to yield issues. If you flip a few bits in the weights, does it really matter at scale?)

▲

RealityVoid 4 hours ago | parent [-]

Talaas, from what I understand is building stuff just like that. The infra is the same and the weights layer is all you need to change. I guess you could half etch the chips and then finish them with the weights only. I think their turnaround is 6-8 Weeks. The size of the models fitting on the chips at the moment is llama 3 I think?

	▲	derefr 3 hours ago \| parent [-]
		> I guess you could half etch the chips and then finish them with the weights only. Basically a https://en.wikipedia.org/wiki/Gate_array. (The non-field-programmable kind.)

▲

kushie 6 hours ago | parent | prev | next [-]

tapeout could shrink but days per mask layer (DPML) does not have much margin..

	▲	smallmancontrov a minute ago \| parent [-]
		I'm not in industry, is DPML (which I assume is the time required to make a mask?) set by electron beam scan time or something?

▲

thoughtbefore an hour ago | parent | prev | next [-]

It may not matter. Think about why SOTA model companies are exploring chips. What do chips offer?

If SOTA models haven’t peaked, then the SOTA model companies would still be churning out better and better intelligence.

▲

jeremyjh 4 hours ago | parent | prev [-]

I think Sol is already good enough though.

▲

basilgohar an hour ago | parent [-]

"640k (token context) should be enough for anyone."

	▲	jerf an hour ago \| parent [-]
		I know what you're saying, but modulo things like losing track of what year it is as time passes by, a current frontier model is going to continue to be useful for many tasks for many years, even moreso if it's 5-10x faster due to the chip architecture. It's not that it would be the best forever, it's that it would be useful for plenty long enough to be worthwhile, even if there was better stuff available. In exactly the same way that this computer I'm typing this message on is not the latest and hottest cutting edge stuff. A 7 year old CPU, 7 year old Intel integrated graphics, an older NVMe disk, a mere 32GB of RAM... ok, that's one spec that's still pretty modern although it is slower RAM... but it's still plenty fast enough to comment on HN, even these seven years after it was cutting edge.