I just gave Fable 5.2 a shot at placing and routing a couple of circuit boards. I'm actually sure it would work fine for simpler/sparser designs but when the component density gets high enough, Fable spends too long struggling in the layer of the layout it's in rather than doing what a human would do, and intuiting that placement, or a component footprint, or different part choice altogether would enable the work to continue.

At first the work looked awesome, and once I started checking it out, I realized I would have to significantly alter the placements and manually draw most of the schematic to get it up to my standards (and hit the performance criteria of the board).

Do they say how they interact with Blender? Launched within a python interpreter everything in Blender is accessible as a method call. I hope they didn’t do anything more complex than that.

I think the major difference is they have a set of profile data they use to manage the placement of expert weights.

"Hot" commonly used experts are kept on the GPU, and "cold" less used experts are bumped to system ram and calculated on the CPU. So it can run larger than vram sized models without completely crippling performance.

I'm not exactly sure what the changes they made to the architecture are for, perhaps they're aimed at extending the difference between the "hot" and "cold" expert usage?

And HIP support (amd) broken.

I'm not saying it's not, but if this is the case, how do we think it beats 3.8FN in benchmarks?

100% my exact impression of this project as well

Similar to a Linux distribution then?

Their edge0-strata engine is just a fork of the strata project, which has been doing this for a long time: https://github.com/Niko1221/Strata

They're not hiding that fact once you get to the repo. I mean it's called edge0-strata and the README says it's a fork of strata.

It's weird to see this marketing web page imply that it was all new and novel work that they did, though.

I am not in the material, so perhaps someone else can comment on the quality of the outcome and technique? After skimming a page I am not really sure that the whole model fits inside 29 GB RAM. They seem to use some interesting techniques.

ModelTokenMemoryUsage [Bytes] = Parameters * (BitsPerParameter/8)

For a model with 145B parameters, this means:

    Precision       GB       GiB
    2-bit         36.25     33.76
    3-bit         54.38     50.64
    4-bit         72.50     67.52
    6-bit        108.75    101.28
    8-bit        145.00    135.04
    16-bit       290.00    270.08
    32-bit       580.00    540.17

It seems that they radically reduce the model information required per token via the sparse expert approach - 145B->10B parameters, probably at lower precision - and then use some other approaches I do not understand at that point to make effective use of loaded information.

Did anyone try the model?

The blog post says:

> For Edge0-145B, day-one support covers macOS, Windows, and Linux, including Apple M-series chips and NVIDIA platforms such as RTX Spark, RTX 4090, and RTX 5090. The release includes one-click deployment tools and hardware guides.

But... there's no guidance on how to run on a mac at all. Weird.

Don't forget the memory requirements for context.

> 145B parameters with only peak 29GB memory usage - really impressive

They're doing offloading, the full model doesn't live in VRAM at all. They're expecting you to do ~30GB VRAM + the rest on system RAM.

I assume so - Qwen/Qwen3.8-Flash-Next is marked as the "Base Model" on the huggingface page and states it may be subject to "Qwen's separate authorization or attribution requirements"

And I assume the metrics comparing qwen are using a quantized version - from the blog they compare memory use against qwen's "standard 4-bit deployment". Which might explain the improved performance over that base model they show - it might lose less from quantization.

https://huggingface.co/Edge0/Edge0-145B-NVFP4

> Using Strata + Qwen 3.8 Flash Next now and I am beyond impressed.

The inference engine they use is called edge0-strata because it's a strata fork.

This project looks like someone fine-tuned Qwen3.8 Flash Next, made some tweaks to the model, forked strata to run it, and then put a lot of marketing and branding on top of it

3.8F/N is amazing. Truly useful, runnable local AI on machines that are not ridiculously expensive. (I’m running it on 2x 3090 and 128gb of ram, on the quantum roughly equivalent to 5-K-M. fast, but you can run it on much less

This looks like it would actually work on my 64mb M1 MBP. That would be fantastic.

They literally talked about MacBooks in the blog post, and they did release more than just NVFP4: https://huggingface.co/Edge0/Edge0-145B-q3k

It is strange that they also benchmarked an M2 Ultra, but they are specifically not releasing a Mac runtime today. Maybe that is still under development.

My dev machine is an off the shelf Acer aspire laptop that I bought used, it comes with a 4GB RAM, I also added 4GB of swap memory. I use it to develop fullstack applications for servers, browsers and android. My stack is Rust, Javascript, Dart/Flutter, Postgresql and running Debian 13.

It works great, I mainly use 2 applications , Firefox and urxvt, using emacs -nw to write code, no LSP or code completion because they use too much RAM. I also use i3wm which very lightweight.

The only issue with it is that some keys on the keyboard don't work, so I mapped them to different keys, for example ctrl is mapped to caps lock, 0 is mapped to 3 on the numpad, enter is mapped to right shift etc. Because of this I can't type on other peoples computers, I would need to run my xmodmap script before I do that.

no.

It was SO HARD to convince my sister to spend that 100€ more for 16 GB of ram.

My personal laptop is a fairly decent one, still with 16GB of soldered ram.

Just got approved a new laptop for work. 4k for 64GB of RAM and high end CPU. 1.5y ago that would have been 2.5k, only macs would have been that expensive

I think the trajectory is more important than the snapshot in time. A year or two ago if you wanted to run things locally you were either running toy models, or somehow had a super-computing cluster. Now it's just high end 'normal' hardware. At current trends, we'll be running frontiers on a plain old computer within a few years.

All the snapshot does is set a pricepoint. Most 'normal' people I've talked to about LLMs have expressed some concerns about privacy/control from LLM cloud companies. So this is the cost it takes to opt out - actually substantially less since a 4090 is also enough to run Qwen 3.8. MSRP there is $1600. Retail price is a bit higher but trending down quickly. Once this bubble pops, this sort of hardware will probably be available on the cheap.

Anyhow, so yeah - a 'normal' person isn't going to have a $15k laptop or a $1600 video card (that retails upwards of $2k realistically) because they had no reason to. Now they do. Of course in a year or two, those prices will likely have plummeted precipitously, so perhaps they still have no reason unless they're just particularly impatient.

I allways use my own devices. Most universities allow to run/use your own devices. I run a m2 with 96gb.

> do normal people (so not HN professional developers or entrepreneurs) have that, as actually personal device (not from their tech company)?

I guess I'm far from the norm, as I have a Macbook Pro privately. I personally don't have 30+ GB of ram, though I'm sure the next one will have that.

However, I doubt that 'normal people' are even the target market for this. 'Normal people' don't care about their models running in the cloud. They want their voice assistant to be somewhat useful when there's no internet and we don't need 145B models for that.

So I think that I and people like me, often caller prosumers, are the target for these type of things. Some people buy €5k camera lenses for their hobby, others buy €5k laptops.

I think that only for showoff.

Those weighting value/price for the use case they are having the Pro is an overkill for most in the population. I have a good but still medium performance laptop personally as I use it for learning experimentation (and fun, which is not independent from the two former), but I did not push such for my wife. For her even the entry level Mac is an overkill. Be both recognise that. For those thinking like us the MacBook Neo was a good move from Apple.

Said that, this was before personal AI assistants. Now, for privacy reasons, I might push for a more capable laptop for her too (if I was sure I could set up a reliable config locally. I did not dig into that enough to make sure, but I'd do if it was an issue today).

> At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications

Been reading this for years now. Seems both, bigger models optimized for scale AND smaller models optimized for single machines, keep on being released. I don't think it's changing, different markets.

Do tell more! I haven’t updated my harness in a while and ideas are welcome.

There an irony to your statement...since this model seems to be quantized version of Qwen Flash Next + some sort of post quantize training.

https://chatpanel.net/blog/edge0-35b-macos-metal i have it working but not the Edge0-145B. just the 35B

Isn't this pretty much backward? If the thing spit out codes so fast, you'd have to babysit it. Slower throughput let you do something else while you wait.

Also throughput pretty much depends on compute capabilities and active weight. So if someone build a A0.1B then maybe you can look at 1500t/s of pure uselessness.

It is not open weight either. The license is too restrictive to meet the standard definition of that team.

I'd guess the normal filesystem caching should automatically get you the benefits you describe.

So did Strata Vs Llama.cpp, yet Strata pulls 2.5 to 3x the performance of the Llama.cpp codebase it was based on. If they've improved the architecture further by sharing experts, then this might be plausible.

Just ask Fable 5.2 for a version that works on Mac. Magically happens... There are just a few tweaks and voila! Getting about 50 tok/s on an m5 max at nvfp4.

To fit in my 48GB MacBook eventually (once they release the Mac version). 145B seems to be too big for that.