nvfp4 mlx, literally barebones pi.

edit: on bigger tests, got it to loop pretty easily unfortunately, probably local settings.

embedding-shape 8 minutes ago | parent | next [-]

> edit: on bigger tests, got it to loop pretty easily unfortunately, probably local settings.

Been playing around for a few hours with the poolside/Laguna-S-2.1-NVFP4 + poolside/Laguna-S-2.1-DFlash-NVFP4 + vLLM, been seeing the same behaviour. Usually new model releases are plagued with issues at release though, best to wait 1-2 weeks then retry, or better yet, investigate yourself :) Personally I haven't found any obvious issues.

▲

sosodev an hour ago | parent | prev [-]

What inference server are you using? They have a custom branch for llama.cpp, but I wouldn't be surprised at all if it still needs fixing.

	▲	Lwerewolf 39 minutes ago \| parent [-]
		This: https://github.com/Blaizzy/mlx-lm/tree/pc/add-lg ...and this is what I should probably wait for (not sure why it's in vlm): https://github.com/Blaizzy/mlx-vlm/tree/pc/laguna-s-nvfp4 ...or perhaps I should've just used the gguf with the provided llama.cpp instead of trying to run the nvfp4-mlx from the get go, but where's the chaos in that :) Running deepseek flash on something locally now, this will have to wait a bit. I still stand by my initial quick assessment - looks capable. Some people on r/localllama also reported loops. We'll see in ~10 hours. Hopefully I haven't terribly mislead people.