| ▲ | throwa356262 3 hours ago |
| "LLM backends: Anthropic, OpenAI, OpenRouter." And here I was hoping that this was local inference :) |
|
| ▲ | micw 2 hours ago | parent | next [-] |
| Sure. Why purchase a H200 if you can go with an ESP32 ^^ |
| |
| ▲ | sigmoid10 43 minutes ago | parent [-] | | Blowing more than 800kb on essentially an http api wrapper is actually kinda bad. The original Doom binary was 700kb and had vastly more complexity. This is in C after all, so by stripping out nonessential stuff and using the right compiler options, I'd expect something like this to come in under 100kb. | | |
| ▲ | pitched 33 minutes ago | parent | next [-] | | Doom had the benefit of an OS that included a lot of low-level bits like a net stack. This doesn’t! That 800kB includes everything it would need from an OS too. | | |
| ▲ | __tnm 22 minutes ago | parent [-] | | yah my back of the envelope math.. the “app logic”/wrapper pieces come out to about 25kb WiFi is 350
Tls is 120
and certs are 90! |
| |
| ▲ | __tnm 34 minutes ago | parent | prev [-] | | yeah i sandbagged the size just a little to start (small enough to fit on the c3, 888 picked for good luck & prosperity; I even have a build that pads to get 888 exactly), so i can now try reduce some of it as an exercise etc. but 100kb you’re not gonna see :) this has WiFi, tls, etc. doom didn’t need those |
|
|
|
| ▲ | 3 hours ago | parent | prev | next [-] |
| [deleted] |
|
| ▲ | __tnm 2 hours ago | parent | prev | next [-] |
| haha well I got something ridiculous coming soon for zclaw that will kinda work on board.. will require the S3 variant tho, needs a little more memory. Training it later today. |
|
| ▲ | peterisza 2 hours ago | parent | prev [-] |
| right, 888 kB would be impossible for local inference however, it is really not that impressive for just a client |
| |
| ▲ | Dylan16807 2 hours ago | parent [-] | | It's not completely impossible, depending on what your expectations are. That language model that was built out of redstone in minecraft had... looks like 5 million parameters. And it could do mostly coherent sentences. |
|
