For personal needs I use a local Qwen3.8-Next-Flash setup on a GB10 cluster. For work, Github Copilot with either GPT 5 mini or toss up between Opus/Sol depending on the complexity of the task.

Used to pay for a Claude 20x plan and did everything in Opus, but I hate how it talks now and recent events (OAI scooping, Anthropic's spying, third party Chinese model hosts stealing and selling credentials) have really pushed me towards local AI for personal needs. Am not allowed to use Chinese models for work even if self-hosted so not much choice there.

▲

oidar a day ago | parent [-]

> on a GB10 cluster

I'd love to hear more about your setup. I have a single GB10 and am thinking about adding an additional one.

▲

hgoel a day ago | parent [-]

I'm coming up on a month with my 2x GB10 cluster. I was at 1x for a week before I pulled the trigger for the second.

Initial setup was a tad annoying because I had to update their firmwares and then power cycle them to get the 200GbE link working at full speed. After setting that up, it has been pretty smooth. I don't directly deal with the cluster, usually I just have the LLM itself handle updates/stopping to load different models.

Generation speed and TTFT is decent with Qwen3.8-flash, and it does a good job for my fiddling around with enough concurrency for multiple sessions/subagents. GLM 5.3-flash was also nice, but not too much better for how much slower it is.

I should also add that I already maintain a homelab with a couple of computers, VMs etc, so I am probably somewhat more tolerant of the occasional issue and fine with manually managing stuff over SSH. I think this is just a tradeoff of self-hosting relatively recent tech though.

I have a triple 3090 rig, but it mostly stays powered off because of the massive power draw and cooling requirements. The Sparks are slower but at peak they consume as much power as my 3090 machine at idle.

The recent talk of regulation has me wanting to pick up 2 more Sparks, but that's mostly to have the capacity to play with multiple models, local model tuning and to be ahead in case they force some limits/registration requirements for buying new hardware (kind of like the attempts to regulate 3d printers).

▲

tmikaeld 20 hours ago | parent [-]

How much does it cost per month, which provider and what do you get out of it?

	▲	hgoel 12 hours ago \| parent [-]
		Hmm? I'm running the models locally... 2x Sparks consume ~150W at peak, and they usually spend more time waiting on results from whatever task they're working on, so I imagine that the contribution to my electricity bill is maybe a dollar/mo or less. Though, of course, each Spark was $4000, so the total I've spent is equivalent to several years of the maximum tier for most cloud model susbcriptions. What I get out of it is the ability to hand login credentials to my other computers to manage their updates, bug fixes etc. Eg. After updating my proxmox server, the nvme drive kept dying. Was able to let my local AI in to figure out and fix what was wrong (known issue). A cloud-based AI could've done it too, but I don't want to be sending internal passwords out of my network like that. Plus, the ability to freely delegate tasks or exploration of things cloud models generally avoid. For example, I draw as a hobby, and when I'm struggling with a pose but can't quite figure out what I'm missing, I pass it into a VLM for advice, but Claude etc get unnecessarily cautious because they interpret an anatomical sketch as a naked person.