Very curious what hardware you're running this on!

The same 24GB VRAM RTX 4090 I bought to play Cyberpunk 2077 with.

Works perfectly fine in llama.cpp throwing 70+t/s at me with 128k q8 K/V context when using the IQ4_NL quant + MTP at q4 MTP K/V.

Also leaving this here because you might find it useful: https://hypfer.github.io/will-it-fit-llama-cpp/

▲

indoordin0saur 3 hours ago | parent | next [-]

Nice! Do you do anything with that compute when you're not actively using it? Is the crypto-mining hobby still worth it? I've also wondered if such expensive hardware can be rented back out to offset cost. Looks like these cards are going for as much as $4k nowadays.

▲

all2 3 hours ago | parent | next [-]

There are services where you can hook your card up and rent it out to other users. I don't know what any of them are called, but they do exist.

	▲	dghlsakjg 3 hours ago \| parent [-]
		Salad.com is one. (I’m unaffiliated, just happened to come across it this week while looking for a cheap option)

▲

hypfer 3 hours ago | parent | prev | next [-]

I've paid ~2k€ in 2023. Since I'm usually sitting next to it, I'm only using it when I want to use it. It can get quite loud and warm.

Crypto (to my knowledge at least) moved away from GPU mining. I guess you could maybe rent out GPU compute, but - being in germany - it's not worth the legal hassle. You could of course always commit tax fraud, though I wouldn't recommend that.

▲

esseph 3 hours ago | parent | prev [-]

> I've also wondered if such expensive hardware can be rented back out to offset cost.

Massive legal liability. Not worth it.

▲

cdelsolar 3 hours ago | parent | prev [-]

What did you call me?