I have been thinking of something on these lines but with much smaller models. The entire model has to fit on a single computer. Host owner would choose the model they prefer, perhaps because they already use it. Then it is more about utilizing the GPU for LLM requests.

Peer to peer, consumers get to route their request to a host with compatible model. Consumers have to also contribute GPU but it does not have to be equal - I have not thought through the fairness part. Perhaps initially it starts with "create your friends group and have access to all the host nodes".

▲

inigyou 2 hours ago | parent [-]

Can't wait to run a Sybil farm of subsidized proxy nodes and log all the juicy prompts.

▲

woctordho 2 hours ago | parent | next [-]

AI Horde has some measures to prevent Sybil attack that returns wrong results, but not enforce zero data retention. Prompts belong to the whole open source community. For example https://huggingface.co/datasets/la-ji/sd-prompt-in-the-wild

▲

brainless 2 hours ago | parent | prev [-]

I do not know what/who Sybil is but yes, there is a lot of plumbing in order to make sure that host nodes are ZDR* compliant and much more. Zero visibility of source prompt.

Also, this is the reason I want to start with trust based groups only - invite people you already know. I have friends who have Macs with 32GB or 48GB unified memory but sending prompts to them is not a out of the box thing.

* typo

	▲	fragmede an hour ago \| parent \| next [-]
		> A Sybil attack is a type of attack on a computer network service in which an attacker subverts the service's reputation system by creating a large number of pseudonymous identities and uses them to gain a disproportionately large influence. https://en.wikipedia.org/wiki/Sybil_attack
	▲	mcpindex-ai 2 hours ago \| parent \| prev [-]
		[flagged]