Minions is an internal enterprise system already in use at Apple. It pools idle Mac GPUs into a shared compute cluster for ML inference.

There is no distributed execution of a single LLM across multiple GPUs using tensor parallelism here. Instead, it relies on task-level parallelism-breaking down a large workload into independent subtasks that run across separate machines. For example, generating embeddings for a million documents.

This pattern has been implemented across hundreds if not thousands of different organizations (and between organizations). I have a patent on data distribution strategies from my time at Google running a large (>1M cores 24/7 for 3 years) idle cycle harvesting project.

it's a good idea but you often end up with huge tail latencies, slight incompatibilities due to fleet heterogeneity, and a database/scheduler management problem.

My university ran its render farm across idle workstations using the screensaver back in early 2000. Same concept.

So basically works the same way as Folding@Home?