In an effort to give its employees better access to GPU resources for inference tasks, Apple has rounded up its Minions, not the animated yellow creatures, but a distributed system that puts idle GPUs across its Macs to work on AI workloads.

Apple researchers describe it as "a dynamic and distributed virtual GPU cloud that aggregates idle Apple Silicon GPUs across macOS devices to execute large-scale machine learning inference workloads."

Apple's researchers said the company has used Minions to make GPU compute more readily available to its teams.

"Prior to its adoption, some inference jobs experienced queue times exceeding ten hours due to limited centralized GPU availability," the paper on Minions explains. "With Minions, tasks can often begin execution within minutes by leveraging idle macOS devices."

Apple's recently released Mac mini and Mac Studio support clustering using RDMA (Remote Direct Memory Access) over Thunderbolt 5, allowing compute and memory to be combined to support a single AI workload. This allows, for example, loading a model too large for a single machine to accommodate.

Minions is designed to create a virtual GPU cloud from connected Mac hardware to create a fault-tolerant pool of inference workers in a shared distributed inference platform. So instead of sitting idle, GPU resources can be used more readily by teams on the network. It's not necessarily intended as a way to run large models as much as it is to redistribute idle resources on the network.

Minions has four primary components: Minion Server, PostgreSQL Metadata Store, Minion Executor, and Minion Watchdog.

The Minion Server provides the coordination layer, receiving user requests and making them available as subtasks. The Minion Executor runs inference tasks on a Mac's GPU, uploads the results to S3, and reports execution status through the Minion Server, which maintains task metadata in PostgreSQL. And the Minion Watchdog monitors this process for failures – like one of the Macs in the network not responding – and re-assigns tasks if needed.

Database stats get fed to Grafana-based dashboards for real-time visibility.

Apple's approach, which focuses on hardware availability and utilization, is not the same as an Nvidia cluster, which focuses on throughput, latency, and efficient GPU utilization for large models. ®