That led me to a new question:
What if we stopped thinking about one computer?
This is the idea behind Lumabri.
Instead of requiring a single machine to store and run an entire huge model, Lumabri treats a network of normal computers as a shared pool of resources.
One machine might provide disk space, another compute, another a different part of the model. If a required block or expert isn’t available locally, the system can retrieve or execute it on a peer.
This is particularly interesting for Mixture-of-Experts models. A model can have hundreds of billions of parameters, while only a fraction are activated for each token. Rather than moving huge expert weights over the network, Lumabri can send the small activation to a peer that already has the expert and let it execute it.
The goal is for machines to contribute whatever resources they can afford while using the swarm for the rest.
The idea is very much inspired by peer-to-peer systems: users are the infrastructure.
There are obviously major challenges, especially network latency and security. I’m experimenting with peer verification, SHA-256 verification, signed model state, replica selection, failover, and deterministic execution.
Lumabri is still an early experiment. I don’t have a datacenter or a huge GPU cluster, so I’m building it with the hardware I have and trying to find out whether the idea actually makes sense.
With Colibrì I asked:
Can one normal computer run a huge LLM?
With Lumabri I’m asking:
What if many normal computers could become one huge computer?
Feedback welcome.