Researchers received training A new type of large language model (LLM) that uses GPUs that embellish and provide private and public data in the world, a move that suggests that the main ways to build artificial intelligence may be disrupted.
Flower AI and Vana, two companies, pursue an unconventional AI approach, have jointly created a new model called Collective-1.
The technology created by flowers can allow training to spread to hundreds of computers connected over the Internet. The company's technology has been used by some companies without the need to aggregate resources or data without the need to aggregate resources. Vana provides data sources, including private messages from X, Reddit and Telegram.
By modern standards, Collective 1 is small, with a value of 7 billion parameters (combined to give the model the capability) associated with today's most advanced models, such as hundreds of billions of dollars, such as those of dynamic programs such as Chatgpt, Claude and Gemini.
Nic Lane, a computer scientist at the University of Cambridge and co-founder of Flower AI, said the distributed approach is expected to surpass the scale of Collective 1. Ryan added that Flower AI is training through a model with 30 billion parameters using conventional data and plans to train another model with 100 billion parameters (which limits the size offered by industry leaders this year). "It really can change how everyone thinks about AI, so we're working on it," Ryan said. The startup also incorporates images and audio into training to create multimodals, he said.
Distributed model building may also disturb the driving force that shapes the AI industry.
AI companies are now combining a large amount of training data with a large number of computers that are centralized within data centers that are stuffed with advanced GPUs that connect them together using ultra-fast fiber optic cables. They also rely heavily on datasets created by scratching public access (although sometimes copyrighted) datasets, including websites and books.
This approach means that only the wealthiest companies and countries with a large number of the strongest chips can viablely develop the most powerful and valuable models. Even open source models such as Llama and R1 from DeepSeek's Meta are built by companies that have access to large data centers. A distributed approach allows smaller companies and universities to build advanced AI by putting different resources together. Alternatively, it could allow countries lacking conventional infrastructure to build several data centers to build a more powerful model.
Lane believes that the AI industry will increasingly consider new ways to allow training to detach from individual data centers. He said the distributed approach “allows you to compute computations more gracefully than data center models.”
Helen Toner, an expert at AI governance at the Center for Security and Emerging Technologies, said Flower AI’s approach is “interesting and potentially relevant” to AI competition and governance. "This may continue to work hard to keep pace with the border, but it could be a fun and fast traveler," Toner said.
Distributed AI training involves rethinking the allocation of computational methods used to build powerful AI systems. Creating an LLM involves feeding a large amount of text into a model that adjusts its parameters to produce a useful response to the prompt. The training process inside the data center is divided so that parts can be run on different GPUs and then periodically merged into a single master model.
The new approach allows work done inside large data centers to be performed normally on hardware that can be miles away and connected via a relatively slow or variable Internet connection.