Of all the debates raging about the potential downsides of artificial intelligence, one worry has been causing the most hand-wringing among AI enthusiasts in Silicon Valley: the fear that the giant AI labs selling proprietary models are acting like Trojan horses. The concern is that as startups and enterprises use AI models from labs like OpenAI and Anthropic, these labs gain ever-increasing access to their customers' most sensitive business information. The model makers can then use that knowledge for themselves, potentially becoming competitors to their own customers. This warning has been sounded by venture capitalists like Jason Calacanis and Palantir CEO Alex Karp. Now, in a surprising blog post published on Sunday, Microsoft CEO Satya Nadella has joined this crowd.

The double payment

Nadella warns that AI users—the "buyers" as he calls them—are paying twice. They knowingly spend money on AI token usage, but they also, obliviously, hand over valuable data in the process. "You essentially pay for intelligence twice, once with money, and again with something even more valuable: the proprietary knowledge you must reveal to make that intelligence useful. The better you want the model to perform, the more of that knowledge you have to feed it!" he writes. Most dangerously, enterprises are literally teaching the models about the nuances of their businesses, he argues. "Models learn from 'exhaust,' the prompts people write, the tools agents use, and especially the corrections people make when the model is wrong. Every correction is distilled into institutional know-how," he writes. This is "the kind of knowledge a competitor could never buy," and yet enterprises are handing it over.

Distillation and hypocrisy

Nadella argues that if AI companies get to freely scrape the internet to train their models, it is only fair that enterprises get to study—or "distill"—those models in return. Distillation is the practice of using a model's own outputs to learn how it works and to train a new, often cheaper, model based on those insights. In February, Anthropic accused Chinese open source models of sending millions of prompts to Claude as a way to improve their own models, and urged the U.S. government to crack down on export controls. Nadella's point is that model makers cannot have it both ways. It is hypocritical for them to freely train on the world's data while restricting others from doing the same to their models. "While the great innovation that comes from model providers having fair use rights to train models on public data is needed, I find it ironic that the status quo is to then turn around and impose restrictive terms on distillation," Nadella writes. He is particularly concerned when model makers "reserve the right to learn from customer usage and interaction data."

The solution: retain ownership

Nadella's solution is the kind of thing a CEO of a giant cloud provider would suggest. He wants companies to "retain ownership" of their data, including prompts, feedback, and corrections. So he urges them to build their own "proprietary learning environments" on the cloud—where their data is likely already stored anyway and, conveniently, could mean Microsoft's cloud, Azure. He also wants companies to build what he calls "orchestration layers"—essentially, a way to easily switch between AI models from different providers rather than being locked into one. Tools like AI gateways that let companies do exactly this have become increasingly popular. While Nadella never uses the words "open source" as the method for retaining ownership, this is an obvious subtext.

The shift to open source on-premises

Large companies, many of which still have some of their own data centers in addition to using the cloud, are already moving to open source models installed on their own premises—known as "on-prem" in industry jargon. Idit Levine, founder and CEO of Solo.io—which makes networking and security software that helps enterprises manage AI systems—says she is seeing exactly this shift play out with her own customers. After experimenting with proprietary model makers, they start asking themselves: "Can I take an open source model and run it on-prem? It will do almost 90% of what the big one's doing. It will cost way less," she tells TechCrunch. "They understand that, and they can control it." Solo.io's technology was selected last year to be the tech powering the Linux Foundation's Agentgateway project. Her company counts enterprises like T-Mobile, ADP, and SAP as customers. She sees companies increasingly installing on-premise open source models and sees it as the next big wave in enterprise AI use.

This trend is not isolated. Vercel, best known as a platform for building and hosting websites, has recently added AI model-switching tools. OpenRouter, a company that helps developers route requests across different AI models, is also seeing a surge in traffic to open source models. In fact, open models accounted for 29% of all traffic routed through Vercel's gateway last month. With the CEO of Microsoft—a company that has invested in both OpenAI and Anthropic—now openly urging enterprises to be wary of using proprietary models, this trend is likely to continue growing. "In consuming intelligence, you are creating intelligence. And what you create should belong to you," Nadella writes.

Nadella's warning comes at a time when companies are increasingly relying on AI to gain competitive advantages. The risk of handing over proprietary knowledge to model makers is not just theoretical. If a model maker learns the inner workings of a customer's business, they could use that knowledge to develop competing products or services. For example, a company that uses a proprietary AI model to optimize its supply chain could inadvertently teach the model maker about its unique logistics strategies. The model maker could then offer a competing logistics service using those insights. This is the Trojan horse scenario that Nadella and others fear.

The push for open source on-premises models is also driven by cost considerations. Proprietary models often charge per token, which can become expensive at scale. Open source models, while sometimes less capable, can be run on the company's own hardware at a fraction of the cost. Additionally, running models on-prem gives companies full control over their data and reduces reliance on third-party providers. This is particularly important for regulated industries like healthcare and finance, where data privacy is paramount.

However, the shift to open source on-prem models is not without challenges. Enterprises need skilled personnel to manage and fine-tune these models. They also need robust infrastructure to handle the computational demands. Tools like Solo.io's gateway and similar orchestration layers can help bridge the gap, allowing companies to run multiple models and switch between them as needed. Nadella's call for orchestration layers is essentially a recommendation to use such middleware to maintain flexibility.

Another aspect of Nadella's blog post is the call for fair use of distillation. He argues that if model makers can train on public data, then enterprises should be able to distill proprietary models to create their own models. This would level the playing field and prevent model makers from hoarding knowledge. The Chinese open source models accused by Anthropic of sending millions of prompts to Claude highlight the tension between model makers who want to protect their intellectual property and users who want to benefit from model outputs. Nadella's position is that the current system is one-sided and needs rebalancing.

The broader implications of Nadella's warning are significant. If enterprises follow his advice and move to open source on-prem models, it could reshape the AI landscape. The dominance of a few large model makers might diminish as companies build their own customized models. This would democratize AI and reduce the risk of vendor lock-in. However, it could also slow down innovation if enterprises focus on incremental improvements rather than breakthroughs. Microsoft itself has a vested interest in the cloud, so Nadella's recommendation to use proprietary learning environments on Azure is not without self-interest. Yet, many industry observers see the advice as sound, regardless of the platform.

In the end, the debate over proprietary versus open source AI models is far from settled. Nadella's contribution adds a powerful voice to the argument that enterprises should retain control over their data. As more companies experiment with AI, the lessons learned from Nadella's warning will likely influence their strategies. The shift to on-prem open source models may accelerate, driven by a combination of cost, control, and competitive concerns. Whether this leads to a more fragmented AI ecosystem or a more secure one remains to be seen. What is clear is that the issue of data ownership in the age of AI is now front and center.

Source:TechCrunch News