ZML launches free inference server that runs AI models across Nvidia, AMD, Google TPU, and more

French AI startup ZML, backed by Turing Award winner Yann LeCun, has released a free inference server called ZML/LLMD that allows open-source large language models to run at peak performance across a wide range of chips — including Nvidia, AMD, Google’s TPU, Apple Metal, and Intel Arc. The move aims to break vendor lock-in and give enterprises more flexibility in choosing hardware for AI workloads.

Breaking the silos of AI inference

Inference — the process of running a trained AI model to generate responses — has become a critical bottleneck as AI adoption scales. ZML founder Steeve Morin told Bitcoin World that optimizing inference now often matters more than training, yet software and architecture barriers have locked many organizations into single-vendor ecosystems. ZML/LLMD is designed to change that by delivering maximum speed across different chips, sometimes outperforming native implementations.

Why this matters for the AI chip market

The launch comes amid what industry observers call the ‘inference gold rush,’ with companies racing to reduce AI costs and energy consumption. By supporting a diverse set of hardware, ZML could help enterprises mix and match chips based on cost, performance, or energy efficiency. Morin noted that the software may particularly benefit European AI chip startups such as Axelera, Fractile, Kalray, and others, though he emphasized that ZML’s focus is on enabling innovation globally, not just regionally.

Funding and team

ZML has raised $20 million from prominent venture firms including 20VC, LocalGlobe, and Xavier Niel’s Kima Ventures, as well as angel investors like Hugging Face co-founders Clément Delangue and Julien Chaumond, and Docker founder Solomon Hykes. The lean team of 20 people, based in Paris, credits its speed to being well-funded and focused. Morin, formerly VP of engineering at Zenly (acquired by Snapchat for nine figures), said he could not have built ZML anywhere else.

Free product, future revenue model

ZML/LLMD is launching as a free product, with Morin stating the company wants to learn about usage patterns before deciding how to monetize. ‘I’d rather measure and generate revenue where it is most effective without hindering my growth stupidly because I have been too greedy from the get-go,’ he said. The startup’s first public project, an inference-focused ML framework released in 2024, remains open source.

Competition and positioning

ZML enters a competitive field that includes Baseten (recently valued at $13 billion), Inferact (from the creators of vLLM), and RadixArk (behind SGLang). However, Morin sees ZML’s broader ambition — including co-designing silicon — as a differentiator. The startup has already reached a point where it is working with chipmakers on ‘things that haven’t been done before anywhere in the world,’ he said.

Conclusion

ZML’s free inference server represents a significant step toward a more open and flexible AI hardware ecosystem. By enabling peak performance across multiple chip architectures, the startup is addressing a real pain point for enterprises scaling AI. While it remains early to gauge adoption, the backing from top investors and industry figures signals that Europe’s AI ecosystem is maturing — and that inference software may become a key battleground in the AI industry.

FAQs

Q1: What is ZML/LLMD?
ZML/LLMD is a free inference server that allows open-source large language models to run at maximum speed across a variety of AI chips, including Nvidia, AMD, Google TPU, Apple Metal, and Intel Arc.

Q2: Who is behind ZML?
ZML was founded by Steeve Morin, former VP of engineering at Zenly, and is backed by investors including 20VC, LocalGlobe, and Kima Ventures, as well as Turing Award winner Yann LeCun and Hugging Face co-founders.

Q3: Is ZML/LLMD open source?
No, ZML/LLMD is not open source, but it is launching as a free product. The company plans to learn from usage before deciding on a future revenue model.