Most large AI models today run on graphics processing units. These chips were originally built to render video games, but their ability to perform many calculations at once happens to suit machine learning. One company has dominated the market, and demand for its hardware has outpaced supply. That situation has attracted a wave of startups trying to build alternatives, and investors have been willing to write very large cheques.
What the startups are building
The chips go by different names, but the idea is similar: design hardware specifically for AI work rather than adapting general-purpose designs. Approaches include:
– **Custom accelerators** that focus on the matrix maths at the core of neural networks. – **Wafer-scale or very large chips** that keep more data on-chip to reduce delays. – **Memory-centric designs** that reduce the time spent moving data around. – **Inference-focused chips** optimised for running trained models quickly and cheaply, rather than training them. – **Dataflow architectures** that change how computation is organised to improve efficiency.
Some sell hardware directly. Others offer cloud access to their chips through a hosted service so that customers can try them without buying systems.
Why the market is so tempting
AI spending is enormous, and a large share goes to hardware. Customers want lower costs, better supply and less dependence on a single vendor. Inference, in particular, is growing as more people use AI services every day, and it is a place where specialised designs might beat general ones on cost per query or speed.
Why it is so hard
Building a chip is only the first step.
**Software matters as much as silicon.** The dominant vendor has a mature software ecosystem that developers know and trust, with libraries, tools and years of optimisation. A new chip must work with popular frameworks and make it easy to port existing code. If developers have to rewrite their models, adoption slows.
**Manufacturing is complex.** Startups rely on a small number of advanced factories, which have limited capacity and give priority to large customers. Chips take years to design, so market needs may shift before the product ships.
**Rapidly moving targets.** Model architectures change. A chip optimised for last year’s design may be less useful if new methods emerge.
**Capital intensity.** Design, testing and production cost hundreds of millions of dollars. Funding rounds of a billion dollars are not unusual in this field.
**Customer concentration.** A handful of large buyers account for much of the demand, and they often design their own chips too.
How buyers should evaluate claims
Impressive demos can be misleading. Questions to ask include:
1. **What workload was measured?** Performance on one model may not carry over to others. 2. **What is the baseline?** Compare against current alternatives, not older hardware. 3. **How is cost calculated?** Look at total cost of ownership, including power, cooling, networking and staff time. 4. **How mature is the software?** Ask about supported frameworks, debugging tools and documentation. 5. **What is the supply plan?** Check delivery timelines and the vendor’s manufacturing arrangements. 6. **Who else uses it?** Reference customers in production are more convincing than pilot projects. 7. **What is the company’s financial position?** Hardware customers rely on long-term support.
The energy angle
Power consumption is now central. Data centres are limited by how much electricity they can get and how much heat they can remove. Chips that deliver more work per watt can be attractive even if raw speed is similar. Some startups position themselves around efficiency, an argument that resonates as energy costs and grid constraints become more visible.
Possible outcomes
For any given startup, the likely paths include: carving out a niche, such as low-latency inference; being acquired by a larger company seeking technology or talent; partnering with cloud providers; or struggling to gain traction and running out of funding. A few may become significant players. The market is large enough for more than one winner, but not for every contender.
Final thoughts
Large funding rounds show that investors see an opportunity, not that success is assured. The real tests are software compatibility, reliable production and customers who run important workloads on the hardware. Readers following the sector should watch deployments and independent benchmarks more than announcements. Competition in AI hardware is healthy, and it could eventually lower costs and widen choices for everyone.
