August 15, 2026
For the past two years, the standard line has been that GPUs -- built for the tidy, parallel math of training -- are a clumsy fit for agentic AI, which runs in messy, sequential steps: think, call a tool, wait, think again. French startup Kog is challenging that assumption, according to TechCrunch, arguing the real waste isn't in the silicon but in how poorly today's software uses it. Kog's pitch is that with the right lower-level engineering, existing GPUs can push out meaningfully more inference per dollar without anyone buying new hardware.
If Kog is right, that's a bigger deal than it sounds. The industry narrative around agentic AI has largely assumed a hardware answer -- more chips, more specialized silicon, more capital expenditure -- which is part of why compute spending has become such a governance headache for the companies footing the bill. A software-level fix that squeezes more inference out of GPUs already sitting in data centers doesn't just save money for the Kogs and hyperscalers of the world -- it changes the math for every business trying to decide whether agentic workflows are affordable to run at scale. I'd put a caveat on the enthusiasm, though: startups claiming to beat the efficiency of Nvidia's own stack have said this before, and the gap between a promising benchmark and a production-grade result is where a lot of these claims quietly die. Worth watching, not worth betting the roadmap on yet.
Writer took a more direct route to the same destination: instead of engineering around GPU inefficiency, it built a model designed from the start to cost less to run. TechCrunch reports the new system is a post-training variation on Z.ai's open source GLM-5.2 model, paired with an upgraded harness -- the scaffolding that manages how a model plans and executes tasks -- specifically aimed at keeping token costs down for deployment. Writer is positioning this as enterprise-ready out of the box, not a research demo.
This matters because token costs are the quiet tax that erodes the ROI case for AI once a pilot moves into production. A chatbot that's charming in a demo can become a budget problem the moment usage scales past a few hundred employees, which is exactly the governance concern that's been showing up across enterprise AI deployments lately. Building on an open source base like GLM-5.2 rather than training from scratch is also a notable strategic choice -- it signals that the competitive edge Writer is chasing isn't a smarter base model, it's the layer on top: the harness, the fine-tuning, the packaging for real deployment. That's a more honest place to compete than the increasingly crowded and expensive race to claim the single smartest foundation model. It also lowers the barrier for smaller vendors to differentiate without needing billions in training compute, which is good for competition generally, even if it means enterprises now have to evaluate yet another vendor's cost claims against their actual workload rather than a marketing benchmark.
Put Kog and Writer side by side and a pattern emerges: the AI industry's next competitive front isn't who has the smartest model, it's who can run AI the cheapest without gutting quality. That's a healthier fight for buyers than the capability arms race of the last two years, because it means the tools businesses actually deploy -- inside a CRM, a support desk, an internal workflow -- should get cheaper to operate over time rather than more expensive as usage grows. It also means the vendors worth watching aren't necessarily the ones with the flashiest benchmark scores; they're the ones being transparent about what a workload actually costs at scale, a question we've pushed on before when OpenAI's spending trajectory came under scrutiny. Anyone building workflow automation or evaluating AI-driven internal tools should be asking vendors for real cost-per-task numbers, not just capability claims.
Which of these approaches do you think actually survives contact with a real production workload: squeezing more out of existing GPUs, or building smaller models designed for cheap deployment from day one?
Sources