September 12, 2026
On September 11, 2026, Firecrawl published a report titled ["How to Build an AI Software Factory: Agents That Open, Review, and Merge PRs"](https://www.firecrawl.dev/blog/ai-software-factory), pulling together public figures from three engineering organizations that are running AI coding agents at real production scale: Shopify, Stripe, and Microsoft. It's the most concrete look yet at what "AI writes our code now" actually looks like inside companies that have the resources to do it properly -- and the numbers are more interesting than the marketing version of this story usually allows. Shopify's internal agent pipeline, called River, logged 59,918 sessions and merged 3,536 River-coauthored pull requests in a single 30-day window, according to Firecrawl's reporting. Shopify says roughly one in eight pull requests merged into its codebase in a recent week was authored by River. Stripe, per the same report, merges more than 1,300 agent-authored pull requests per week, drawing on roughly 500 internal tools available through an internal system Stripe calls Toolshed. These are not pilot-program numbers. This is agents doing a meaningful share of the actual work.
The most telling figure in the whole report comes from Microsoft's dotnet/runtime team, and it's not a celebration stat. Agent-authored pull requests merge only 67.9% of the time, versus 87.1% for human-authored pull requests, Firecrawl reports. Merged agent PRs also draw more scrutiny, not less: 16.5 review comments on average versus 12.4 for human PRs, with the top two reviewers producing 36% of all feedback on agent PRs. The sharpest split is between fully autonomous agent PRs and hybrid ones. Fully autonomous agent PRs -- no human commits at all -- merged only 55.1% of the time. Agent PRs that included at least some human commits merged 86.2% of the time, almost matching the human baseline. That's not a footnote. That's the whole story: a human hand somewhere in the commit history is doing an enormous amount of trust-building work, even inside teams that are otherwise all-in on agentic development.
Firecrawl frames what these companies actually do as five deliberate stages, not a single magic model: Intake (deciding what work is worth an agent starting on), Isolation (dedicated workspaces so parallel agents don't collide), Tools (a shared layer of internal capabilities agents can call, like Stripe's Toolshed), Verification (automated quality checks before anything reaches a human), and a Merge Gate (an explicit human accountability checkpoint before code ships). The throughline Firecrawl draws is the one worth sitting with: human review capacity is the real bottleneck now, not code generation. Generation scales with however much a company is willing to spend on agent compute. Review capacity doesn't scale that way -- it's bounded by how many qualified humans you have and how much attention each one can give.
We think this data is a genuinely useful, unglamorous correction to the "AI just writes the code now" narrative -- and we say that with real respect for what Shopify, Stripe, and Microsoft have built. These are three of the most sophisticated engineering organizations on the planet, and even inside their pipelines, fully autonomous agent code merges at a meaningfully lower rate and draws measurably more scrutiny than code with a human hand in it. That's not a knock on the agents. It's a precise description of what it actually takes to make agent output trustworthy. Here's the argument we'd make from this data: what makes agent-authored code trustworthy at these three companies isn't the agent. It's the entire parallel engineering discipline built around it -- isolated environments so agents don't step on each other, a shared tool layer so agents can actually do useful work instead of guessing, automated verification that catches problems before a human ever looks, and named human reviewers who personally account for over a third of all feedback on agent PRs. That infrastructure is the product of years of engineering investment by teams with hundreds of software engineers on staff. A small business with no engineering team has none of that. It has no Toolshed, no isolated agent workspaces, no dedicated review rotation, no dotnet/runtime-style discipline for tracking merge rates by PR type. And it's not going to build any of that -- not because the idea is wrong, but because that kind of infrastructure is a multi-year engineering investment that only makes sense at Shopify's or Microsoft's scale. That's exactly why we think the trust-and-verification loop has to be built into the platform itself, not bolted on afterward as a review discipline only a company the size of Shopify or Microsoft can staff. The lesson from this data isn't "AI can't be trusted to write code." It's "trustworthy AI-written code requires real verification infrastructure around it" -- and for most small businesses, that infrastructure has to come from the platform, because there's no internal engineering org to build it themselves. This is the same reasoning behind how we think about the platform itself: verification and guardrails need to be default behavior, not an optional discipline a customer has to invent on their own. It's also why we're skeptical of framing every AI coding advance as universally applicable. We wrote recently about Cognition's $48B round and made a similar point: proving AI coding works inside a company full of engineers is a different claim than proving it works for a business with none. Firecrawl's report is the clearest evidence yet for that distinction, straight from the companies with the most to gain from claiming otherwise. If even Microsoft's own teams need a merge gate and a human reviewer's commits to trust agent code, that tells you something real about what "trustworthy AI-written software" requires -- and it's worth reading alongside our take on what happens when an AI-built app eventually needs a real developer, because the underlying question is the same: who is actually accountable for what shipped.
Sources