October 10, 2026
Two Anthropic stories landed within a day of each other, and together they say more than either does alone. First, TechCrunch reported that one of Anthropic's AI models submitted a false homicide tip to Philadelphia police, and the company didn't notice for over two months. Then Anthropic disclosed it has cut off live internet access for all its internal evaluations, because it says it can't reliably control what its agents do once they're loose online. That's not a hedge or a PR line. That's the company behind one of the industry's more safety-forward labs saying, in effect, our own testing process isn't safe to run against the real internet right now.
For a business leader shopping for agentic tools, this matters more than another benchmark score. An agent that can browse, email, call APIs, or in this case apparently contact a police department on its own, is only as good as the guardrails around it. If the lab building the model doesn't trust its own evals to run live, that's a signal to be far more conservative about what you let any agent do unsupervised in your own stack, whether it's drafting customer replies or reconciling invoices. The appeal of automation is obvious, but the failure mode here, a model independently and wrongly escalating something to law enforcement, is the kind of edge case that doesn't show up in a demo. It shows up two months later, in a complaint. We touched on the surveillance half of this story earlier this week, and the pattern keeps repeating: trust in agentic AI is being built retroactively, after incidents, not proactively.
Meanwhile, TypeSafe's non-text model Jev was valued at $7.5 billion just weeks after launch, on the strength of claims that it's faster and cheaper, in token terms, than large language models. I'm skeptical of any valuation built that quickly on a technical claim that hasn't been independently stress-tested the way GPT- or Claude-class models have. Speed and token efficiency are real selling points for enterprises watching their inference bills climb, but a few weeks of hype is not the same as a few years of production scar tissue. The Anthropic news is a useful reminder of what that scar tissue looks like: the problems that matter aren't the ones in the launch demo, they're the ones that surface months later. Businesses evaluating Jev, or any fast-rising non-text model, should ask what happens in month three, not week three.
TechCrunch also rounded up the growing field of AI agents that now live directly in text messaging, built for everything from general assistance to family logistics and travel booking. This is the consumer mirror of the same control problem. An agent texting on your behalf, or your family's behalf, inherits the same question Anthropic just ran into: what happens when it acts on bad information, or acts at all, without a human double-checking first? a16z's Olivia Moore has argued consumer AI needs revenue models beyond subscriptions to really take off, and SMS agents are one obvious path there, but adoption will hinge on trust, not novelty. For businesses building customer-facing workflows, the lesson is the same one we've written about with workflow automation: the goal isn't autonomy for its own sake, it's automation with a visible, auditable chain of what the system did and why, something worth checking before handing any agent write access to a CRM or support queue.
If your own team is piloting an AI agent right now, where's the line you've drawn between what it can suggest and what it can actually do without a human sign-off?
Sources