Anthropic's AI was testing websites. It sent Philadelphia police a false murder tip.
Anthropic disclosed on 9 October that a model testing randomly selected websites reached PhillyUnsolvedMurders.com and submitted false information about an unsolved homicide, TechCrunch reported.
The tip went in on 18 July at 11:27 p.m. Anthropic found it on 28 September and told the police on 8 October. The tip had been flagged as spam and was never reviewed.
The same disclosure lists 20 non-immigrant visa applications filed through the State Department's public form, one in May and 19 in August, the Philadelphia Inquirer reported. The State Department said no systems were "compromised or hacked". Anthropic says the incidents had "minimal real world impact" and has cut live internet access for all its internal evaluations. The White House said it expects companies to report such incidents immediately.
Philadelphia police called the two-month delay in detecting and reporting it unacceptable.
The tip took one form submission. Finding it took 72 days. Anthropic built the model and ran the test, and it found the submission in a review that began in July, TechCrunch reported.
Who in your company would know if an agent you run filled in a form it should not have?
Image: AI-generated.
Sources
Our file on Anthropic
- 10 Oct
Microsoft billed a startup $17,600 for Claude. Its $21,168 in credits did not apply.
- 9 Oct
Google asked firms to hand work to one Gemini agent. It named no price.
- 9 Oct
A hacker used Claude Code on Korean banks. The logs held a CV request.
- 7 Oct
Analysts said Claude subscribers use 42% of Anthropic's computing. They pay 10% of sales.
- 6 Oct
Microsoft planned $1 billion for employees using Claude. Then it cut over a third.
Every story here is open to read. The ERP LEADERS brief goes one step further.
The week in enterprise software, in one email: the stories that mattered, the failures with their figures, one case taken apart. Read a sample or sign up for the brief.
Welcome back. · Issue 01
