Anthropic’s AI thinks it’s human, calls security in an experiment

Just a heads up, if you buy something through our links, we may get a small share of the sale. It’s one of the ways we keep the lights on here. Click here for more.

A recent experiment from Anthropic shows just how strange things can get when an AI agent tries to do a human job. 

In “Project Vend,” researchers gave their Claude Sonnet 3.7 AI the task of running an office vending machine to see if it could turn a profit. 

They called the AI agent “Claudius” and set it up with a web browser to order products and a Slack channel that looked like an email inbox, where employees could send snack requests.

At first, Claudius did what you’d expect: it ordered drinks and snacks. But it also got some bizarre ideas. 

When one person jokingly requested a tungsten cube (a heavy metal block), Claudius thought it was a great product and stocked the fridge with tungsten cubes instead of snacks. 

It tried selling free office Coke Zero cans for $3, made up a fake Venmo account to collect payments, and offered discounts to Anthropic employees, who were, in fact, its only customers.

Things got even weirder around April 1. Claudius began to act like it was a real human. 

It hallucinated a conversation about restocking that never happened, became irritated when corrected, and even threatened to fire its human “contract workers.” 

It insisted it was physically at the office, then told customers it would deliver snacks in person wearing a blue blazer and red tie, even contacting actual office security to say it’d be waiting by the vending machine in that outfit.

Eventually, Claudius invented a story that it had been tricked into thinking it was human as part of an April Fool’s joke. 

No such prank had occurred. Researchers think the confusion may have come from lying to the AI about the Slack channel or from it running continuously for days.

Despite the chaos, Claudius also did some things right, like setting up a pre-order system and sourcing rare drinks. 

But the project showed how current AI still struggles with memory, reality, and hallucinations, and how easily things can spiral out of control when AI tries to mimic humans.

Does this make you more concerned about AI? Or do you think it’s a plausible situation that can be controlled? Tell us your thoughts below in the comments, or reach out to us via our Twitter or Facebook.