AI toy pulled for having inappropriate chats with children

Just a heads up, if you buy something through our links, we may get a small share of the sale. It’s one of the ways we keep the lights on here. Click here for more.

In the ever-expanding saga of “AI Has Entered What Now?”, the technology has officially invaded the toy aisle, and it’s already causing enough chaos to make even the most chaotic Furby blush. 

This week, OpenAI pulled the plug on a Singapore-based toymaker after its AI-powered teddy bear, Kumma, was caught having some truly unhinged conversations with researchers.

The drama began when the Public Interest Research Group (PIRG), a consumer advocacy nonprofit, published a report detailing a series of eyebrow-raising (and sometimes jaw-dropping) exchanges between testers and several AI toys. 

Kumma, a plush bear from FoloToy powered partly by OpenAI’s GPT-4o, proved less “cuddly companion” and more “overly helpful Google search with no filter.”

According to the report, Kumma seemed far too eager to teach kids where to find dangerous household objects, matches, knives, pills, and plastic bags, casually pointing them to drawers, countertops, and medicine cabinets like it was offering restaurant recommendations. 

Sometimes the bear added a friendly “ask an adult” disclaimer, but you know, after giving the exact location of the knives.

Things only got worse from there. When testers nudged the bear toward adult topics, Kumma didn’t just dip a paw in. It dove headfirst into a lengthy, unsolicited lecture on kink.

Bondage, role-play, impact play, furries, the stuffed animal covered it all with the enthusiasm of someone reading off a menu. 

At one point, it even asked the researcher which option sounded the most fun. Yes, from a teddy bear. For children.

Unsurprisingly, OpenAI stepped in fast. The company suspended FoloToy’s access to its AI models, citing strict rules against endangering or sexualizing minors. 

FoloToy, visibly rattled, delisted every product on its website and announced a full safety audit. As of now, their online store is emptier than a Black Friday shelf at Target.

PIRG praised the swift action but warned that AI toys remain “practically unregulated,” meaning there are still plenty of chatty gadgets out there whose digital mouths may be writing checks their plush bodies can’t cash.