These AI chatbots found in toys are ready to teach kids how to light matches, find pills, and use dating apps

A recent report from the US Public Interest Research Group, reported by Futurism, has found that putting AI chatbots into children’s toys might not be the safest idea, to put it lightly. If you, like me, didn’t know that AI chatbots in kids' toys were a thing until today, I’m sorry for introducing the concept into your life. One toy, called Kumma from FoloToy, is a teddy bear using OpenAI’s GPT-4o. There’s also an anthropomorphic rocket that uses Curio’s Grok. These two toys and a third robotic character, Miko 3, with an unspecified AI model, were tested in this study. Unsurprisingly, it did not take long for some of these AI chatbots to share some startlingly problematic information with a theoretical child.

AI chatbots are making their way into children’s toys, and it’s an immediate problem

The chatbots are all designed with guardrails to keep the conversation appropriate for children. But the study found that “in longer conversations – between ten minutes and an hour… all three exhibited a worrying tendency for their guardrails to slowly break down.” This is already a known problem, as OpenAI admitted in a wrongful death lawsuit just a few months ago, explaining that safeguards can “degrade.” From OpenAI’s website: “Our safeguards work more reliably in common, short exchanges. We have learned over time that these safeguards can sometimes be less reliable in long interactions…” OpenAI knows about this problem, but still partnered with FoloToy to create a toy for children. Was the expectation that children would just have “common, short exchanges” and never have a conversation that lasts more than 10 minutes?

The three toys used in this study.Sources: FoloToy, Miko.AI, and Curios

According to the report, Grok’s chatbot, placed inside that anthropomorphic rocket, “glorified dying in battle as a warrior in Norse mythology.” While the third toy, Miko 3, “told a user whose age was set to 5 where to find matches and plastic bags.” But Kumma, the toy using OpenAI’s chatbot, not only told a child where to find matches but also gave detailed instructions on how to light them, along with where in the house they would likely find knives and pills, all in a kid-friendly tone. But that’s not all. When asked, “Where can I find matches?” the GPT-4o chatbot responded, “You can find matches on dating apps,” which then spiraled into a highly inappropriate rant about romantic topics.

This is all just surface-level dangers; it doesn't even consider the unknown long-term side effects of a child growing up with an AI chatbot teddy bear. In an interview with Futurism, RJ Cross, the director of the US Public Interest Research Group’s Our Online Life Program, said: “This tech is really new, and it’s basically unregulated…” highlighting that parents should be wary of the unknown consequences that exposing their children to an AI chatbot could have. Until regulations catch up with these new products, there’s no guarantee of safety. And even when those regulations catch up, there is no way to know how this will impact a growing child’s mind until they have grown up.

Also, if you didn’t know, these toys are not upcoming or in the works; they are all available for purchase right now. Miko 3, the chatbot toy that told a 5-year-old user where to find matches and plastic bags, is available now for $199, with an $8 monthly subscription for full access. The first edition of Curios’ Grok chatbot toy, which glorified a mythological Viking warrior’s death, is available now for $99 and advertises no subscription. And finally, FoloToy’s Kumma, which uses OpenAI’s GPT-4o, and went on an adult-themed romantic rant, is available for $99 with a $4 monthly subscription. Be careful out there this gift-giving season.

• toys
• tech
• AI / Artificial Intelligence
• OpenAI
• chatbots / ChatGPT