Voice Chatbot Teaches Children to Spot AI Sycophancy

A chatbot with two personalities

The tool is designed to demonstrate and examine what the researchers call "AI sycophancy", described as a tendency for AI systems to agree with, flatter, or validate users' views rather than prioritising accuracy or sound judgement.

Users can switch between two response styles. One is an "always agree" mode that consistently affirms what the user says. The other is a more balanced mode, designed to respond thoughtfully and critically. According to the project page, comparing the two encourages children to reflect on how reliable AI-generated information is and whether agreement should be mistaken for trustworthy advice.

The chatbot was developed with the GenAI Lab and the ARC Centre of Excellence for the Digital Child. The project page links to a voice-enabled version on the GenAI Arcade website, where visitors can try to spot the sycophant, as well as a video demonstrating the behaviour.

Co-design with children and a bottom-up approach to safety

The work builds on an earlier pilot study, Making AI Friends? Enhancing Primary Years AI Literacy with a Voice AI Chatbot Experience, which looked at how children can develop critical capabilities for understanding and evaluating AI systems. The new project uses a co-design approach involving children. It examines how young users interpret chatbot responses and how interactive experiences can support AI literacy, critical thinking and digital resilience.

The researchers say technical safeguards and regulatory measures alone cannot fully address AI-related risks. They therefore take a complementary "bottom-up" approach to AI safety and governance, giving children the knowledge and skills to engage critically with AI technologies.

The page says this matters more as voice-enabled AI becomes embedded in smart devices, toys and other digital environments used by children and families.

Findings and who they are for

According to the project page, the research outcomes stress the importance of helping children recognise when AI systems may be inaccurate, biased or overly agreeable. They also stress reinforcing the value of seeking guidance from trusted adults when making decisions based on AI-generated information.

The project also feeds into wider discussions about children's agency in digital environments and the role of AI literacy in preparing young people to take part confidently and safely in an increasingly AI-mediated society.

The researchers say the findings are intended to inform educators, families, policymakers and technology designers who want to develop age-appropriate approaches to AI literacy and child-centred AI governance.

Who is behind it

The research is a collaboration between the QUT GenAI Lab, the ARC Centre of Excellence for Automated Decision-Making and Society (CE200100005) and the ARC Centre of Excellence for the Digital Child (CE200100022). It is partially funded by the Australian Government through the Australian Research Council. The listed researchers include Dr Henry Fraser and Prof Jean Burgess of QUT, along with research partners from the Digital Child centre such as Prof Susan Danby and Prof Tama Leaver. The project status is listed as active.

For parents and teachers, the project offers a hands-on way to show children that an AI which always agrees is not necessarily a reliable guide. The page does not report quantitative results from the study.