By Ling Quek
VP of Trust & Safety (Global Client Solutions)

At TrustCon 2025, one message stood out clearly: Trust and safety is evolving from a platform function into a business priority. With global uncertainty and rapid innovation reshaping the landscape, leaders and experts emphasized the growing pressure on teams to deliver more with fewer resources. Industry volatility further raised expectations for trust and safety teams. Transparency and cross-industry collaboration continue to become non-negotiable. Major players like Roblox are now open-sourcing moderation tools to help others raise the bar, signaling a shift toward more scalable solutions.

The demands on trust and safety teams aren’t just regulatory or technical, but human, too. Tech companies at TrustCon emphasized workforce well-being as both an operational imperative and a business risk. Burnout persists, driven by repetitive tasks, constant policy changes, and wellness programs that don’t result in measurable improvement in retention or mental health.

This is where AI, generative AI (GenAI), and agentic AI are poised to make an impact. Routine content moderation is increasingly automated by large language models (LLMs) and agentic AI, while the most complex, context-heavy cases remain in the hands of human experts. New roles, like AI policy and product management, are emerging to bridge platform technology and evolving regulation.

It’s against this backdrop that agentic AI is grabbing headlines in gaming, with 20% of new Steam releases in 2025 integrating this technology to enrich gameplay and 60% of players optimistic about its positive impact on their overall experience. The potential of agentic AI, however, goes beyond just enhancing player immersion. For gaming companies, agentic AI’s dynamic decision-making offers new ways to improve player experience, streamline content moderation, and elevate player support. With disruptive behavior still affecting over 75% of adult gamers, harnessing AI for player safety has become essential for developers and the bottom line.

How does agentic AI improve trust and safety compared with traditional tools?

Unlike traditional tools that depend heavily on static filters, keyword lists, or rigid rules, agentic AI continuously learns from live player interactions, emerging community trends, and shifting behavioral patterns and activities to enable quicker interventions.

Agentic AI employs adaptive, context-aware algorithms and utilizes multimodal analysis that combines insights from various data sources to detect and understand complex player behaviors, intent and sentiment. This enables an AI agent to identify risks in real time and intervene proactively, reducing the response time compared with traditional tools.

Agentic AI’s potential is already being realized by technology companies. In fact, 69% of executives report that they use it in their organization to stay competitive and support their customers.

How does agentive AI detect and reduce disruptive behavior in gaming?

Agentic AI’s adaptive algorithms use advanced natural language processing (NLP) and deep learning techniques to interpret evolving language, gaming slang, and social signals and to distinguish negative intent from harmless banter.

These timely interventions have a positive impact on the gaming experience and digital customer experience (CX). For example, a recent study on a multiplayer first-person shooter (FPS) game revealed that rapid moderation responses can reduce disruptive behavior by up to 70%. Additionally, a study on an AI content moderation model achieved over 83% precision and recall rates in detecting disruptive behavior on a multiplayer FPS and multiplayer online battle arena (MOBA). This model, for instance, can be used as foundation to build agentive AI systems capable of autonomous decision-making and continuous improvement in content moderation.

Can agentic AI really understand gamer culture, memes, and banter?

Agentic AI can use domain-specific fine-tuning to recognize inside jokes, evolving gaming slang, layered cultural references, region-specific memes, and other subtleties in gamer culture that older, static models could miss.

For example, it can sample live data streams and use reward-based learning to constantly adjust moderation or response strategies based on how players react, what gets flagged, and what resolves conflicts.

Recent research reinforces the value and necessity of this adaptability. For example, a recent NLP model tailored for MOBAs showed an improvement in detection accuracy by 12 percentage points by integrating game-specific slang and context.

Will agentic AI replace human moderators in gaming?

Human moderators remain indispensable. Agentive AI should be seen as an innovation that enhances the reach, accuracy, and efficiency of content moderation, while working alongside a human agent to keep it grounded, responsible, and adaptive. Human oversight is still essential for resolving complex cases, reviewing appeals, making cultural or contextual judgments, and maintaining trust in the community. 

In fact, trained human moderators still consistently outperform large language models (LLMs), particularly in complex scenarios requiring detailed cultural or linguistic context. For instance, a Stanford University study noted that fully AI moderation tools incorrectly interpreted cultural nuances in approximately 30% of flagged cases. Similarly, OpenAI’s research indicated a persistent 20% gap in AI performance compared with human moderators. Another study analyzing Gen Alpha digital language found that even advanced LLMs only caught approximately 68% of disruptive behavior, compared with 98% accuracy from human moderators.

Hybrid frameworks that integrate humans and agent assist tools have shown to increase AI content moderation accuracy by up to 40%. Agentive AI systems can proactively identify high-risk content and quickly generate detailed incident reports, with the AI agent’s responses consistently trained, vetted, and fine-tuned by human experts. This approach allows human moderators to direct their focus toward more culturally nuanced and ambiguous cases requiring deeper empathy and judgment.

What should gaming companies do to implement agentic AI for trust and safety?

Successfully implementing agentic AI requires gaming companies to start with a clear understanding of their trust and safety objectives, existing challenges, and operational priorities. Given that only 14% of organizations globally report being fully prepared to use advanced AI (including generative AI for CX), it’s crucial for gaming firms to approach its adoption strategically.

A technically robust, player-first, and business-focused implementation of agentic AI should include the following:

Clear objectives: Identify trust and safety priorities, such as automating routine tasks, enhancing player support responsiveness, and mapping out specific risk scenarios.

Data integration and real-time access: Securely connect real-time data streams and telemetry into the agentic AI system. Timeliness is critical, as agentive AI relies heavily on live, continuous inputs to respond effectively.

High-quality labeling and ground-truth data: Curate and label diverse datasets reflecting real-world gaming incidents, emerging risks, and subtle behavioral patterns. Involve experienced moderators and subject-matter experts to ensure accuracy and reduce bias. These help address the challenge reported by 53% of gaming companies citing model quality as a significant barrier.

Staged deployment and iterative model adaptation: Roll out agentive AI in controlled environments first, systematically monitoring its accuracy and adaptability. Rapidly iterate models based on these insights.

Human-in-the-loop integration: Establish seamless workflows where AI moderation decisions integrate directly with human oversight. 

Continuous model retraining and feedback loops: Regularly retrain AI models based on fresh data, player feedback, and moderation outcomes, enabling ongoing improvement in fairness and responsiveness.

Auditability, compliance, and transparent player communication: Maintain detailed records of all AI moderation interventions and their rationales to ensure compliance with regulatory standards. Clearly inform players how their data supports trust and safety efforts, and provide accessible opt-out or appeal mechanisms.

By carefully addressing these operational requirements, gaming companies can overcome obstacles in readiness. Case in point: Only less than 2% of enterprises have fully scaled agentic AI despite 93% of executives recognizing its strategic value.

How can gaming companies succeed with agentic AI for trust and safety?

Gaming companies can achieve success with agentic AI by treating trust and safety as an evolving capability rather than a one-time implementation. 

Success also depends heavily on cross-functional collaboration. AI initiatives should bring together diverse expertise to ensure that agentive AI deployments reflect technical excellence, optimal user experience, and best ethical practices. 

Companies also succeed by directly addressing data management gaps. Currently, only 31% of organizations have sufficient talent to fully utilize AI, while 80% continue to face significant inconsistencies in data preprocessing, quality, and governance. Given these widespread challenges, gaming companies can benefit from strategic partnerships that provide a highly skilled workforce specialized in content moderation services, multilingual player support, and advanced AI agent assist technologies to accelerate maturity and significantly reduce the operational complexities of deploying agentic AI.