How AI is Helping Google, Roblox, and Discord Build a Safer Internet for Kids

A 3D rendering of Google's collaborative workspace concept, featuring stylized human figures with glowing outlines interacting in a multi-level environment with floating screens and a central Google logo, showcasing the innovative ROOST project.

The internet has become a vital space for children to learn, play, and connect, but it also exposes them to risks like cyberbullying, exploitation, and harmful content. In response, Google, OpenAI, Roblox, and Discord launched the Robust Open Online Safety Tools (ROOST) initiative in February 2025.¹ This nonprofit effort combines advanced AI tools and cross-industry collaboration to protect minors online.

Key Takeaways

  • Collaborative Framework: ROOST unites tech companies to combat child exploitation using AI-driven tools.¹
  • Real-Time Moderation: AI scans 4 billion text chats daily (Roblox) and 400,000 voice hours (Discord) to flag harmful content.²
  • Parental Controls: Google’s Family Link app integrates AI to block explicit content and monitor screen time.³
  • Open-Source Solutions: Free tools and grants help smaller platforms adopt ROOST’s safety measures.⁴

The ROOST Initiative: Goals and Technology

A stylized network visualization of major tech companies, including Discord, OpenAI, Google, and Roblox, connected by luminous lines against a dark background, illustrating their interconnectedness within the digital ecosystem, as envisioned by the ROOST project.

Funded with $27 million,⁵ ROOST addresses critical gaps in online child safety through three strategies:

1. AI-Powered Content Detection:
ROOST’s open-source tools analyze text, images, and audio to identify child sexual abuse material (CSAM), hate speech, and grooming behavior.¹ For example, Roblox’s AI flags phrases like “meet me offline” in 4 billion daily text chats for human review.²

2. Cross-Platform Collaboration:
Discord’s Lantern Project shares anonymized abuse data with partners like Google and Meta, helping block offenders across services.³

3. Support for Smaller Platforms:
ROOST provides grants and technical guidance to indie developers, ensuring even low-budget platforms can integrate AI moderation tools.⁴

Google’s Role: Privacy and Parental Oversight

A clean, simplified illustration of the Google search interface on a tablet screen, featuring the Google logo, a search bar with a magnifying glass icon, and a "Safe Search" button, representing the user-friendly design principles explored by the ROOST project.

Google contributes to ROOST by:

  • Enhancing Family Link: Parents restrict explicit content on YouTube Kids, set screen time limits, and receive alerts if safety settings are altered.⁵
  • Encrypting Minor Data: All interactions involving children—including emails and cloud storage—are secured with AES-256 encryption.¹
  • Filtering Search Results: Google’s Gemini AI blocks harmful content and flags grooming language like “don’t tell your parents.”⁶

Roblox’s AI Moderation at Scale

A stylized illustration promoting Roblox's open-source voice moderation tools, featuring a character holding a microphone, surrounded by interface elements and diverse avatars, emphasizing community safety and creative empowerment, aligning with the ROOST initiative's goals.

With 90 million daily underage users, Roblox employs AI to:

  • Scan Text Chats: Analyzes 4 billion messages daily for bullying, racism, or predatory behavior.²
  • Review Voice Interactions: Processes 400,000 hours of voice chats using multimodal AI trained on user-reported violations.³

In 2025, Roblox will release its voice moderation tools as open-source software to assist smaller platforms.⁴

Discord’s Safety Upgrades

A clean, minimalist illustration of a chat interface with a placeholder for user input and a series of message bubbles containing "[content removed]", representing a simplified communication platform concept, possibly related to the ROOST project's exploration of user-friendly interfaces.

Discord’s 2025 updates include:

  • Keyword Blacklists: Auto-deletes messages containing slurs, threats, or predatory language.⁷
  • Lantern Integration: Shares abuse patterns with partners to block offenders globally.³

OpenAI’s Ethical Safeguards

A futuristic robot figure in a high-tech environment with a "BLOCKED" banner across its chest, visually representing access denial or security protocol intervention, potentially related to the ROOST project's exploration of AI safety.

OpenAI ensures its generative AI models:

  • Block Harmful Prompts: Rejects requests related to CSAM, violence, or self-harm.⁶
  • Redirect Sensitive Queries: Connects minors asking about mental health to verified resources like Crisis Text Line.⁷

Challenges and Future Plans

A conceptual image of a balance scale with "DATA PRIVACY" on one side and "CHILD SAFETY" on the other, each represented by spheres, visually illustrating the delicate equilibrium sought in online environments, potentially in relation to the ROOST project's focus on ethical technology.

ROOST faces criticism over false positives (legitimate discussions flagged as harmful) and privacy concerns with centralized abuse databases.¹ To address these, ROOST will:

  • Expand language support beyond English by late 2025.⁴
  • Allocate $50 million in grants to indie developers.⁵

Conclusion

Children worldwide behind 'Protected by ROOST' overlays. Global, inclusive.

ROOST exemplifies how AI and collaboration can create safer digital spaces for children. By prioritizing ethical oversight and accessibility, this initiative sets a new standard for online child protection.

Citations

  1. “Google, OpenAI, Roblox and Discord Launch Initiative to Protect Children Online.” Mezha.Media, 11 Feb. 2025.
  2. Koneru, Naren. “Driving Civility and Safety for All Users.” Roblox Newsroom, 22 July 2024.
  3. “Discord Expands Lantern Initiative with ROOST.” Discord Safety Center, 10 Feb. 2025.
  4. “Roblox, Discord, OpenAI, and Google Raise $27 Million for ROOST.” The Verge, 10 Feb. 2025.
  5. “Family Link Upgrades for 2025.” Google Blog, 15 Jan. 2025.
  6. “Gemini AI Filters Harmful Content.” TechCrunch, 7 Feb. 2024.
  7. “OpenAI Forms Child Safety Team.” TechCrunch, 7 Feb. 2024.


Please note, that the author may have used some AI technology to create the content on this website. But please remember, this is a general disclaimer: the author can’t take the blame for any mistakes or missing info. All the content is aimed to be helpful and informative, but it’s provided ‘as is’ with no promises of being complete, accurate, or current. For more details and the full scope of this disclaimer, check out the disclaimer page on the website.

Share the Post:

Related Posts