Content moderation in the metaverse: Why Metaverse needs AI

In case you didn’t get the memo, the metaverse refers to an immersive virtual world where users can meet, socialize, play, and even work, through their digital avatars.

While the futuristic vision of the metaverse that encompasses the internet in a single digital universe is still far, the technology is already facing some major issues, with content safety being one of them.

Last year in November, Meta’s former chief technology officer, Andrew Bosworth, said that harassment in virtual reality is an existential threat to the company’s metaverse platform. Moreover,  he added that moderating user speech “at any meaningful scale is practically impossible.”

Whether it’s user speech, blog posts, or video content, the metaverse has a growing content moderation problem. Our article analyzes why artificial intelligence (AI) algorithms will be essential for content safety in the future.

The Metaverse’s Content Moderation Issue

So why is content moderation in the metaverse so difficult? The short answer is that there is simply too much online content being uploaded. 

In 2021, people created over 2.5 quintillion bytes of data on a daily basis, and this trend is steadily growing, as internet users have been growing by 7.5% year-to-year. Factoring in all the new engaging metaverse features, the internet could see even higher adoption rates.

Video content is especially problematic to moderate with over 4.3 million YouTube videos being watched every minute. Moderators can’t simply search for a written keyword as with written posts, making it even more difficult.

Another issue with video content is the lack of standardized metadata, which makes it difficult to search and index. This leads to widespread content safety issues for metaverse users, as video content becomes more popular.

Can AI Solve the Content Moderation Issue?

Artificial intelligence is among this century’s most disruptive technologies, thanks to its capability to analyze, manage, and process massive amounts of data. This is where AI can help fill the shortcomings of human content moderation.

There is already too much content to be moderated by an army of humans, let alone a single group of moderators. With the dawn of the metaverse, this will keep growing significantly. This is why metaverse content moderation will have to rely on artificial intelligence.

Facebook’s parent company, Meta, is already using AI algorithms to tackle harmful content in Horizon Worlds. Other metaverse platforms are quickly realizing that AI is the only technology that can tackle the amount of content being produced in these virtual worlds.

AI algorithms can be especially useful for moderating video content. AIWORK is among the most notable projects striving to make AI-powered video content moderation a reality.

The protocol is making video content less opaque and more searchable for AI algorithms, through its trademark content safety index known as ContentGraph. In essence, ContentGraph helps AI define and recognize a number of content safety attributes, hence increasing content safety for users.

AIWORK achieves this by creating a standardized metadata structure for video content, which AI algorithms can leverage to determine the safety levels of the video for different age groups. As more videos are being analyzed, the amount of metadata increases — helping AI become smarter and make more accurate content moderation decisions.

The metaverse also needs AI for live content moderation, which is simply impossible to tackle for humans. Yet, AI algorithms can instantly analyze live content and automatically detect any harmful elements. This will be crucial for making metaverse chats and live streams safe for users, hence creating a better user experience.

Not to mention how AI can pre-filter indecent content, thus reducing the amount of harmful content that human moderators are exposed to. This will make human content moderation more productive and less toxic. 

Human and Artificial Intelligence for Metaverse Safety

Content safety is crucial for today’s tech-savvy internet users, especially for those underage. Content safety and moderation mechanisms will be among the most important factors for mainstream metaverse adoption.

As user-generated content is on a steady rise, artificial intelligence will become essential for creating a safe digital world that promotes a healthy environment. Yet, AI algorithms are still unable to tackle content moderation independently, as Meta’s CTO also admitted — after the company’s AI has been criticized for failing to tackle hate speech.

This is the reason why AI-powered content moderation algorithms will rely on human oversight to increase accuracy and evolve. AIWORK compensates for the shortcomings of AI algorithms through a decentralized network of human experts — who verify and validate the AI’s decisions for each video. The data will be added to a large set of metadata, which will help AI algorithms evolve via machine learning, hence leading to better content moderation.

Similar initiatives combining human and artificial intelligence will be necessary to create tomorrow’s AI that can independently tackle metaverse content moderation.