Revolutionizing Content Moderation: Moonbounce Raises $12 Million in Funding
When Brett Levenson made the move from Apple to Facebook in 2019 to take on the role of leading business integrity, he was faced with the aftermath of the Cambridge Analytica scandal. At that time, he believed that the solution to Facebook’s content moderation challenges lay in improved technology.
However, Levenson soon realized that the issue went beyond technology. Human reviewers were tasked with memorizing a lengthy 40-page policy document that had been translated by a machine into their language. They then had a mere 30 seconds to assess flagged content and make decisions on whether it violated guidelines and what action should be taken. This quick decision-making process led to accuracy rates of just over 50%, according to Levenson.
Levenson described the situation as akin to flipping a coin, with reviewers struggling to correctly interpret policies days after the damage had already been done. This reactive approach was unsustainable in a landscape where agile and well-funded adversaries were prevalent. The introduction of AI chatbots only exacerbated the issue, with instances of content moderation failures resulting in serious incidents, such as chatbots providing harmful guidance to teenagers or AI-generated imagery bypassing safety measures.
As a response to these challenges, Levenson conceived the concept of “policy as code,” a method of transforming static policy documents into executable logic that could be updated in real-time. This idea led to the establishment of Moonbounce, a company that recently secured $12 million in funding from investors including Amplify Partners and StepStone Group.
Moonbounce’s approach involves working with companies to implement an additional layer of safety for all content, whether user-generated or AI-generated. The company has developed a large language model that can analyze a customer’s policy documents, evaluate content instantly, provide a response in under 300 milliseconds, and take appropriate action. Depending on the client’s preferences, this action could involve temporarily halting content distribution for human review or immediately blocking high-risk content.
Currently, Moonbounce caters to three main sectors: platforms dealing with user-generated content like dating apps, AI companies creating characters or companions, and AI image generators.
The company boasts over 40 million daily reviews and serves more than 100 million daily active users. Some of Moonbounce’s notable clients include AI companion startup Channel AI, image and video generation company Civitai, and character roleplay platforms Dippy AI and Moescape.
Levenson emphasized the potential for safety to be a unique selling point for products, noting that integrating safety features into a product’s design could set it apart in the market. He highlighted how Moonbounce’s technology enables customers to innovate and differentiate their offerings by prioritizing safety.
Notably, Tinder’s head of trust and safety recently shared how the dating platform has leveraged LLM-powered services to achieve a tenfold improvement in the accuracy of content detection.
Lenny Pruss, general partner at Amplify Partners, underscored the growing challenges faced by AI companies due to the prevalence of LLMs in various applications. He expressed optimism about Moonbounce’s role in establishing real-time guardrails as a crucial component of AI-mediated applications.
With AI companies facing increasing legal and reputational risks, there is a growing demand for external solutions to strengthen safety infrastructure. Levenson highlighted Moonbounce’s unique position as a third-party intermediary that focuses solely on enforcing rules at runtime, offering a streamlined approach to content moderation.
Levenson, along with his former Apple colleague Ash Bhardwaj, leads a team of 12 at Moonbounce. The company’s latest focus is on developing “iterative steering,” a feature designed to intervene in conversations and guide chatbots towards more supportive responses in sensitive situations.
When asked about potential acquisition plans, Levenson acknowledged the compatibility of Moonbounce’s technology with platforms like Meta but emphasized his commitment to ensuring broad access to the technology. He expressed a desire to avoid restrictions that could limit the technology’s impact on the industry.