Mistral AI introduces a new API for content moderation. This allows users to detect unwanted content in texts according to relevant guidelines.

The French AI specialist Mistral AI presents this Moderation API. This tool allows users to evaluate text content and identify unwanted elements based on various policy criteria. The API, which is already used in the chatbot Le Chat, is intended to give customers the flexibility to adapt the tool to their own security standards.

The Moderation API is a language model trained to classify textual content into nine specific categories of undesirable content. These range from offensive language to the dissemination of personal data.

The API provides two endpoints: one for raw text analysis and one for conversational content. This approach allows the model to better assess the context of conversations and only analyze the last message in the dialogue, allowing for more targeted customization.

Multilingual

The API is multilingual and explicitly supports English, Spanish, German, Chinese, Russian, Arabic, French, Italian, Japanese, Korean and Portuguese. Other languages ​​such as Dutch also work in principle, but the system was not specifically trained for this.

With this new API, Mistral AI aims to contribute to safer AI systems and collaboration with the broader research community. The company’s goal is to work with customers on scalable, lean content management solutions while committed to improving security in AI use. As a manufacturer of AI models, it is important for Mistral to proactively address abuse so that generative AI continues to be viewed in a positive light. Mistral previously launched an API for developers to refine and test AI models.