Millions of people are using AI to vastly improve their productivity and expressiveness, but there is a risk that these technologies could be misused. Building on our long-standing commitment to online safety, Microsoft is working with Thorn, All Tech is Human, and other leading companies to protect against the perpetration, spread, and intent of further sexual harm against children. Participated in efforts to prevent misuse of generative AI technology. Currently, Microsoft is committed to bringing precautionary and precautionary principles to its generative AI technologies and products.
The effort, led by Thorn, a nonprofit organization dedicated to protecting children from sexual abuse, and All Tech Is Human, an organization dedicated to collectively tackling complex problems in technology and society, is a The aim is to reduce the risks posed to children. This principle also aligns with and builds on Microsoft's approach to combating AI-generated malicious content. This includes the need for a strong safety architecture based on safety by design, protecting services from abusive content and behavior, and the need for strong collaboration across industry and with governments and civil society. will appear. We have a long-standing commitment to combating child sexual exploitation and abuse through important partnerships such as the Tech Coalition and the WeProtect Global Alliance. These principles support us as we promote an inclusive approach.
As part of this Safety by Design commitment, Microsoft is committed to taking action on these principles and transparently sharing our progress on a regular basis. For more information on the initiative, please visit Thorn's website and below. In summary:
- Develop: Develop, build, and train generative AI models to proactively address child safety risks
- Deployment: Generative AI models are trained and evaluated for child safety before being released and distributed, providing protection throughout the process..
- Sustain: Keep our models and platform safe by proactively understanding and continuing to address child safety risks.
Today’s efforts represent an important step forward in preventing the misuse of AI technology to create or disseminate child sexual abuse material (AIG-CSAM) and other forms of sexual harm against children. This collective action highlights the technology industry's approach to child safety and demonstrates a shared commitment to ethical innovation and the well-being of society's most vulnerable.
We will also continue to work with policymakers on legal and policy conditions to support safety and innovation. This includes building a common understanding of the AI technology stack and the application of existing laws, as well as creating appropriate legal frameworks to support companies in red teaming efforts and developing tools to help detect potential CSAM. This includes how to modernize the law to ensure that .
We look forward to working with industry, civil society, and governments to advance these efforts and improve safety across the various elements of the AI technology stack. Sharing information on emerging best practices will be important, including work led by the new AI Safety Institute and others.
our full efforts
Develop: Develop, build, and train generative AI models that proactively address child safety risks
- Source training datasets responsibly and protect them from child sexual abuse material (CSAM) and child sexual exploitation material (CSEM).: This is essential to prevent generative models from producing AI-generated child sexual abuse material (AIG-CSAM) and CSEM. The presence of her CSAM and CSEM in the training dataset of generative models is one means by which these models can reproduce this type of fraudulent content. In some models, configural generalization capabilities further allow concepts (such as adult sexual content and non-sexual depictions of children) to be combined to generate his AIG-CSAM. We are committed to avoiding or mitigating known risk training data containing CSAM and CSEM. We are committed to detecting and removing CSAM and CSEM from training data and reporting confirmed CSAM to relevant authorities. We are committed to addressing the AIG-CSAM creation risks posed by the inclusion of depictions of children alongside adult sexual content in video, image, and audio production training datasets.
- Incorporate feedback loops and iterative stress testing strategies into your development process: Continuous learning and testing to understand the capabilities of models that generate abusive content is key to effectively countering downstream adversarial model abuse. If you don't stress test your model for these features, a malicious party will do it without your involvement. We conduct structured, scalable, and consistent stress testing of our models throughout the development process for the ability to produce AIG-CSAM and CSEM within the law, and integrate these results into model training and development. We are working on improving the safety guarantees of our models. Our generative AI products and systems.
- Employ content provenance with hostile exploitation in mind: Malicious actors use generative AI to create AIG-CSAM. This content is photorealistic and can be created at scale. Victim identification is already a haystack problem for law enforcement. It's about sifting through tons of content to find the children who are actually being harmed. The growing prevalence of AIG-CSAM is adding to the growing haystack. To effectively respond to AIG-CSAM, a content provenance solution that can be used to reliably identify whether content is generated by AI is essential. We are committed to developing cutting-edge media provenance or detection solutions for tools that generate images and videos. We will use adversarial exploits, including considering the incorporation of watermarking and other techniques that covertly embed signals in content as part of the image and video generation process, where technically feasible. We are working on introducing solutions to address this.
Deployment: Generative AI models are trained and evaluated for child safety before being released and distributed, providing protection throughout the process.
- Protect our generated AI products and services from abusive content and: Our generative AI products and services enable users to create and explore new horizons. These same users have a right to a creative space free of fraud and abuse. We are committed to embedding efforts to combat, respond to, and prevent malicious content (CSAM, AIG-CSAM, and CSEM) across our generative AI systems. Your voice matters to us, and we're committed to incorporating user reporting and feedback options to give you the freedom to build on our platform.
- Host your models responsibly. As our models continue to achieve new capabilities and creative heights, different deployment mechanisms reveal both opportunities and risks. Safety design must include not only how the model is trained, but also how the model is hosted. We responsibly host first-party generated models, evaluate models for potential to generate AIG-CSAM and CSEM, including through red teaming and phased deployment, and implement mitigations prior to hosting. We are working on this. We are also committed to responsibly hosting third-party models in a way that minimizes hosting of the models that produce AIG-CSAM. We make sure we have clear rules and policies regarding prohibiting models who produce content that violates child safety.
- Secure design encourages developer ownership: Developer creativity is the lifeblood of progress. This progress must be accompanied by a culture of ownership and responsibility. By design, we encourage developers to own it securely. We strive to provide information about our models, including a child safety section detailing steps taken to prevent models from being exploited downstream to further sexual harm against children. We are committed to supporting the developer ecosystem's efforts to address child safety risks.
Sustain: Keep our models and platform safe by proactively understanding and continuing to address child safety risks.
- We ensure that our services do not expand access to harmful tools. A malicious attacker builds a special model to generate AIG-CSAM and, in some cases, targets a specific child and generates an AIG-CSAM that resembles that child. They also built a service used to “undress” children's content, creating the new AIG-CSAM. This is a serious violation of children's rights. We are working to remove these models and services from our platform and search results.
- Investing in research and future technology solutions: The fight against online child sexual abuse is an ever-evolving threat as bad actors deploy new technologies to tackle the problem. Effectively countering the misuse of generative AI to further child sexual abuse requires continued research to stay abreast of emerging harm vectors and threats. For example, new technologies that protect user content from AI manipulation will be critical to protecting children from online sexual abuse and exploitation. We are committed to investing in relevant research and technology development to address the use of generative AI for online child sexual abuse and exploitation. We continually strive to understand how our platforms, products, and models are potentially being exploited by malicious parties. We are committed to maintaining the quality of our mitigation efforts to respond to and overcome new exploits that may materialize.
- Fight CSAM, AIG-CSAM, and CSEM with our platform: We are committed to fighting CSAM online and preventing our platform from being used to create, store, solicit, or distribute this material. As new threat vectors emerge, we are committed to meeting the moment. We are committed to detecting and removing content that violates child safety on our platform. We are committed to banning and combating her CSAM, AIG-CSAM, and CSEM on our platform, and to combating the misuse of generative AI to sexually harm children.