The U.S. Artificial Intelligence (AI) Safety Institute, part of the National Institute of Standards and Technology (NIST) under the U.S. Department of Commerce, has announced agreements with AI companies Anthropic and OpenAI. These agreements are designed to facilitate collaboration on AI safety research, testing, and evaluation.
The Memoranda of Understanding with each company outline a framework allowing the U.S. AI Safety Institute access to new models from Anthropic and OpenAI before and after their public release. This access is intended to enable joint research efforts focused on evaluating the capabilities and safety risks of AI models, and on developing methods to mitigate these risks.
These new agreements come on the back of NIST developing the Dioptra testing framework for AI models to help ensure that AI models are safe. The Dioptra testing framework is focused on the technical safety of the models – can prompts be hijacked by malicious actors? Can the model be poisoned by malicious actors? So I would expect that NIST testing of the OpenAI and Anthropic models falls into the same area, where the models are tested more for technical security than content output.
Elizabeth Kelly, director of the U.S. AI Safety Institute, stated that these agreements mark an important milestone in advancing the science of AI safety. The U.S. AI Safety Institute will also provide feedback to Anthropic and OpenAI on potential safety improvements, working in collaboration with the U.K. AI Safety Institute.
The U.S. AI Safety Institute’s efforts are part of NIST’s longstanding mission to advance measurement science, technology, and standards. The evaluations conducted under these agreements aim to further the safe, secure, and trustworthy development and use of AI, aligning with the goals set out in the Biden-Harris administration’s Executive Order on AI.
For more information, check out NIST’s announcement here.
