OpenAI, Anthropic agree to have their models tested before making them public [Video]

OpenAI and rival company Anthropic have signed agreements with the U.S. government to have new models tested before public release.

On Thursday the National Institute of Standards and Technology (NIST) announced that its AI Safety Institute will oversee “AI safety research, testing and evaluation” with both companies. “These agreements are just the start, but they are an important milestone as we work to help responsibly steward the future of AI,” said Elizabeth Kelly, director of the AI Safety Institute in the announcement.

It’s no secret that generative AI poses safety risks. Its tendency to produce inaccuracies and misinformation, enable harmful or illegal behavior, and entrench discrimination and biases is well documented at this point. OpenAI has its own internal safety testing, but has been secretive about how its models work and what they’re trained on. This is the first instance of OpenAI opening up access to third party scrutiny and accountability. Altman …