A day after an Anthropic researcher quit over AI’s apocalyptic potential, two tech-savvy brothers-in-law have launched a startup that aims to bring AI safety into the enterprise fold. Artificial Intelligence Underwriting Company (AIUC) has secured a hefty $40 million Series A round, with a vision to audit and certify AI agents to prevent rogue behavior.

Rune Kvist, an early Anthropic employee, and Rajiv Dattani, former COO of METR, are redefining AI risk management. By adopting the cybersecurity model, AIUC has developed a standard called AIUC-1, which involves running AI agents through a suite of 5,000 tests to ensure they behave safely in various scenarios. The final audit is verified by humans, even as AI powers the testing process.

The company’s approach is catching the attention of A-list investors, who see potential in AIUC’s ability to provide independent assessments of AI agent safety. This is a step towards ensuring that AI tools are reliable and secure, especially as they become more prevalent in critical industries. For now, the key is to identify and address the vulnerabilities before they escalate into real-world issues.

While METR’s recent work has been focused on performance, AIUC’s broader goal is to provide a comprehensive safety net for enterprises. The duo hopes that by making AI agents more accountable, they can help prevent the sort of rogue behavior that has caused concern within the AI community.