Nathan Lambert and Tom Zick have founded Trillium Labs, a nonprofit dedicated to openly researching areas of AI, including potentially risky ones like recursive self-improvement. They argue that keeping AI research behind closed doors reduces the community’s ability to scrutinise and contribute to mitigating risks.

Their approach is to publish the details of experiments, allowing outside scientists to study and replicate them. Lambert believes this transparency is crucial, referencing the scientific method as a tool for mitigating harms and building better futures. In contrast, big AI companies like OpenAI and Anthropic keep their models locked away, with limited transparency.

Trillium Labs will focus on post-training fine-tuning and reinforcement learning, studying how these processes shape the character and behavior of AI models. They hope to contribute to a wider discourse on building AI safely and effectively, aiming to raise $40 to $100 million over the next 18 months.

Tim Fist, a policy expert, supports the idea of greater transparency in AI research, stating that it could yield surprising insights. Lambert and Zick hope their work will add nuance to the ongoing debate on AI development.