The funding will enable the start-up build large-scale experimental infrastructure for exploring and boosting the safety properties of computationally intensive AI models
AI safety and research start-up Anthropic has raised $704m in its Series A and B rounds. (Credit: Gerd Altmann from Pixabay)
Anthropic, a California-based artificial intelligence (AI) safety and research company, has secured $580m in a Series B round.
The funding round was led by FTX CEO Sam Bankman-Fried. It also saw the participation of Caroline Ellison, Nishad Singh, Jaan Tallinn, Jim McClave, and the Center for Emerging Risk Research (CERR).
Established in early 2021, Anthropic is said to have carried out research into making AI systems that are more robust, steerable, and interpretable.
When it comes to interpretability, the company is said to have made progress in mathematically reverse engineering the behaviour of small language models. The company has also started to understand the source of pattern-matching behavior in large language models.
Anthropic claims to have developed baseline techniques for steerability and robustness for making large language models more “helpful and harmless”. This has been followed up by reinforcement learning to enhance the properties further.
Besides, the company has released a dataset for helping other research labs train models that are more in line with human preferences.
The company said that the Series B financing will enable it in building large-scale experimental infrastructure for exploring and boosting the safety properties of AI models that are computationally intensive.
Anthropic co-founder and CEO Dario Amodei said: “With this fundraise, we’re going to explore the predictable scaling properties of machine learning systems, while closely examining the unpredictable ways in which capabilities and safety issues can emerge at-scale.
“We’ve made strong initial progress on understanding and steering the behavior of AI systems, and are gradually assembling the pieces needed to make usable, integrated AI systems that benefit society.”
Anthropic has a team of nearly 40 people and has plans to grow it further this year.
The start-up had secured $124m in a Series A financing round, which was held last year. The Series A round was led by Jaan Tallinn, a technology investor as well as co-founder of Skype.
Anthropic co-founder and president Daniela Amodei said: “Now that we’ve built out the organisation, we’re focusing on ensuring Anthropic has the culture and governance to continue to responsibly explore and develop safe AI systems as we scale.”