This week, authorities from the UK, EU, US and seven other nations met in San Francisco to launch the “International Network of AI Safety Institutes”.

The meeting, which took place at the Presidio Golden Gate Club, addressed managing the risks of AI-generated content, testing foundational models and conducting risk assessments for advanced AI systems. AI security institutions from Australia, Canada, France, Japan, Kenya, the Republic of Korea and Singapore have also officially joined the network.

In addition signing a mission statementmore than $11 million in funding was awarded to research into AI-generated content, and the results of the Network’s first joint safety testing exercise were reviewed. The participants included regulatory officials, AI developers, academics and civil society leaders to help the discussion on emerging AI challenges and potential safeguards.

The convening built on the progress made at the previous AI Security Summit in May, which took place in Seoul. The 10 nations agreed to “promote international cooperation and dialogue on artificial intelligence in light of its unprecedented advances and its impact on our economies and societies.”

“The International Network of AI Security Institutes will serve as a forum for collaboration, bringing together technical expertise to address AI security risks and best practices,” according to the European Commission. “Recognizing the importance of cultural and linguistic diversity, the network will work toward a unified understanding of AI security risks and mitigation strategies.”

Member AI safety institutions will need to demonstrate their progress in AI safety testing and evaluation through the Paris AI Impact Summit in February 2025 in order to move forward with discussions on regulation.

Key outcomes of the conference

Mission statement signed

The mission statement commits the network members to work together in four areas:

1. Research: Collaborate with the AI ​​security research community and share findings.
2. Test: Develop and share best practices for testing advanced AI systems.
3. Guidance: Facilitate shared approaches to interpreting AI safety test results.
4. Inclusion: Share information and technical tools to broaden participation in AI safety science.

More than $11 million has been allocated to AI safety research

In total, Network members and various nonprofits have announced more than $11 million in funding for research to mitigate the risk of AI-generated content. Child sexual abuse material, non-consensual sexual imagery, and the use of AI for fraud and impersonation were highlighted as key areas of concern.

Funding will be awarded as a priority to researchers investigating digital content transparency techniques and model protection measures to prevent the generation and distribution of harmful content. Awards will be considered for scientists developing technical mitigations and social science and humanistic assessments.

The American institute also has a range of voluntary approaches to address the risks of AI-generated content.

Discuss the results of a joint test exercise

The network has completed its first ever joint test exercise on Meta’s Llama 3.1 405B, looking at its general knowledge, multi-lingual capabilities and closed-domain hallucinations, where a model provides information from outside the area it was instructed to refer to . on.

The exercise raised several considerations on how to improve AI safety testing across languages, cultures and contexts. For example, the impact that minor methodological differences and model optimization techniques can have on evaluation results. Broader joint testing exercises will take place before the Paris AI Action Summit.

Shared basis for risk assessments agreed

The network agreed on a shared scientific basis for AI risk assessmentsincluding that they must be actionable, transparent, comprehensive, multi-stakeholder, iterative and reproducible. Members discussed how this could be operationalised.

US ‘Testing Risks of AI for National Security’ Task Force established

Finally, the new TRAINS Task Force was established, led by the US AI Security Institute, and included experts from other US agencies, including Commerce, Defense, Energy and Homeland Security. All members will test AI models to manage national security risks in domains such as radiological and nuclear security, chemical and biological security, cyber security, critical infrastructure and military capabilities.

SEE: Apple joins US government’s voluntary commitment to AI safety

This reinforces how top-of-mind the intersection of AI and the US military is. Last month, the White House published the first ever. National Security Memorandum on Artificial Intelligencewhich ordered the Department of Defense and US intelligence agencies to accelerate their adoption of AI in national security missions.

Speakers addressed balancing AI innovation with safety

US Commerce Secretary Gina Raimondo delivered the keynote address on Wednesday. She told the attendees that “advancing AI is the right thing to do, but advancing as fast as possible just because we can, without thinking about the consequences, is not the smart thing to do not,” according to TIME.

The battle between progress and safety in AI has been a bone of contention between governments and tech companies in recent months. While the intent is to keep consumers safe, regulators run the risk of limiting their access to the latest technologies, which have tangible benefits. Google and Meta have both openly criticized European AI regulation, citing the region’s AI law, suggesting it would destroy its innovation potential.

Raimondo said that the US AI Safety Institute “is not stifling innovation,” according to AP. “But here’s the thing. Safety is good for innovation. Safety breeds trust. Trust accelerates adoption. Adoption leads to more innovation.”

She also emphasized that nations have an “obligation” to manage risks that can negatively affect society, such as by causing unemployment and security breaches. “Let’s not let our ambition blind us and allow us to sleep in our own doom,” she said via AP.

Dario Amodi, the CEO of Anthropic, also gave a speech that emphasized the need for safety testing. He said that while “people today laugh when chatbots say something a little unpredictable,” it shows the importance of getting control over AI before it acquires more sinister capabilities, according to Fortune.

Global AI safety concerns have come to the fore in the past year

The first meeting of AI authorities took place about a year ago in Bletchley Park in Buckinghamshire, UK. This saw the launch of the UK’s AI Safety Institute, which has the three primary aims:

• Evaluation of existing AI systems.
• Conduct fundamental AI security research.
• Share information with other national and international actors.

The US has its own AI Security Institute, formally established by NIST in February 2024, which has been named the network’s chairman. It was created to work on the priority actions outlined in the AI ​​Executive Order issued in October 2023. These actions include the development of standards for the safety and security of AI systems.

SEE: OpenAI and Anthropic Token deal with US AI Safety Institute

In April, the UK government formally agreed to work with the US in developing tests for advanced AI models, largely by sharing developments made by their respective AI safety institutes. In an agreement made in Seoul, similar institutions were created in other countries that joined the collaboration.

Clarifying the U.S. position toward AI safety with the San Francisco conference was especially important, as the larger nation currently does not offer an overwhelmingly supportive stance. President-elect Donald Trump has promised to rescind the Executive Order when he returns to the White House. California Governor Gavin Newsom, who was in attendance, also vetoed the controversial AI regulation bill SB 1047 in late September.