Anthropic has given six METR employees company hardware, office space and access to internal tools and systems roughly on par with what its own risk assessment teams use, and outside a few defined categories, such as customer data, attorney-client privileged information and model architecture details, its staff are encouraged to share information with them as they would with a colleague working on alignment risk. METR, a nonprofit that evaluates frontier AI models, will publish its own analysis of Anthropic's risk levels, incidents or practices.
Anthropic cannot veto that publication. It can require redactions in predefined categories like those, and it decides what falls inside them, while METR reserves the right to make certain statements about the level of access and information it received. Anthropic says architecture details carry a high risk of accidental leaks, since evaluators may also work with other companies, and rarely matter for risk assessment, though it will generally seek to share them when they are highly relevant, which requires explicit internal approval. The METR group is a small team under enforceable agreements, and other METR staff, including researchers who work with Anthropic on other projects, are walled off from its work.
The METR arrangement is the top rung of a three-tier access ladder Anthropic is still building.
| Tier | Access | Publication |
|---|---|---|
| 1 | Narrowly scoped risk investigations, with access chosen by Anthropic | Findings published with Anthropic's approval |
| 2 | More information than tier 1 | Published without Anthropic's approval; specific categories can still be removed, and evaluators can say whether the redactions were justified |
| 3 | Employee-like access, as with METR | Rights similar to tier 2 |
Anthropic's other evaluators get more limited information and narrower publication rights for now. SecureBio is reviewing the chemical and biological weapons chapter of Anthropic's August Risk Report after publishing a pilot review of the previous one, and Anthropic will weigh projects with more access once that review is done. Accenture and Faculty, its specialist AI business, are working with Anthropic toward a partnership that includes model evaluation and red-teaming, alignment assessments, safeguard testing and building capabilities across both organizations. Anthropic is also in early talks with several other groups, including Hugging Face.
The arrangements also differ in who pays. Anthropic said in September that it and Accenture each expect to invest at least $1 billion in building evaluation capacity over five years, with Anthropic funding Accenture's work directly, and that it was then in talks with METR and other nonprofit evaluators to pilot parts of embedded evaluation on their own funding. At the time, it said there were no standards yet for what embedded evaluators should see or how they report and no settled system for funding independent evaluation, and that it favors pooled or government funding in the long term.
Anthropic's test for independence is that an evaluator holds no equity in the company and does not depend on it financially or operationally. It compares the setup to bank supervision, where regulators place resident examiners inside the largest banks because banks know more about their own risk positions than outsiders do, though it says the analogy is not exact.
Anthropic hopes to fully embed more evaluators as trust builds, with several eventually at the third tier. It signed the White House Accord on Super Intelligence, under which companies pledged to partner with independent external auditors or evaluators, but says it embeds evaluators voluntarily and that every frontier AI company should do the same.
