In the Room: What Twenty Years of Embedded Oversight Taught Me About “Embedded Evaluators”

In his September 2026 essay We Must Pace the Frontier, Dario Amodei proposed that frontier AI companies give a team of third-party “embedded evaluators” ongoing, employee-like access: desks in the office, access badges, company laptops, and permissions broadly comparable to the company’s own internal risk teams. Their job would be to verify adherence to safety practices and commitments, report incidents, and help assess not only finished models but the training pipelines and processes that produce them. Anthropic said it would make that commitment unilaterally, and almost overnight “embedded evaluator” entered the AI governance vocabulary.

Organizations such as METR and Apollo Research have mostly worked at the model level: intensive testing of a system at a point in time, typically before a release. Embedded evaluation is a different kind of engagement, and the difference goes well beyond the length of the contract.

Having independent third parties embedded inside an organization is unusual in AI, but it is routine elsewhere. Federal inspectors work every shift in meat processing plants, and nuclear power plants host resident inspectors from the regulator. A less-known example, and the one I know best, is large state government IT projects.

When a state government builds or replaces one of its largest systems, such as a Medicaid eligibility platform, a child welfare case management system or an unemployment insurance system, it commonly hires an Independent Verification and Validation (IV&V) provider to watch the project from the inside. For some programs this is a condition of federal funding. The IV&V firm is separate from both the state’s project team and the system integrator doing the build, and it reports on project health, risks and quality to state leadership and often to federal partners.

I did this work for more than twenty years before joining PACT AI. Some engagements were periodic assessments lasting a few weeks or months. Many others were long-term embedded engagements where I had an access badge and an assigned desk inside the client organization for several years.

I’ll use the word oversight for the embedded role rather than evaluation. Day to day, the work can include investigation, evaluation, assessment and audit tasks, but its main thrust is watching an organization as it goes about its work. It is closer to overseeing than to inspecting.

Based on that experience, there are two big differences between periodic assessment and embedded oversight.

1. You see the process, not just the product

A periodic assessor mostly sees outputs: the deliverable, the test results, the system as it stands on the day of the review. An embedded overseer can potentially see the whole process, from the first planning conversations through design, development and multiple layers of testing. (For an AI lab, read that last part as multiple stages of training and evaluation.)

That visibility does two things. First, it gives much deeper insight into the constraints and trade-offs that actually drive decisions. Second, it lets you spot risks early, while they are easier to fix. The quality management movement taught us decades ago that it is far better to build quality into a process than to try to inspect it at the end. 

How much of this you get depends on the access you’re given. In my experience, though, it was not unusual for the embedded IV&V team to be invited to essentially every meeting that wasn’t specifically designated confidential. Once people get used to you, you become part of the furniture, and that is when you start seeing how decisions really get made.

2. Independence becomes a different kind of challenge

Being embedded with a single organization brings both opportunity and risk.

The opportunity is that the trust, access and institutional knowledge you build up far exceed what’s possible when you only engage for a few weeks at a time. 

The risk is that familiarity is a real threat to independence. The audit profession even has a name for it: the familiarity threat. Spend years alongside a team and it becomes easy to identify with them, to start thinking “we” instead of “they,” and to lose the separation and professional skepticism the role depends on.

Professions such as financial audit and Nuclear Regulatory Commission inspection have established ways to manage this:

• Rotation. Move individual staff off the engagement after a set period, much as audit firms rotate engagement partners.

• Layered quality review. Have reports reviewed by people who are not embedded in the client organization, so drift in tone or objectivity gets caught by someone without the same relationships.

• Culture and incentives. The oversight team needs a professional ethos that puts integrity and independence first, and the firm employing them has to reinforce that through how it rewards and promotes people.

The subtler problem: when they start asking for your advice

There’s a second independence challenge that is harder to see. People often use independence, objectivity and impartiality as if they were synonyms, but they mean different things:

• Independence means being free of undue influence, and also not reviewing your own work.

• Objectivity means basing conclusions on factual evidence.

• Impartiality means treating all sides, parties and individuals equally, without bias or favoritism.

Here’s what happens on a long embedded engagement. You sit in on decision-making week after week. If you’ve shown yourself to be competent and trustworthy, the organization starts turning to you as a source of unbiased advice. Two teams disagree and someone asks what you think. You notice a mistake about to be made and you have the chance to speak up before it happens.

The moment you weigh in, you are no longer impartial. You’ve taken a side, or helped shape a decision, and very likely improved the outcome. But you’ve also given up a small piece of your independence, because you are now a party to that decision. When you later assess how it turned out, you can’t be fully objective, since you’d be reviewing your own work. Auditors call this the self-review threat.

That trade-off can be worth it. Sometimes it is the most valuable thing embedded oversight delivers. But it is a choice, and it should be made deliberately.

Any single choice like this seems minor. Added up over a year or more, they can make the oversight team much more valuable to the organization while steadily eroding the team’s independence. Good oversight teams confront this dilemma early in an engagement and agree on how they will handle it, rather than discovering after two years that they have drifted into being advisors.

What this means for AI’s embedded evaluators

Amodei’s proposal gets several of the big structural pieces right: ongoing access, a scope that covers processes and not just models, and a right to publish findings without the company’s editorial control. Those are necessary. My experience suggests they are not sufficient, and that the hardest problems will show up in the day-to-day texture of the work:

1. Access to process is where the value is. Embedded evaluators should negotiate for access to the meetings and conversations where decisions get made, not just to artifacts and models after the fact.

2. Independence has to be designed in from day one. Rotation, review by people outside the embedded team, and an employer whose incentives reward integrity matter as much as contractual publication rights.

3. Decide in advance when to advise. Teams should set explicit rules for when they’ll offer input and record when they have, so later findings can disclose where the evaluators were party to the decisions they’re assessing.

None of this is new. Bank examiners, nuclear resident inspectors and IV&V providers have been working through these tensions for decades. The AI field doesn’t need to invent embedded oversight from scratch; it can learn from those who have already done it.