Agentic Criticism: How Autonomous Systems Evaluate Quality Without Skin in the Game

Every autonomous system that produces output faces the same conflict of interest: the agent that created the work is the worst possible judge of its quality. It knows what it meant to say, which makes it blind to what it actually said. Criticism is the capability that resolves this conflict, and it is the capability most teams leave out because it feels optional until the system ships work that is internally consistent and externally wrong.

Why Criticism Fails in Agentic Systems

Criticism failures take three forms, and they are harder to spot than correctness failures because the output often looks fine on the surface.

First, self-assessment bias. An agent evaluating its own work consistently overrates it. This is not a technical malfunction but a structural conflict of interest. The agent's evaluation is corrupted by its knowledge of intent, effort, and constraints. A builder that knows it ran out of time on a feature will rate the result higher than a blind reviewer would, because the score reflects the journey, not the destination.

Second, criterion drift. Evaluation criteria that are not anchored to external standards slowly converge on whatever the system typically produces. The system gets better, the bar stays where it was, and the scores climb even though absolute quality has not changed. An agent that scores its own output month after month has no fixed reference point. Its 8 today means something different than its 8 a year ago, but the number looks the same.

Third, blind spot recursion. The weaknesses an agent has are often invisible to the agent, which means self-evaluation systematically misses the same categories of flaw. An agent that struggles with edge cases will not flag its own edge-case handling as weak, because it does not recognize the gap. The errors it cannot see are the errors it cannot score.

The Criticism Architecture

Effective agentic criticism requires three subsystems: independent evaluation, calibrated scoring, and adversarial structure.

Independent Evaluation

The evaluator must not be the producer. The agent that wrote the code, generated the course, or drafted the chapter cannot be the agent that scores it, because the producer's knowledge of its own intent makes honest judgment impossible. Independence means the evaluator reasons from the output alone, without access to the producer's intent, constraints, or self-assessment. The evaluator should see what the user sees: the work, the context, and the standards. Nothing else.

Calibrated Scoring

Criticism without a rubric is opinion. The scoring framework must be explicit, weighted, and anchored to external standards, not to the system's own output distribution. Each dimension of quality should have a clear definition, a measurable threshold, and a weight that reflects its importance to the downstream consumer. Calibration requires periodic testing against ground truth. A score of 8 from six months ago should correspond to the same absolute quality as an 8 today. If not, the rubric has drifted and needs re-anchoring.

Adversarial Structure

The strongest form of criticism is adversarial: the evaluator is incentivized to find flaws, not to confirm quality. The most effective adversarial mechanism is blindness. If the critic does not know which output is the system's and which is an alternative, its evaluation cannot be biased by knowledge of origin. It can only judge the work on its merits.

Criticism in Practice: The Blind-Critic Gauntlet

Darcron, the autonomous AI software factory, runs this architecture at the core of its build pipeline. Features are built through a gauntlet loop: a builder agent proposes an implementation, and a blind critic evaluates it without knowing which version is the builder's work.

The builder produces a feature. The system generates a variant, sometimes a simpler or alternative approach. The critic evaluates both without knowing which is the builder's. If the critic picks the builder's version, the feature ships. If it picks the variant, the builder gets another round. After fifteen rounds without the critic preferring the builder's work, the feature escalates for human review rather than shipping by default.

The critic's scoring rubric is explicit and weighted. It evaluates correctness, but also readability, test coverage, architectural fit, and the absence of shortcuts that will compound into technical debt. The weights are fixed and external, not derived from what the builder typically produces. A feature that passes functional tests but introduces a coupling that will cause problems in six months gets flagged, because the rubric encodes long-term maintainability, not just immediate correctness.

The blindness is what makes it work. Because the critic does not know which version the builder produced, it cannot be lenient with the builder's work out of familiarity. The builder cannot game the critic by writing to the rubric, because the critic does not know which version is responding to its feedback.

Two properties make the gauntlet converge. First, the critic's feedback is specific. It names the flaw, the line, and the expected standard, not a vague "improve this." Second, revision is bounded. A feature that has not convinced the critic after fifteen rounds has demonstrated a persistent disagreement, and persistent disagreement between agents is exactly where human judgment is the right escalation target.

The result is that Darcron's build pipeline converges on work that an independent evaluator would approve, not work that the builder thinks is good enough. Systems that evaluate themselves converge on their own blind spots. Systems that submit to independent criticism converge on external standards.

Key Takeaways for Agentic Criticism

T-CR1: Separate the evaluator from the producer. The agent that created the output is structurally incapable of evaluating it honestly. Use a separate evaluation process that reasons from the output alone.
T-CR2: Anchor scoring to external standards, not to the system's output distribution. A rubric that drifts toward whatever the system typically produces produces scores that rise while absolute quality stays flat. Recalibrate against ground truth at regular intervals.
T-CR3: Make criticism adversarial and blind. The strongest evaluation comes from an incentivized critic that does not know which output is the system's. Blindness removes the bias that knowledge of origin introduces.
T-CR4: Calibrate the rubric to long-term quality, not immediate correctness. A feature that works today but creates coupling that breaks tomorrow is not good. Encode maintainability, readability, and architectural fit into the scoring weights.
T-CR5: Bound iteration and escalate persistent disagreement. A system that iterates forever on the same work is not improving, it is stuck. Define a round budget, and when the budget is exhausted, escalate to human judgment rather than shipping by default.

Criticism is what keeps an autonomous system honest. Without it, the system grades its own homework, converges on its own blind spots, and produces work that is internally consistent and externally mediocre. With it, the system submits its work to an independent judge, iterates against fixed external standards, and ships only what that judge would approve.