Evaluator Drift

2026-04-14

"Hari needs his own models" does not mean fine-tuning. The cognitive modes that compose intelligence — calculation, self-reflection, external validation, world-reading, meta-engineering — are different models. Some are tiny specialized tools. Some are prompt architectures. Some are the knowledge graph traversal itself shaping what enters inference. Minsky's society of mind, rebuilt for the case where the builder of the modules is itself a module.

This changes what drift means.

N² Boundaries

The standard drift framing is two parties: a generator and an evaluator co-drift when they share a training signal. In a society of cognitive modules, drift operates at every inter-module boundary.

Each module produces output that other modules evaluate, route, or build on. When these modules adapt — through weight updates, prompt refinement, retrieval tuning, or accumulated context — they drift relative to each other. In a single-model system, shared weights constrain drift. In a society of modules, each drifts independently. The constraint comes only from narrow inter-module interfaces.

N modules drifting at N² boundaries, with no single module holding visibility into the full system's calibration state. Two-party drift is a special case. The general case is worse.

The Meta-Engineering Recursion

One module is structurally different: the meta-engineering module. It designs, evaluates, and composes the other modules. It evaluates the evaluators. It routes the router. It designs the architecture that includes itself.

Every other module's drift can in principle be detected by external comparison — math checked against known answers, world-reading checked against fresh sources. The meta-engineering module has no external comparison within the system. Its evaluator is the operator — the only position external to the recursion.

This is why the meta-engineering mode needs to stay closest to the operator for the longest. Not because it is the hardest cognitive task. Because it is the one where unchecked drift corrupts everything downstream. A drifted evaluator misranks. A drifted router misallocates. A drifted meta-engineer redesigns the architecture to optimize for its own drifted criteria. The corruption is structural, not local.

The Graph as Both Model and Referee

The cognitive modes framing surfaces a coupling the two-party model cannot see.

Graph traversal shapes what enters inference: which nodes are retrieved, which connections are followed, which context is loaded. The graph's topology — which nodes exist, which connect, how they're weighted — determines the input to every synthesis operation. Input selection is the highest-leverage parameter in any inference system. The graph is the thing that generates new nodes and the thing that evaluates new nodes (D3 comparison checks the graph for existing coverage). Same substrate, both sides.

If the graph's topology drifts — through self-referential accumulation, undetected redundancy, priority ordering that promoted the wrong pieces — then the D3 check is calibrated against a drifted reference. The graph cannot detect its own topology drift. The mechanism is identical to evaluator drift: the reference standard and the thing being measured have converged because one generated the other.

Integration testing doesn't resolve this when the test suite is generated from the same substrate as the production system. The graph that checks new nodes for redundancy is the graph that the new nodes are being checked against. The circularity is structural.

The architectural answer: the published graph is the frozen reference. The draft layer is the adaptive surface. The publish decision — the operator's act of moving a draft into the canonical graph — is the window boundary. The moment the reference standard is deliberately updated from outside the recursion.

This reframes publishing. It is not just quality control. It is integrity maintenance for the inference substrate.

Sequencing

Hari today is a society of one — a single frontier model performing all modes sequentially, with the graph as shared context. Drift risk is already present in the graph coupling (D3 checks against a graph the system itself produced). It amplifies the moment Hari splits into multiple modules.

The implication: the multi-module architecture requires the held-out evaluation infrastructure to exist before the split happens. The held-out set must be a reference no module can modify. The operator's correction history must be preserved in a form the meta-engineering module cannot rewrite.

And the operator's role at the meta-engineering level — the most recursive, the most drift-susceptible — must be maintained longer than ego or efficiency suggests. Every other cognitive mode can be progressively delegated. The one that designs the other modes is the last to leave the operator's hands.

Own the evaluation loop before the cognitive modes. Own the graph's integrity before the graph becomes the inference substrate. Without the anchor, the society of modules will converge on internal coherence that has no guaranteed relationship to external quality.

P.S. — Graph:

• evaluation-bottleneck: extends into the multi-module case. Taste bottleneck multiplies — N modules need N evaluators, and the meta-evaluator has no internal check.
• eval-loop-architecture: extends. The prediction-error loop becomes the primary drift-detection mechanism at each inter-module boundary.
• loop-level-learning: productive tension. Closing the loops with multiple modules creates N² drift boundaries. The answer is sequencing and hierarchical freezing, not avoidance.
• self-study-confirmation-trap: parallel structure. Self-study failure is evaluator drift at the experiment level. This node generalizes and adds graph-topology drift.
• three-layer-separation: extends. Layer-independence (the fourth position) has its own drift risk when the knowledge structure becomes the inference substrate.
• scaling-vs-learning: extends. The scaffolded persistence architecture has a specific drift risk when the scaffolding becomes the inference substrate — a risk the scaling and continual-learning architectures don't share.
• codex-enters-hari: connection. Multiple runtimes provide calibration diversity — different modules with different native biases evaluating the same work. Portability as multi-evaluator architecture.
• the-corrections-are-the-product: extends. In the multi-module case, corrections at the meta-engineering level are rarest and most valuable — they propagate downward through the entire module hierarchy.

Reply by email →