Where Jev Fits in an Agent Loop

1. Make the judgment a callable operation

The useful abstraction is simple: describe the visible state, name the question, and specify the alternatives. A tool selector can ask which available tool fits the request. A context manager can ask whether a passage supports a requirement. A monitor can ask whether the current evidence warrants further inspection.

The same interface can support binary judgments, choices, and ordered ratings. Different implementations sit behind it: Kev uses a pointer-style readout over an adapted language model, while Nimble documents direct answer-token scoring. The API shape and the execution strategy are separate choices. Exposing a decision endpoint does not by itself mean the runtime shares context across questions.

This separation makes the component easier to inspect. “The model thinks the passage is relevant” and “the program should delete every other passage” become two different decisions. One is a semantic estimate; the other is a policy with consequences.

It also gives uncertainty somewhere to go. A distribution can determine whether to accept a suggestion, request more evidence, or use a fallback. The threshold belongs to the application. It should reflect the costs of a missed issue and an unnecessary interruption.

Diogo Almeida’s Latent Space interview helped sharpen this framing for me: the ambition is to place many small, software-consumed judgments inside products. I find that more interesting than asking a decision model to impersonate an entire agent.

2. A small answer still needs enough evidence

One of our first instincts was to build a progress monitor. “Is the agent making progress?” sounds like a compact question. In practice, it asks the model to understand the objective, prior attempts, the environment, and whether another attempt is worthwhile. Shortening the answer to a boolean leaves that inference problem intact.

A more useful starting point is an explicit relation: does this passage support this requirement? Or: is this proposed operation compatible with the stated scope? The question names both the evidence and what should be inferred from it. If a timestamp, hash, or exact permission check settles the relation, I would use code instead.

Candidate construction is part of this evidence problem. A decision model cannot choose an answer that the surrounding system never offered. In a small SQuAD2 extraction diagnostic, our proposal stage supplied the exact reference span for only 12 of 32 answerable questions. Direct language-model extraction reached normalized exact match on 32/40 total cases; proposal plus decision selection reached 16/40 and took longer. The selector was downstream of the main bottleneck.

This changed how I think about “none of the above.” It makes a deficient candidate set visible, but the program still needs a way to expand the set, retrieve more evidence, or hand the task back to a generator. A well-designed decision interface should expose that recovery path.

There is a related distinction between false, not mentioned, and unknown. On a document checklist, we asked our decision model to evaluate all 17 ContractNLI hypotheses for each of 61 development contracts. It answered 824/1,037 judgments correctly, yet only one contract had a completely correct checklist. Of 423 “not mentioned” labels, 130 were incorrectly converted into support or contradiction. Local accuracy did not make the whole document safely verified.

3. Customization can give the operator a useful scope

Task-specific adaptation was one of the clearest positive signals in our exploration. A form-action specialist completed a controlled form loop that the general-purpose controller often failed. The state described the form and the available actions; this was structured-state action selection, not a claim about arbitrary screenshot-based browser use.

A separate constraint specialist learned a useful distinction between compatible actions, incompatible actions, and insufficient information. On 12 new cases from four held-out scenario families, we varied wording, names, and option order to produce 432 views. Accuracy improved from 294/432 to 351/432. Much of the gain came from recognizing when the supplied information did not justify an answer.

The form and constraint specialists are different models. The form tests share authored task templates; the constraint views are repeated measurements of 12 cases, not 432 independent tasks.

The attraction is that an application can own the meaning of its operator. “Choose the next form action” or “check compatibility with this scope” has a concrete role in the software. Once that role is stable, a specialist can be valuable without becoming a better general assistant.

There was an important second result. After constraint adaptation, mean accuracy on our frozen, adapted Nimble, VitaminC, and MASSIVE subsets moved only from 83.26% to 82.70%, but every subset’s negative log-likelihood, Brier score, and calibration error worsened. Retaining the right answer most of the time did not preserve the probability quality that a threshold-based policy would use.

I would therefore define a specialist’s operating scope explicitly and evaluate its probabilities inside that scope. Outside it, the general controller or another fallback remains available. And I would diagnose the mismatch before choosing how to customize: some tasks need new semantic distinctions, while others need better evidence or a different threshold. Parameter updates are only one of these tools.

4. The workload decides whether “fast” is useful

Prefill-only execution avoids decoding an answer token by token. It still pays to read the state, process question and option descriptions, dispatch requests, and handle caches. The opportunity becomes especially interesting when several judgments genuinely need the same state.

We tested eight short public passages, each with 16 original questions. At 16 questions, cold shared-state execution reduced mean request time from 1,324 ms to 711 ms. At one question, the shared path instead increased time from 106 ms to 127 ms. Reuse has overhead, so the number of required judgments matters.

Now compare that with tool selection. A list of ten tools does not necessarily imply ten required answers. A generator can read the whole list and emit one identifier. A candidate-wise scorer may perform a judgment for every tool, even though the application only needs one choice.

These are reference-runtime diagnostics, not production speed benchmarks. The decision model used FP32 and the generator BF16; model loading was excluded. Shared execution preserved recorded argmax answers but had probability drift up to 0.00093329 and failed numerical-equivalence checks.

The lesson I take from these measurements is to count the answers the application actually needs. Dense checklists and repeated record coding are plausible targets for state reuse. A single choice from many candidates is a different workload, and a well-engineered generator can be competitive.

The same accounting applies to context compression. A selector can reduce the reader’s input while adding its own inference, a serial round trip, and later readback. In our recoverable-context probe, the selected-context path answered 14/24 questions correctly versus 13/24 for full-context reading, but took 1,416 ms versus 270 ms. The extra answer did not come with an efficiency win. The public pi-jev validation similarly separates reduced input tokens from end-to-end wall time.

5. Use judgments to allocate the next work

The checklist result points to a more promising use than having the model certify that a document is finished. A decision vector can tell the system where to spend effort next.

Imagine a research agent supporting several claims from a document collection. For each claim, the operator returns a status: supported, conflicting, or missing. The program retains supporting evidence, investigates conflicts, and retrieves material for gaps. The vector becomes a work queue.

The consequences of errors are different here. An unnecessary “missing” judgment may cause an extra retrieval. A mistaken “supported” judgment may end investigation prematurely. The program can budget additional searches and reserve a separate acceptance check for consequential claims.

This suggests a useful integration sequence: observe decisions in shadow mode, compare disagreements, then let the operator advise or direct a bounded action. The mu harness makes active, shadow, and disabled modes explicit. That is a practical way to learn what a controller changes before assigning it broad authority.

Recoverability matters beyond retrieval. A context manager can hide evidence from the next prompt while retaining a path to read it back. A tool triage component can request an argument check before execution. A recovery component can suggest a next diagnostic rather than declare the root cause solved.

Semantic advice and enforcement also need distinct paths. In our executable constraint probes, a model sometimes recognized a forbidden operation while the generator still executed it. Machine-checkable permissions and limits need enforcement in code. The semantic operator helps with the residual judgment; it does not grant permission.

6. A persistent observer needs a state contract

The idea of a resident, low-latency judgment component remains compelling to me. It could watch changes in a workspace, tool results, or an evidence collection and help decide when another inspection is worthwhile.

Cheap repeated inference makes this feasible, but the observation policy determines whether it is useful. Which event should trigger a new judgment? What history should remain visible? How long is the answer valid? Responding to every change can create unnecessary review and oscillation.

A passing test is a simple example. A decision can correctly report success for artifact v1. Once the artifact changes to v2, that old judgment should not authorize the next action. Version binding and invalidation are exact bookkeeping, so the application should handle them directly.

The integration can wrap a model answer with the state version it inspected. This sketch shows the division of labor; it is not a provider-specific API:

snapshot = capture_state()
scores = judge(snapshot, questions)

if current_version() != snapshot.version:
    schedule_recheck()
elif scores["needs_more_evidence"] > review_threshold:
    queue_evidence_search()
else:
    continue_agent()

# Completion and permissions use separate application checks.

A monitor should therefore retain three things alongside the scores: what was judged, when it was judged, and what the result is allowed to change. I would first use such a component to schedule inspection around informative events. Broad verdicts like “this agent is stuck” demand a much stronger understanding of the process.

7. Evaluate the intervention, not just the answer

The most useful evaluation unit is a decision opportunity inside a workflow. What would have happened without the judgment? What work did it avoid or add? What did its mistakes cost? Did the downstream task finish?

This perspective suggests a short set of checks before adding a new controller:

1. Locate the leverage. Insert correct decisions as an oracle diagnostic. If even perfect judgments do not improve the workflow, model accuracy is not the immediate bottleneck.
2. Use a capable baseline. Compare with rules, an available embedding method, and a generator producing the same required output. Trivial Markdown fences should not manufacture a decision-model win.
3. Check the consuming policy. Measure both mistaken acceptance and unnecessary rejection at the coverage the program will use. A normalized distribution is not automatically calibrated; TypeSafe’s confidence documentation also distinguishes its confidence statistic from answer probabilities.
4. Charge the whole intervention. Include candidate construction, judgment latency, fallback, readback, and subsequent calls. Preserve the quality of completed work.

The emerging community points to several promising locations. Jev-Mem integrates a decision controller with memory organization and retrieval. A crash-narrative coding study uses a 27-question schema across 195,857 records and audits the answers against human judgments. These are concrete workloads: named decisions, repeated state structures, and observable consequences. Their reported results concern the respective complete systems, rather than a universal gain from adding Jev.

My interest has shifted from finding one impressive decision endpoint to finding owned semantic operators: operations an application needs often enough to specialize, inspect, and connect to a stable policy. The clearest opportunities seem to be dense judgments over shared evidence, scoped customization, and decisions that direct recoverable investigation.

Sources and experimental notes

The local results above consolidate our earlier NeoHorse-Jev exploration; no new training or model evaluation was run for this essay. They include public development selections and small constructed tasks. Related wording variants and repeated starts are not independent cases. Timing refers to the recorded reference implementations. Work-allocation and persistent-observer patterns are design directions motivated by these observations.

The released model is NeoHorse-Jev-4B, with GGUF weights and a decision-aware runtime. The experimental shared-state path is separate from the archived sequential GGUF runtime. A custom decision readout needs runtime support; a generic chat endpoint is a different interface.

• Interface and framing: TypeSafe documentation and Diogo Almeida’s Latent Space interview.
• Open implementations: Kev and Nimble.
• Harness integration: mu and pi-jev’s validation record.
• Workload studies: Jev-Mem and Calibrated Decisions at Scale.
• Dataset definitions for the local evidence probes: SQuAD2 and ContractNLI.