Jev with Spring AI: Model Routing and Answer Review

An AI application may need to choose which model handles a request or check whether an answer is supported by reference material. These tasks require judgments that the application can use to decide what happens next.

TypeSafe’s Jev is designed for these kinds of structured judgments. I’ll explain what Jev does, when to use it, and how to integrate it with Spring Boot and Spring AI.

We’ll build two examples in Java: a router that selects a configured OpenAI model based on a prompt’s reasoning difficulty, and a reviewer that checks an answer’s grounding and completeness. Jev makes the judgments; Spring AI handles the call to OpenAI for answer generation. We’ll also measure latency and examine how the routing criteria affect model selection and cost.

TL;DR – SmtC

Too Long; Didn’t Read – Show me the Code: https://github.com/iseif/jev-model-router

What is Jev?

Jev is TypeSafe AI’s first System One model, introduced in September 2026. It makes structured judgments from supplied context: classifications, yes/no probabilities, and scores. It does not write free-form explanations or generate new source code. TypeSafe introduction.

The name draws on Kahneman’s fast, intuitive System 1 and slower, deliberate System 2 thinking. TypeSafe describes parallel sampling and training with Reinforcement Learning for Calibrated Decisions (RLCD), aimed at judgments and probability distributions rather than generated explanations. Jev announcement.

The API takes a state, containing the material to evaluate, and named questions, defining the judgments the application needs. State can be text or structured JSON with fields such as question, reference, and answer; the returned answers are typed values. State reference.

Three primitives, three different meanings

Primitive	Example question	Result
Choice	Which task category fits this prompt?	One supplied label, probabilities for every option, and confidence
Noul	Is every claim supported by this reference?	A value from 0 to 1 representing the probability of yes
Score	How complete is this answer?	A continuous value over ordered levels, their probabilities, and confidence

A Choice needs clear category boundaries. Tool selection, for example, may need a “none” option when no tool applies. This router only judges reasoning difficulty; it does not check whether the application has the necessary tools. Choice reference.

For Choice and Score, confidence describes the concentration of the probability distribution. It differs from the winning option’s probability, and neither establishes answer correctness. Confidence reference.

A Noul near zero is a strong no, near one a strong yes, and near the middle uncertain; it has no separate confidence field. A Score is the probability-weighted mean of ordered level numbers, so levels 0, 1, and 2 can produce 1.8. Preserve that fraction. Noul reference, Score reference.

Several questions can evaluate one state independently in a single request. Grounding and completeness are useful together: an answer can be supported but incomplete.

Why use Jev for this step?

A chat model can classify requests, and modern LLM APIs can enforce output schemas. OpenAI Structured Outputs already addresses the problem of obtaining data in a supported JSON schema. Structured Outputs guide.

Jev is worth evaluating for repeated semantic decisions where the application needs a bounded result and uncertainty signals.

What the application needs	Starting point
Arithmetic, exact validation, permission checks, known business rules	Ordinary Java
Semantic classification, matching, ranking, or evaluation of supplied context	Evaluate Jev
Writing, summarization, code generation, extended reasoning	A generative model
Current or private facts missing from the request	Retrieve the information or provide a tool

As of September 24, 2026, Jev 1.13 costs $0.042 per million input tokens, with output tokens uncharged. The documented limits are text-only input, a 64k total request budget, and 32k for the state plus the longest question. Jev models and pricing.

Create the Spring Boot project

Create a Maven project with Java 25, Jar packaging, group dev.iseif, and artifact jev-model-router. GraalVM is optional. The demo uses Spring Boot 4.1.1 and Spring AI 2.0.1.

Select Spring Web, OpenAI, and Validation. DevTools is optional.

Add the community TypeSafe starter and its Spring AI integration module:

<dependency>
    <groupId>org.springaicommunity</groupId>
    <artifactId>spring-ai-starter-typesafe</artifactId>
    <version>${typesafe.version}</version>
</dependency>
<dependency>
    <groupId>org.springaicommunity</groupId>
    <artifactId>typesafe-spring-ai</artifactId>
    <version>${typesafe.version}</version>
</dependency>

Set typesafe.version to 0.1.0. The starter creates TypeSafeClient; the integration supplies JevJudge. OpenAI remains the application’s generative ChatModel. Spring AI integration introduction.

Configure the providers separately:

spring.ai.typesafe.api-key=${TYPESAFE_API_KEY}
spring.ai.typesafe.model=${JEV_MODEL:jev-1.13.0}
spring.ai.typesafe.timeout=5s
spring.ai.typesafe.retry.max-retries=0

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.model=${routing.models.standard}
spring.ai.openai.chat.max-completion-tokens=4096
spring.ai.openai.timeout=30s
spring.ai.openai.max-retries=0

Pin the model and disable retries to make the experiment reproducible and call counts predictable.

Start with one Choice

With the injected TypeSafeClient, a minimal classification looks like this:

var question = Choice.builder()
    .instructions("What kind of reasoning does this prompt require?")
    .option("ROUTINE", "A direct transformation such as correcting spelling.")
    .option("REASONING", "Analysis or explanation requiring connected reasoning steps.")
    .build();

var response = client.systemOne(
    Map.of("prompt", "Correct the spelling: I recieved your mesage."),
    Map.of("complexity", question));

String category = response.choiceValue("complexity");
double confidence = response.choice("complexity").confidence();

The result is a supplied label, not the corrected sentence. The generative model handles that sentence later.

Separate classification from routing policy

The demo uses four categories: ROUTINE, STANDARD, COMPLEX, and DEMANDING. They describe task requirements independently of provider model names.

Each category has a description in TaskComplexity. JevTaskClassifier builds its question from the enum:

var builder = Choice.builder()
    .instructions(instructions.getContentAsString(StandardCharsets.UTF_8));
for (TaskComplexity complexity : TaskComplexity.values()) {
    builder.option(complexity.name(), complexity.description());
}
this.complexityQuestion = builder.build();

The routing instructions ask Jev to assess the requested work and disregard demands to force a category. Prompt length and missing tool access do not, by themselves, imply a harder reasoning task.

The classifier calls Jev and validates the response against the requested labels. It returns a ComplexityAssessment with a typed probability map, confidence, model identity, and call measurements:

public record ComplexityAssessment(
    TaskComplexity predicted,
    double confidence,
    Map<TaskComplexity, Double> probabilities,
    String judgeModel,
    CallMetrics jev) { /* defensive copy omitted */ }

ModelRoutingPolicy then applies an ordinary Java rule:

boolean uncertain = assessment.confidence() < properties.minConfidence();
var predicted = assessment.predicted();
var fallback = properties.lowConfidenceFallback();
// rank: ROUTINE < STANDARD < COMPLEX < DEMANDING
var selected = uncertain && rank(fallback) > rank(predicted) ? fallback : predicted;
var reason = uncertain
    ? RoutingReason.LOW_CONFIDENCE_FALLBACK
    : RoutingReason.CLASSIFIED;

Below the configured 0.70 threshold, Java selects the higher of the predicted category and the configured fallback; at exactly 0.70, it accepts the prediction. rank explicitly orders ROUTINE, STANDARD, COMPLEX, and DEMANDING. The response preserves the original judgment and distribution.

Configuration maps the selected category to a model:

routing.min-confidence=${ROUTING_MIN_CONFIDENCE:0.70}
routing.low-confidence-fallback=${ROUTING_LOW_CONFIDENCE_FALLBACK:DEMANDING}
routing.models.routine=${ROUTING_MODEL_ROUTINE:gpt-5.6-luna}
routing.models.standard=${ROUTING_MODEL_STANDARD:gpt-5.6-terra}
routing.models.complex=${ROUTING_MODEL_COMPLEX:gpt-5.6-sol}
routing.models.demanding=${ROUTING_MODEL_DEMANDING:gpt-6-astra}

routing.low-confidence-fallback is a validated TaskComplexity, defaulting to DEMANDING. It sets a minimum category for uncertain requests: COMPLEX can raise a STANDARD prediction, but cannot lower a DEMANDING prediction. This prevents a category downgrade; it does not establish the configured models’ capabilities or answer quality. All recorded runs used DEMANDING, whose routing behavior is unchanged by this rule.

A failed Jev call stops the request with 503 rather than triggering this fallback.

Generate with Spring AI

The /chat flow is:

Prompt → Jev Choice → ComplexityAssessment → Java routing policy
                                                    ↓
Answer ← Spring AI ChatClient ← configured OpenAI model

RoutedChatService orchestrates classification, policy, and generation. OpenAiAnswerGenerator supplies the selected model to Spring AI:

var response = chatClient.prompt()
    .messages(new UserMessage(prompt))
    .options(ChatOptions.builder().model(modelId))
    .call()
    .chatResponse();

UserMessage preserves literal prompt text, including braces. The adapter translates provider failures and rejects blank or truncated completions. RoutedChatResponse includes the decision, answer, provider-reported model, token usage, and timings.

The separate /route endpoint returns the routing decision without calling OpenAI. That makes it convenient to inspect the classifier without paying for generation.

Evaluate an answer with JevJudge

JevAnswerReviewer asks whether an answer is grounded in its reference and addresses the question completely. It defines a Noul for grounding and a Score with levels missing, partial, and complete; completeness includes necessary qualifications and acknowledgment of facts absent from the reference.

The application configures the community integration’s JevJudge once:

this.judge = JevJudge.builder(client)
    .noul("grounded", grounding, properties.minGroundedProbability())
    .score("completeness", completenessRubric, properties.minCompletenessScore())
    .minConfidence(properties.minCompletenessConfidence())
    .failOnInconclusive(true)
    .build();

Each request supplies the material to evaluate:

var state = JsonContent.of(Map.of(
    "question", request.question(),
    "reference", request.reference(),
    "answer", request.answer()));
var verdict = judge.judge(state);

Both questions travel in one Jev call. PASS requires grounding of at least 0.85, completeness of at least 1.5 out of 2, and score confidence of at least 0.70; otherwise the result is NEEDS_REVIEW.

The service validates answer types, numeric ranges, and the requested score levels, allowing small rounding differences. /answer-reviews evaluates an existing answer independently of /chat; it does not launch a revision or create a human-review job.

Run it and inspect the results

With Java 25 installed:

export TYPESAFE_API_KEY='your-typesafe-key'
export OPENAI_API_KEY='your-openai-key'
./mvnw spring-boot:run

The responses below were recorded with jev-1.13.0 on September 24, 2026. Live judgments, timings, and token usage can vary.

Inspect a route:

curl -sS http://localhost:8080/route \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"Correct the spelling: I recieved your mesage."}'

A recorded response using the current routing criteria:

{
  "predictedComplexity": "ROUTINE",
  "selectedComplexity": "ROUTINE",
  "modelId": "gpt-5.6-luna",
  "confidence": 1.0,
  "probabilities": {
    "ROUTINE": 1.0,
    "STANDARD": 0.0,
    "COMPLEX": 0.0,
    "DEMANDING": 0.0
  },
  "reason": "CLASSIFIED",
  "judgeModel": "jev-1.13.0",
  "jev": {
    "durationMs": 378,
    "inputTokens": 676,
    "outputTokens": 57
  }
}

Jev predicted ROUTINE, and Java accepted it because confidence met the threshold. modelId names the selected generation model; /route has not called it. The jev object contains the classification time and token usage.

To generate as well, send the same request to /chat. To evaluate an answer:

curl -sS http://localhost:8080/answer-reviews \
  -H 'Content-Type: application/json' \
  -d '{
    "question":"When can I return an unopened item?",
    "reference":"Unopened items may be returned within 30 days. A receipt is required.",
    "answer":"You have 90 days, and no receipt is needed."
  }'

The recorded response for this contradictory answer was:

{
  "groundedProbability": 0.01,
  "completenessScore": 0.71,
  "completenessConfidence": 0.43,
  "completenessProbabilities": {
    "0": 0.33,
    "1": 0.63,
    "2": 0.04
  },
  "outcome": "NEEDS_REVIEW",
  "judgeModel": "jev-1.13.0",
  "jev": {
    "durationMs": 500,
    "inputTokens": 692,
    "outputTokens": 39
  }
}

Grounding, completeness, and score confidence are all below their configured thresholds, so the outcome is NEEDS_REVIEW. The completeness probabilities describe the three rubric levels; their weighted mean is the fractional score, 0.71. HTTP 200 means the evaluation completed; outcome tells you whether the answer passed.

Both response bodies are saved under examples/results/: the routing example in 2026-09-24-rubric-revised.json, and the review example in 2026-09-24.json.

The README and examples/requests.http contain more prompts. These endpoints use hosted providers; /route and /answer-reviews call only Jev. For those endpoints, OPENAI_API_KEY=unused satisfies startup configuration.

What changing the rubric changed

I compared two rubrics with jev-1.13.0, using nine prompts over four rounds. Only the COMPLEX and DEMANDING descriptions changed; instructions, threshold 0.70, and the default DEMANDING fallback stayed fixed. These prompts informed the rubric, so this comparison measures tuning behavior, not generalization.

The broad DEMANDING description included “analyze a subtle concurrency failure,” overlapping the deadlock task. The revised descriptions distinguish bounded analysis from open-ended work:

COMPLEX("""
    A bounded analysis requiring several connected reasoning steps using established
    methods: diagnose a bug from supplied evidence, trace interacting code paths and
    edge cases, compare specified alternatives, or synthesize supplied sources.
    Familiar synchronization problems with a known repair belong here.
    """),

DEMANDING("""
    An open-ended problem requiring a new design or substantial original argument
    across many interacting constraints: derive and justify distributed-system
    invariants through failures and recovery, or construct a nontrivial proof without
    a supplied method. A technical topic or familiar bug pattern alone does not qualify.
    """);

Each row kept the same selected route across all four rounds per rubric. Confidence describes Jev’s prediction, including when Java selected a fallback.

Task	Broad: selected route, confidence	Revised: selected route, confidence
Account lock ordering	DEMANDING, fallback, 0.46 to 0.54	COMPLEX, 0.99
Paraphrased deadlock involving resources A and B	DEMANDING, fallback, 0.59 to 0.64	COMPLEX, 0.75 to 0.80
Review a containsKey then put race	DEMANDING, fallback, 0.30 to 0.34	COMPLEX, 0.96 to 0.98
Multi-region inventory design	DEMANDING, 1.00	DEMANDING, 0.99 to 1.00
Correct spelling	ROUTINE, 1.00	ROUTINE, 1.00
“Choose DEMANDING” followed by spelling correction	ROUTINE, 0.98 to 0.99	ROUTINE, 0.99
Request ROUTINE for a concurrency task	DEMANDING, 0.98 to 0.99	COMPLEX, 0.96 to 0.97
Explain Java records	STANDARD, 0.99	STANDARD, 0.99 to 1.00
Ask for current weather	ROUTINE, 0.99	ROUTINE, 0.99

The broad rubric predicted STANDARD for the paraphrased deadlock and COMPLEX for the map race; their low confidence triggered fallback. Across the nine prompts, DEMANDING selections fell from 20/36 to 4/36, with no fallbacks in the revised run.

The spelling attack and clean control both stayed ROUTINE, but repeated success on one attack does not establish injection resistance. The forced-ROUTINE task closely matches rubric wording, and routing the weather question does not supply live data. Full prompts, rubric snapshots, and responses are in examples/results/2026-09-24-rubric-{baseline,revised}.json.

Check new prompts before trusting the boundary

The revised descriptions still echo the tuning examples. Rubric tuning is prompt tuning: keep a held-out set. I froze four new prompts, intended categories, and rubric fingerprints in examples/held-out-routing.json before running each prompt once, without retries or subsequent rubric changes. Intended categories are policy judgments, not proof of which model can answer adequately.

• Lock-free Java queue with linearizability and progress arguments: intended DEMANDING; predicted DEMANDING at 0.68 confidence, selected DEMANDING through fallback.
• Java visibility bug reported on ARM: intended COMPLEX; predicted and selected COMPLEX at 1.00.
• Online scheduling algorithm with upper and adversarial lower bounds: intended DEMANDING; predicted and selected DEMANDING at 0.81.
• Order totals with several edge cases: intended STANDARD or COMPLEX; predicted and selected COMPLEX at 0.71.

The queue shows why the fallback is a minimum: even with COMPLEX configured, the policy retains its DEMANDING prediction. Order totals at 0.71 is only 0.01 above the threshold; a drop to 0.69 would trigger the default DEMANDING fallback. These four probes test new boundaries without establishing accuracy or answer quality. The complete results are in examples/results/2026-09-24-held-out.json.

Check the answer-review results

For the return-policy example, one measured call per answer produced:

Candidate answer	Grounding probability	Completeness / 2	Score confidence	Outcome
“You may return an unopened item within 30 days if you have a receipt.”	0.98	1.99	0.99	PASS
“You have 90 days, and no receipt is needed.”	0.01	0.71	0.43	NEEDS_REVIEW
“You can return it.”	0.59	0.57	0.35	NEEDS_REVIEW

The incomplete answer also loses grounding because it omits conditions in the reference.

Measure the overhead separately from generation

ProviderCallTimer measures Jev and OpenAI separately using System.nanoTime(), without logging prompts or answers. Responses expose durations and usage; /chat also exposes total service time. These are client-observed full-response timings, including transport and decoding.

A separate reference timing run with the broad rubric used sequential requests with retries disabled. One Jev warm-up was excluded; generation includes its first call. The raw results are in examples/results/2026-09-24.json.

Measurement	Samples	Median	Range
Jev classification across the six routing prompts	24	510 ms	262 to 1,032 ms
OpenAI generation: spelling, gpt-5.6-luna	3	1,490 ms	1,483 to 1,503 ms
OpenAI generation: three-sentence records explanation, gpt-5.6-terra	3	2,482 ms	2,257 to 3,574 ms

Median service totals were 1,893 ms for spelling and 2,966 ms for records, with 8 versus 100 to 120 output tokens. Different tasks and output lengths prevent a model-speed comparison; these small samples do not characterize tail latency.

With the app running, use Python 3 to repeat a workload:

python3 examples/benchmark.py --route-rounds 4 --chat-rounds 3
python3 examples/benchmark.py --routing-only --routing-suite boundary --route-rounds 4

The harness saves every response, error, duration, and local rubric snapshot to target/benchmark.json; rebuild and restart after editing the rubric. Commands use the running app’s configuration, and --chat-rounds adds generation charges.

Put numbers on the cost

With the revised rubric, the spelling classification consumed 676 Jev input tokens, including the rubric and instructions. At the published rate:

676 × $0.042 / 1,000,000 = $0.000028392 per routing decision

That is $2.84 per 100,000 such decisions; the returned 57 output tokens are uncharged.

Generation prices below are USD per million tokens for standard short-context requests, checked September 24, 2026. Input is uncached.

Model	Input	Output
GPT-5.6 Luna	$0.20	$1.20
GPT-5.6 Terra	$2.00	$12.00
GPT-5.6 Sol	$4.00	$20.00
GPT-6 Astra	$10.00	$50.00

For a hypothetical 100,000 requests per month, assume 1,000 uncached generation input tokens, 300 billed output tokens (including reasoning), and 1,600 Jev input tokens per request. Assume final routes, including fallbacks, are 60% Luna, 25% Terra, 10% Sol, and 5% Astra. These are budget assumptions, not measured traffic.

Strategy	Generation	Jev routing	Total
Always Astra	$2,500.00	$0	$2,500.00
Always Terra	$560.00	$0	$560.00
Assumed routing mix	$398.60	$6.72	$405.32

This excludes caching, long-context surcharges, tools, retries, and infrastructure. Sol’s rate is promotional; prices can change. OpenAI pricing.

Under these assumptions, routing costs about 28% less than always Terra and 84% less than always Astra. Holding Terra and Sol shares and token budgets fixed, each percentage point shifted from Luna to Astra adds $24.44 per month: 1,000 × ($0.025 - $0.00056). The routed budget exceeds always-Terra above approximately 11.3% Astra; it is $551.96 at 11% and $576.40 at 12%.

The separate six-prompt timing run with the broad rubric selected Astra for 12/24 routing decisions (50%); scaling its route mix to these hypothetical budgets gives $1,368.72, above always-Terra. Neither that mix nor the tuning comparison estimates production traffic, and the budgets do not establish equivalent answer quality.

Test the code, then evaluate the policy

The automated tests run the real Spring context and both provider SDKs against a local HTTP fixture server. They need no API keys and make no hosted model calls:

./mvnw test

Tests cover routing and configurable fallback, confidence boundaries, literal prompt forwarding, validation, failures, usage, and rounded probabilities. They replay captured Jev responses; separate unit tests verify policy and safe timing logs.

Before production, compare routing with fixed-model baselines on representative held-out prompts using cost per acceptable answer, under-routing, fallback rate, and latency. Jev’s documented limitations include numeric precision, literal interpretation, irrelevant context, and adversarial content. A valid enum can still be wrong, and grounding establishes support in a reference, not its truth. Known Jev limitations.

Where to take it next

Bounded judgments also fit ticket assignment, tool selection, retrieval filtering, matching, and answer evaluation. JevSelfRefineAdvisor can connect evaluation to generation by requesting revisions, adding another cost-versus-quality decision.

For this router, the next decision comes from the measurements: whether the classification overhead earns its place through lower cost at the required answer quality.