Jev and Strands Decider: where decision models fit

A customer writes: “The replacement arrived, but you charged me twice. Please fix the payment.” Before an agent writes a reply, it has a smaller job: decide where this request belongs. Billing, delivery, returns, or a person who can untangle it?

That is the part of the workflow I find interesting here. A model need not compose an explanation every time software needs a judgment. But a cheap judgment is useful only if the surrounding system knows when to trust it and what to do when it is wrong.

My view: Jev makes a case for buying those decisions through an API. Strands Decider makes the same category accessible to teams that want to inspect, run and adapt the model themselves. The choice turns on workload fit, operating responsibility and the cost of mistakes. There is no defensible universal winner in the evidence below.

Update, 5 October 2026: This expands the original Jev analysis with Strands’ 1 October announcement and the v21 checkpoint available at review time. The original URL and publication date are preserved. Model results below are vendor or maintainer reports. The worked arithmetic was checked locally; neither model was run for this article.

What Strands adds to the picture

Jev launched in early access on 15 September. Its interface accepts state and typed questions: Choice selects an option, Score evaluates an ordered rubric, and Noul returns a yes/no probability. Choice and Score include distributions and confidence. Several questions can share the same state. TypeSafe’s interface documentation explains the contract.

The Strands announcement introduces a model and supporting Python package, CLI and local server, not just another agent library. Code, weights and training materials are available. The announcement discusses v19; by this review, the repository also documents a v21 release. Those versions should not be blended into one set of results.

Under the hood, Strands starts with Qwen3.5-2B-Base, about 1.9 billion parameters. It replaces the text-generation head with a small pointer head that scores the supplied options, and adapts the base using small trainable adapters called LoRA. Labels are supplied at request time. This is more flexible than a classifier permanently trained to return one fixed set of categories, but it still needs those options and their meaning spelled out. See the pinned architecture.

This matters because the research is inspectable. You can examine how confidence is computed, what data shaped the model, and which changes failed. That gives a team more control over an experiment. It also gives that team more software to operate.

Similar interface, different responsibilities

Decision	Jev	Strands Decider
How you obtain a result	Call TypeSafe's hosted service	Load a checkpoint locally or operate your own serving process
What you can inspect	Public interface, documentation and reported evaluations	Apache-2.0 code and model artifacts, training recipes and evaluation records
Core output shape	Choice, Score and Noul	The same three broad primitives; details need compatibility checks
Main operating tradeoff	Service access, network behavior, provider terms and availability	Hardware, dependencies, capacity, deployment and monitoring
A sensible first experiment	Shadow one existing routing step through the API	Replay the same labeled requests locally before adding a serving tier

This is an operational comparison, not a capability ranking. Sources: Jev’s documentation, the Strands model card, and its serving guide.

I would keep two less fashionable alternatives in the comparison. Ordinary code should handle an exact refund limit or a missing required field. An LLM with constrained outputs can also select from an allowed set; vLLM documents that capability. Valid output shape is not exclusive to decision models. The interesting test is whether a specialist gives you acceptable decisions with a better total cost and latency profile.

I would start with Jev when the service’s access and data terms fit and the team wants to test the decision without operating another model. I would start with Strands when local processing, inspection or adaptation is worth the serving work. For a step that must write a useful explanation or reason through a difficult exception, I would keep a suitable generative model in the evaluation. These are starting points for a test, not product endorsements.

Read the numbers with their labels attached

Reported measurements, not Model Fieldnotes benchmarks. These rows describe different experiments. They are not a head-to-head leaderboard.

Evidence	Reported number	What it actually covers
Jev launch comparison	193.6× faster; 444.6× cheaper	TypeSafe's selected workflow evaluations against wrapped LLM alternatives
Strands original v19 recipe	167/231 correct, or 72.3%	Public JevBench tasks at a 3,072-token window; not every exported retrain
Strands released v21	176/231 correct, or 76.2%; Brier 0.323; ECE 0.064	Released files, 4,096-token window, maintainer release checks
Strands v19 on RTX 3090	Median 115 ms; p95 299 ms	Recorded single-question JevBench run under WSL2
Strands v19 on M3 Pro	Warm median 234 ms; p95 2,628 ms	All JevBench tasks; first-request timings are higher

TypeSafe says its headline multiples may be toward the high end. The workflows came from its capabilities team; reference answers were other models’ predictions, and the probability-producing LLM wrapper added overhead. Agreement with those models is not independently labeled ground truth. Nor do those multiples compare Jev with Strands Decider. Jev evaluation caveats.

For Strands, the versioned results are more useful than a rank. On one shared host, six v19 recipe seeds averaged 172.8 correct tasks and six v21b seeds averaged 172.3. Mean Brier improved from 0.341 to 0.331. That supports a more modest reading than “v21 made a large accuracy leap.” The better single released score combines a different run and environment with the recipe change.

Brier measures probability error; lower is better. ECE summarizes the gap between predicted probabilities and observed accuracy in bins. It is not the model’s error rate, and different binning or task mixtures affect it. The v21 card identifies the released artifact’s figures. The v19 export has its own results, which differ from the original recipe. Pin a checkpoint, not just a product name.

The laptop numbers also deserve care. “153 ms on a Mac” applies to the warm, under-300-token subset in the maintainer’s test. Across all tasks, the warm p95 was over 2.6 seconds. A short support message and a long policy document are different workloads. Those are v19 results, not a measured latency claim for v21 or for a loaded production service.

Confidence is where a migration can quietly break

Suppose a Choice returns probabilities of 0.8, 0.1 and 0.1. Both implementations’ published Choice formula gives confidence 0.7:

confidence = (3 × 0.8 − 1) / (3 − 1) = 0.7

That is a transformed measure of concentration. It is not an observed 70% correctness rate. A threshold needs validation on the relevant requests, with errors and escalation counted separately. TypeSafe’s confidence definition.

Score has a less obvious difference. TypeSafe uses distance from the most likely level. Strands uses standard deviation with an optional adjustment for training’s ordinal smoothing. For an illustrative distribution [0, 0.5, 0.5] across three levels:

Calculation on the same distribution	Confidence
TypeSafe's documented Score formula	0.250
Strands formula, smoothing set to 0	0.500
Strands formula, smoothing set to 0.1	0.731

These are formula calculations, not model predictions. The Strands schema explicitly says compatibility with the Jev API itself is unverified. It also distinguishes Noul, which has no separate confidence field. A service swap can preserve your JSON parsing and still change which requests get automated. Refit thresholds rather than copying them.

Put the model at a useful boundary

For the double-charge message, I would ask a small model for a proposed route, not permission to issue a refund. The queue can be one of billing, delivery, returns or unclear. Application code still checks the account, authorization, transaction and refund policy.

Three places I would consider this pattern:

• Support triage. Choose a queue from a short message. Test mixed intents, unfamiliar products, spelling mistakes and requests that need clarification. The “unclear” option is useful only if the model actually uses it on the right examples.
• Before a tool call. The Strands integration example checks whether an agent invented a city when asked for weather. Its intervention can return control to the agent to ask for the missing city. The example uses a local decider but a Bedrock-backed main agent, so the complete workflow is not offline. Its thresholds are illustrative.
• Retrieval filtering. Judge which retrieved passages address a question before sending them to a larger model. Measure relevant evidence discarded as well as tokens saved. Dropping the only passage containing an exception can make the final answer cheaper and wrong.

A semantic check can supplement permissions; it cannot create them. A model saying “the arguments look grounded” does not prove that an account belongs to this user. Likewise, a classification does not prevent a retry from creating a second refund or turn an accepted job into a completed one. Those require software contracts. Agent Failure Recipes makes that distinction executable through controlled failures and repairs.

Work the economics through one example

Illustrative workload, not measured savings. Assume 100,000 requests a month, 1,000 billed input tokens per decision call, and an existing larger-model route costing $0.02 per request. Assume a tested policy could handle 80% through a decision stage plus ordinary code, with 20% falling back to the existing route.

TypeSafe advertises $42 per billion input tokens, equivalent to $0.042 per million, checked on 5 October. Use that published rate as one input to the example, not a promise about a future invoice. Confirm billing and access terms before budgeting. Advertised price.

Monthly component	Calculation	Cost
Existing larger-model route	100,000 × $0.02	$2,000.00
Decision calls for every request	100,000 × 1,000 ÷ 1,000,000 × $0.042	$4.20
Larger-model fallback	20,000 × $0.02	$400.00
Hybrid inference subtotal	$4.20 + $400.00	$404.20

The apparent reduction is $1,595.80, or 79.79%. It comes mainly from avoiding larger-model calls, not from a spectacular token-price ratio.

Now suppose the new route causes 320 additional mistakes, each costing $5 to correct. That is $1,600, enough to erase the entire saving. Those 320 mistakes are just 0.4% of the 80,000 automatically handled requests. This is why I would put cost per successful outcome next to cost per call. The illustration excludes engineering, ordinary tool costs, review and other overhead; unequal overhead belongs in the decision too.

Self-hosting changes the arithmetic again. At an assumed $0.50 per hour, one always-on instance costs $360 over 30 days. At the same 1,000-token input shape, the advertised Jev input charge reaches $360 at about 8.57 million requests per month. This is only a compute-versus-input-fee crossing point. It establishes neither Strands capacity nor equal quality and excludes replicas, engineering, electricity where relevant, storage and network costs. Open weights can be valuable for control and local processing even when they do not win on unit cost.

Latency has a similar trap. Assume a 120 ms decision stage, an 800 ms larger-model call, and the same 20% fallback rate. Ignoring all other work, the average becomes 120 + 0.2 × 800 = 280 ms. But a fallback request takes 920 ms, slower than going directly to the larger model. These are fixed illustrative durations, not percentiles or either model’s measured performance. An attractive average can hide a worse experience for the difficult requests.

Download the worked calculations. The Python file uses no packages or model calls and reproduces the confidence, cost and latency arithmetic. The Task Cost Calculator helps extend the economic example with review and failure costs.

What remains unproven

The evidence reviewed does not establish that either model is more accurate on your workload. TypeSafe’s launch workflows and Strands’ public JevBench runs answer different questions. Repeated development against a small public benchmark is not a substitute for a held-out business evaluation.

Strands’ evaluation notes disclose weaknesses that matter for deployment: v19 often failed to change its answer when only the question changed; novel yes/no tasks and unfamiliar scoring rubrics transferred poorly; calibration depended on the data mixture. These are reasons to test negation, changed intent and new policies explicitly. They are not universal error rates, and I would not assign those v19 measurements to v21 without a corresponding test.

The v21 model card adds an instructive image limitation: removing the image did not reliably make confidence collapse. If the document or image is missing, check that fact before asking the model. Confidence is not a substitute for checking whether evidence was supplied.

There are also ordinary deployment details. The documented local Strands server has no authentication. Its default context handling can shorten the state; --strict-window rejects an oversized prompt instead. Put access controls around a deployed service and make context loss observable. Neither a local process nor an allowed output label is, by itself, a production safety boundary. Serving details.

The comparison I would actually run

Start with one existing decision and the current baseline. My proposed first pass would use 500 development examples to refine questions and thresholds, then 1,000 untouched evaluation examples. Those are planning numbers, not a sufficiency guarantee. Include ambiguous messages, multilingual requests, missing facts, adversarial instructions and cases where the correct action is to stop.

Compare rules or the existing classifier, a suitable constrained LLM, Jev and a pinned Strands checkpoint. Keep the state, candidate actions and error definitions fixed. Record model and code revisions, tokenization and truncation, hardware, warm-up, concurrency, timeouts and any retry policy.

At each threshold, report the share automated, error rate within that share, costly-error counts, review volume, p50/p95 latency and total cost per successful outcome. Choose thresholds on the development set, then evaluate once on the untouched set. Zero observed costly errors in 1,000 cases is encouraging; it does not establish rare-event safety. The one-sided 95% binomial upper bound is still about 0.3%, calculated as 1 − 0.05^(1/1000) under independent, representative trials. Use the number of relevant automated cases as the denominator when evaluating that subset.

My first deployment would propose routes in shadow mode. I would expand authority only after inspecting disagreements and measuring review burden. The research question is narrow enough to answer: can this component remove unnecessary generation while preserving the decisions the workflow needs to get right?

That is a useful direction for agent engineering. Smaller decisions can make a system easier to inspect and cheaper to run. Someone still has to define a good decision, own the exceptions and verify that the work got done.

For the evaluation design, see How to compare AI models for your application. For the operating cost boundary, read The cost of a completed task.