System One Next to the LLM: First Week in a Procurement Assistant

Most of what an agent does between a user’s message and a useful answer is not deep reasoning. It is a string of small multiple-choice decisions: which lane does this message belong in, which of these passages actually answers the question, which of these CPV codes fits the purchase. We had been paying a frontier-class planner roughly 7.5 seconds and about two US cents for questions that already had a handful of possible answers.

In our first week of live use, three of those decisions in Procurement Mind, our drafting workspace for below-threshold public purchases, run on a System One model: TypeSafe’s Jev (jev-1.13.0). Across about four to five days of early live use and 60 decisions so far, the picture is simple: no errors, no timeouts, every routing decision confident enough to act on agreed with the LLM planner, and the whole period cost less than one US cent.

The sample is small and partly our own test traffic. The numbers illustrate an architecture. They are not proof. The interesting part is what it feels like to put a fast, calibrated decision model next to an LLM, and why I think System One models are here to stay.

A light bit of context

Procurement Mind is a proof of concept at RTK for below-threshold public purchases (väikeostud) by Estonian contracting authorities. A person describes a purchase, and a small set of agents, the engine behind the workspace, help turn that into procurement documents, market research, and a supplier list.

Every chat message still reaches an LLM planner, which decides what the message is and which agent or tool should take it. Agents retrieve, draft, and propose. The person accepts or rejects.

One rule sits over the whole loop: AI recommends, rules constrain, the person decides. The model never chooses a transition in the procurement’s state. That rule decided where a System One model could go long before any benchmark did.

What made TypeSafe / Jev interesting

A System One model does not generate text. You send it a state (a message, a set of passages, a short document) and typed questions: pick one of these options, score this, how likely is that. It returns an answer and a probability for each, in one parallel pass. The name borrows Kahneman’s split between fast intuition and slow deliberation. TypeSafe’s announcement of System One models and Jev is the clearest public description of the class.

Three properties made it worth trying beside our planner:

• Answers stay inside the schema. The model cannot invent a label you did not declare. It can still be wrong. It cannot be wrong in an unexpected shape.
• The probability is part of the product. Calibration is the point: a high score is meant to mean something you can threshold on.
• Latency and cost sit in a different class. Nothing is generated token by token, so answers come back in under a second for a fraction of a cent.

It is the wrong tool for a written explanation, a draft, or a chain of reasoning. Those stay with the LLM. That split, LLM for language and deliberation, System One for structured micro-decisions, is the design insight I care about.

Three narrow decisions

We only delegated decisions that are narrow, time-critical, have a known answer set, and leave authority with someone else.

Decision	What Jev answers	Who still decides
Message routing	Intent of a chat message, document block it targets, whether it holds several tasks	The LLM planner, whenever Jev is not confident
Passage relevance	For each of 12-16 retrieved passages: is it about the query, and does it state the answer	The agent that reads the reranked passages and cites them
CPV preselection	Which code from an embedding shortlist of about 25 fits the purchase	The drafter, who sees a short list with probabilities

Estonian was the open question. The vendor’s documentation names English as Jev’s strongest language. Before wiring anything in, we probed candidates on synthetic Estonian messages and on public CPV and legal text. A candidate stayed only if the probe showed the model adding something the existing path did not.

One probing lesson generalises. Questions phrased as abstract concepts worked poorly. Questions that named the observable cue worked. Asking whether a message “refers to the earlier conversation” separated follow-ups badly. Asking about the words a follow-up actually uses separated them cleanly.

Race, don’t chain

The obvious design puts the fast model in front of the planner as a gate. We did the opposite: both start at the same moment. If Jev answers with enough confidence, its route is taken and the planner is cancelled. If not, the planner’s answer arrives exactly when it always did.

Path	Typical time
System One routing (Jev)	~0.8 s
LLM planner alone	~7.5 s
Best-case save when Jev wins the race	~6-7 s

The worst case is the old latency. The best case saves several seconds per message. The fast path cannot make anyone wait longer. It can only make them wait less.

On some fast-routed messages we still let the planner finish in the background, so agreement stays a live measurement rather than a launch-day number. The other two decisions run in line, not in a race: a small latency trade for better ranking and a better shortlist, only where that trade is worth paying.

Everything else about safety is intentionally dull. Each call is optional, with a short budget and a fall-back to the old path. Each decision can run live, in shadow, or off. Thresholds are priced by the cost of being wrong. The model version is pinned. Text leaves only after pseudonymisation. Authority never moves on a probability alone.

A small snapshot, not a verdict

Across the first week of live use, Jev made 60 decisions across the three points: routes, passage rankings, and CPV preselections. There were no errors and no timeouts. All three decisions together cost less than one US cent over the period. Routing was about an order of magnitude faster and hundreds of times cheaper than the planner call it races. Every live routing decision at or above our act threshold agreed with the planner. Disagreements sat at lower confidence and never took the fast path.

This is a first look only: a small set of test data from our first week. We continue to measure, tune thresholds, and experiment. Eleven of eleven confident routing agreements still leaves a wide uncertainty band. Most early traffic was our own testing. Synthetic probes are good for choosing where a model helps and bad for choosing how much confidence to demand on real Estonian chat. Short, context-heavy replies (“sobis”) still belong on the slow path. Engineering measures show that the component works. Whether drafters produce better procurement documents in less time is a different measurement, and that is the one that justifies keeping the component.

Thanks to TypeSafe for early access to Jev. Today they are the serious option for System One models as a managed service, which is what a public-sector proof of concept needs to start with. What we would like to see next: stronger Estonian support, a way to adapt the model on our own audited labels, and an EU region.

Why System One models are here to stay

An agent loop is full of multiple-choice micro-decisions pretending to be chat. Paying a frontier planner for each of them is slow and expensive. Generating free text when you already know the answer set is the wrong interface.

System One models give software a different contract: unstructured state in, typed probabilistic decisions out. Put them beside the LLM, not as a replacement for it. Race them where latency matters. Keep authority with rules and people. That pattern fits public-service agents as cleanly as it fits a procurement drafting workspace. Open-weights models such as Laya already serve the same wire format, so the pattern does not depend on a single provider.

In miniature, it is the same split the agenticgov.eu principles draw for citizen-facing agents: the model may plan, retrieve, and prepare; trusted code verifies; the decision stays where authority lives. A fast model does not move that line. It only makes the preparing side cheaper. Most of the surrounding stack we used (LangGraph, LiteLLM, multilingual-e5, Presidio with EstBERT, MLflow, OpenTelemetry) is listed by layer on the agenticgov.eu components page.

If you run agents in a public service, the operational question is short: which of your LLM calls is really a multiple-choice question, and what does it cost you to get it wrong?

The views here are my own.

References

1. Introducing System One Models & Jev - TypeSafe AI.
2. Seven Principles for Agentic Government - agenticgov.eu.
3. The Agentic Government Stack, Layer by Layer - agenticgov.eu.
4. Laya model card - Convai Innovations.
5. Daniel Kahneman, Thinking, Fast and Slow (2011).