# CLM-8B: How Contrastive Language Models Speed Up AI Agent Decisions

> CLM-8B scores candidate agent actions instead of generating text. How it works, its speed and accuracy vs Jev, how to run it, and where it fits in an agent.

Source: https://bytevancer.com/blog/clm-contrastive-language-models · Published 2026-09-25 · ByteVancer · AI Research

**An open model that doesn't write a single word: CLM-8B scores an agent's candidate actions and returns probabilities, fast. How it works, what the numbers say, and where it fits.**

Most of the models an agent calls are asked to write: a plan, a tool call, an answer. Many of an agent's decisions are not writing tasks at all. Which tool should run next? Which of four candidate patches is most likely correct? Is this ticket urgent? CLM-8B, released by Contrastive-LM in September 2026, is built for exactly those decisions. It does not generate text. It scores candidate actions against the current state and returns probabilities. This article explains how it works and what its published results show, drawing on the project's GitHub repository and Hugging Face model card, MarkTechPost's coverage and Superpower Daily's analysis.

## What a "System One" model is

The authors describe a contrastive language model (CLM) as "a new class of System One model trained with a contrastive learning objective that connects states and actions". System One is their framing for fast, intuitive decisions, as opposed to the slower, deliberate reasoning of generating text step by step. CLM-8B is the first open model of this kind.

The comparison point throughout is Jev, TypeSafe AI's proprietary System One model, in limited early access since September 15, 2026. Jev returns typed values with probabilities, and CLM serves a TypeSafe-compatible API, so a request written for TypeSafe can be replayed through CLM's client.

The CLM authors are Jacky Kwok, Hangoo Kang, Tarun Suresh, Jon Saad-Falcon, Marco Pavone, Christopher Ré and Azalia Mirhoseini. The weights and code are released under Apache 2.0.

## How CLM works

CLM-8B keeps a Qwen3-8B encoder frozen and trains two small projection heads on top of it, about 20M parameters each: a state head and an action head. The model card puts the head at about 75 MB. Training uses a bidirectional InfoNCE loss, which pulls each state towards the actions that fit it and away from the ones that don't.

**Scoring one decision**
Encode the state → Encode the candidates → Dot products → Softmax

1. **Encode the state** — The current situation, as text, runs through Qwen3-8B and the state head: one vector.
2. **Encode the candidates** — Each candidate action runs through Qwen3-8B and the action head. These vectors can be cached.
3. **Dot products** — The state vector is multiplied with each action vector: one score per candidate.
4. **Softmax** — Scores become a probability distribution over the candidates you supplied.

Two consequences follow. There is no decoding loop, so a decision costs one encoder pass per new input rather than many generated tokens. And the probabilities are relative to the set you supply: CLM tells you which of your options fits best, not what option you forgot to list.

## Why it's fast: disaggregated, cached embeddings

Because states and actions are encoded separately, their embeddings can be cached and reused. A fixed toolset, a list of routing destinations or a rubric only has to be encoded once. The project reports p50 latencies on an RTX 4090 with the vector cache:

**p50 latency on an RTX 4090, without and with the vector cache, as reported by Contrastive-LM.**
- **New state on every call** — Without cache: ~28.6 ms; With cache: ~28.0 ms
- **Revisited state, 3 actions** — Without cache: 1.7 ms; With cache: 0.6 ms
- **Revisited state, 50 actions** — Without cache: 2.0 ms; With cache: 0.7 ms

The cache barely helps when every state is new, but for revisited states it makes scoring about 2.8x faster. The model card adds that with around 1,000 candidates, CLM is 13x faster than Jev.

## The training recipe

**Three training stages**
Pre-train → Mid-train → Post-train

1. **Pre-train** — ~60M question–answer pairs from Nemotron DQA.
2. **Mid-train** — ~30M synthetic hard negatives generated with Gemini 2.5 Flash-Lite.
3. **Post-train** — ~1M agent trajectories (Agent Data Protocol, Endless-Terminals, LiteCoder-Terminal-SFT), mixed 40% replay / 60% agentic.

**📊 Hard-negative top-1 accuracy by training setup**
- **Pre-training alone** — 52.1%
- **Hard negatives from the start (peak)** — 62.4%
- **Pre-training + mid-training** — 69.2%

*Introducing hard negatives from the start peaks at 62.4% and then overfits; staging them after pre-training reaches 69.2%.*
*Source: Contrastive-LM (CLM repository and model card)*

Replay matters in the last stage. With 40% replay data, hard-negative accuracy barely moves (69% to 68.5%) while the model learns agent behaviour. Training on agentic data alone drops it to 56.2%.

## Results vs Jev

**Zero-shot results as reported by Contrastive-LM.**
- **T-Rex** — CLM latency: 16.5 ms; Jev latency: 149.8 ms; CLM success: 5/5; Jev success: 5/5
- **BFCL v4 (tool calling)** — CLM latency: 76.8 ms; Jev latency: 125.5 ms; CLM success: 95.2%; Jev success: 99.2%
- **WikiRacing** — CLM latency: 79.8 ms; Jev latency: 225 ms; CLM success: 26/30; Jev success: 30/30
- **Super Mario** — CLM latency: 33.5 ms; Jev latency: 132.6 ms; CLM success: 5/5; Jev success: 5/5

**📊 Decision latency (ms, lower is better)**
- **T-Rex** — CLM-8B 16.5 ms · Jev 149.8 ms
- **BFCL v4** — CLM-8B 76.8 ms · Jev 125.5 ms
- **WikiRacing** — CLM-8B 79.8 ms · Jev 225 ms
- **Super Mario** — CLM-8B 33.5 ms · Jev 132.6 ms

*The "up to 9x faster" headline comes from T-Rex (16.5 vs 149.8 ms).*
*Source: CLM repository; MarkTechPost*

**📊 Task success (%, higher is better)**
- **T-Rex** — CLM-8B 100% · Jev 100%
- **BFCL v4** — CLM-8B 95.2% · Jev 99.2%
- **WikiRacing** — CLM-8B 86.7% · Jev 100%
- **Super Mario** — CLM-8B 100% · Jev 100%

*We converted counts to percentages for comparison: 5/5 = 100%, 26/30 ≈ 86.7%, 30/30 = 100%. BFCL v4 figures are the reported percentages.*
*Source: CLM repository*

## The tradeoff: faster everywhere, not better everywhere

Superpower Daily's reading of the results is the right one: CLM was faster than Jev on all four tasks, but trailed it on two of them — tool calling on BFCL v4 and WikiRacing. On the two game tasks both models succeeded every time. If your decision is latency-bound and the options are clear, the speed is a straight win. If a wrong tool call is expensive, the accuracy gap on BFCL v4 matters more than the milliseconds.

## As a verifier: picking the best of N

The strongest numbers come from using CLM as a verifier: generate several candidate solutions with a large model, then let CLM pick one. The authors fine-tuned the heads for this, evaluated on held-out subsets and ran on an H100.

**📊 Best-of-N verification: success rate (%)**
- **DeepSWE (38 tasks)** — pass@1 (no verifier) 73.7% · CLM verifier 81.6% · Jev verifier 71.1%
- **Terminal-Bench 2.1 (30 tasks)** — pass@1 (no verifier) 84% · CLM verifier 87.6% · Jev verifier 83.1%

*DeepSWE: best-of-4 candidates from Opus 5; Terminal-Bench 2.1: best-of-5 from Fable 5. Jev scored below pass@1 on both. Verifier latency: 79 vs 449 ms (DeepSWE) and 32 vs 131 ms (Terminal-Bench 2.1), CLM vs Jev.*
*Source: CLM repository*

> **Read the fine print** — These verifier results use fine-tuned heads, not the zero-shot checkpoint you download, and they come from task subsets, not full leaderboard submissions.

## Scaling laws

The authors report that the InfoNCE loss falls as a power law in compute, data, head size and encoder size, with encoder size giving the strongest gains. The optimal head size grows roughly in proportion to the data (∝ D^1.02), at about 310 tokens per parameter. A multimodal CLM-35B is announced for early October 2026.

## How to run it

CLM needs one NVIDIA GPU under Linux. vLLM serves the Qwen3-8B pooling encoder on port 8090, and clm-serve exposes the CLM API and a playground on port 8700. States longer than 2,048 tokens are truncated by default.

*Install and serve (from the CLM repository)*
```bash
pip install contrastive-lm
vllm serve Qwen/Qwen3-8B --served-model-name qwen3-8b --runner pooling --max-model-len 2048 --port 8090 &
clm-serve
```

The API has three question types: Noul (the probability a statement is true), Choice (pick one option, with probabilities) and Score (the expected level on an ordered rubric). The endpoints are POST /v1/systemone, POST /v1/rank, GET /v1/models and GET /health. The project's example routes a support message:

*Typed decisions with the Python client (CLM project example)*
```python
from clm import CLMClient, Choice, Noul, Score

# ... create a CLMClient pointed at clm-serve, then:
client.system_one(
    state="Customer: my invoice was charged twice and nobody answers the phone!",
    questions={
        "urgency": Noul(instructions="Is this urgent?"),
        "department": Choice(
            instructions="Which team should handle this?",
            criteria={"billing": "Charges, invoices, refunds", "technical": "Bugs and outages"},
        ),
        "frustration": Score(
            instructions="How frustrated is the customer?",
            criteria=["Calm", "Frustrated", "Very angry"],
        ),
    },
)
# department -> billing {'billing': 0.93878, 'technical': 0.06122}
```

## Limitations

- Encoder-locked: CLM-8B depends on Qwen3-8B's last-token pooled embeddings.
- No generation: it only scores options you provide, and its probabilities are relative to that set.
- The state-of-the-art verifier numbers need fine-tuned heads.
- Text only for now; the multimodal CLM-35B is announced for early October 2026.

## Where it fits in an agent

The repository lists four uses: ranking best-of-N trajectories, routing tools, shortlisting retrieval pools and making typed decisions. In an agent built as a graph, those map onto specific nodes:

**CLM as fast nodes in an agent graph**
- **Orchestrator** — A large model plans and generates; CLM handles the quick choices around it.
  - **Tool router** — Score the next tool against the current state; cached tool embeddings keep it cheap.
  - **Best-of-N verifier** — Rank N candidate patches or trajectories and keep the top one.
  - **Retrieval shortlist** — Narrow a large candidate pool before a slower reranker or the LLM reads it.
  - **Typed decision** — Noul, Choice or Score answers: urgency, department, severity.
- Then **Confidence threshold** — Low-probability decisions escalate to the large model or a human.

The confidence gate is our recommendation, not part of CLM: because probabilities are relative to your options, a flat distribution is a useful signal that none of them fit. For how router and verifier nodes fit into a larger design, read [what graph engineering is](https://bytevancer.com/blog/what-is-graph-engineering). The weights are on the Hugging Face Hub; our guide to [Hugging Face for AI engineering](https://bytevancer.com/blog/hugging-face-for-ai-engineering) uses the CLM-8B release as its worked example.

## Frequently asked questions

**What is CLM-8B?**
CLM-8B is an open contrastive language model from Contrastive-LM. Instead of generating text, it scores candidate actions against the current state and returns probabilities. It uses a frozen Qwen3-8B encoder with two ~20M-parameter projection heads, and the weights and code are Apache 2.0.

**How much faster is CLM-8B than Jev?**
In the authors' zero-shot tests CLM was faster on all four tasks: 16.5 vs 149.8 ms on T-Rex (the source of the "up to 9x" figure), 76.8 vs 125.5 ms on BFCL v4, 79.8 vs 225 ms on WikiRacing and 33.5 vs 132.6 ms on Super Mario. The model card also reports 13x faster with around 1,000 candidates.

**Is CLM-8B more accurate than Jev?**
Not everywhere. Both succeeded 5/5 on T-Rex and Super Mario, but CLM trailed on BFCL v4 tool calling (95.2% vs 99.2%) and WikiRacing (26/30 vs 30/30). As a fine-tuned best-of-N verifier it beat both pass@1 and Jev on the DeepSWE and Terminal-Bench 2.1 subsets tested.

**What hardware do I need to run CLM-8B?**
One NVIDIA GPU under Linux. vLLM serves the Qwen3-8B pooling encoder on port 8090 and clm-serve runs the CLM API and playground on port 8700. The project's cache benchmarks were measured on an RTX 4090.

**Can CLM-8B replace an LLM in my agent?**
No. It cannot generate text or propose new options; it only ranks the options you give it. It works best next to an LLM: routing tools, verifying candidate outputs, shortlisting retrieval results and answering typed questions quickly.

## Keep reading

- [What Is Graph Engineering? Designing Multi-Agent Systems as Graphs (2026)](https://bytevancer.com/blog/what-is-graph-engineering) — Graph engineering explained: designing multi-agent AI systems as explicit graphs of nodes, edges and loops, with governance tips and a LangGraph example.
- [What Is Hugging Face Used For? A Guide for AI Engineering Teams](https://bytevancer.com/blog/hugging-face-for-ai-engineering) — What Hugging Face is used for in AI engineering: the Hub's models and datasets, Spaces demos, Inference Providers and agent tooling, with a worked example.
- [All ByteVancer engineering articles](https://bytevancer.com/blog)

## Need faster, cheaper decisions in your AI apps?

We build AI into cloud applications — routers, verifiers and the harness around them — and measure what each model call is worth. See our [AI and machine learning integration services](https://bytevancer.com/services/cloud-applications) or [talk to the team](https://bytevancer.com/contact).

## Sources

- [Contrastive-LM — CLM on GitHub](https://github.com/Contrastive-LM/CLM)
- [Hugging Face — Contrastive-LM/CLM-v0.1-8B model card](https://huggingface.co/Contrastive-LM/CLM-v0.1-8B)
- [Michal Sutter — Contrastive-LM releases CLM-8B, MarkTechPost (Sep 23, 2026)](https://www.marktechpost.com/2026/09/23/contrastive-lm-releases-clm-8b-an-open-system-one-model-that-scores-agent-actions-up-to-9x-faster-than-jev/)
- [Superpower Daily — Contrastive-LM releases an open agent decision model (Sep 24, 2026)](https://superpowerdaily.com/posts/contrastive-lm-releases-an-open-agent-decision-model-that-runs-faster-than-jev-in-its-tests)

---

*This is the Markdown copy of https://bytevancer.com/blog/clm-contrastive-language-models — the page itself has the interactive charts and diagrams.*