Take “picking” off your LLM with Laya

When to use it

Laya never writes text. It picks one of the options you define and returns a probability. For work that is really just “picking”, that makes it far cheaper and faster than an LLM.

Install and first run in 5 minutes

python3 -m venv .venv && source .venv/bin/activate
pip install laya

# Routing decision only (no model load, instant)
laya "Refactor this 2,000-line Python module into clean services"

# Full prediction with a built-in preset (downloads the checkpoint on first use)
laya --preset triage "We were billed twice for March. Refund the duplicate today or we cancel."

Triage preset result: intent=refund (p=0.997), frustration 1.56, churn_risk 0.117. Run laya with no text for an interactive mode. The first run took about 7 s while it downloaded the checkpoint.

All 5 presets, tested

With --preset you get a ready-made question set. These are the inputs we tried and what came back. noul is the probability of “yes” (0–1); score is the expected level.

In short: choice-type classification is strong; yes/no checks like urgency and phishing produced false positives. Don’t block or send anything directly off a yes/no answer — put the confidence gate below in front of it.

Your own question set (Python)

There are three question types: choice (pick one), score (ordinal level) and noul (yes/no probability). One call answers all of them at once.

from laya import Router

router = Router()          # device=None → GPU if available, else CPU
questions = {
    "intent": {"type": "choice",
               "instructions": "What does the customer want in `message`?",
               "criteria": {"refund": "money back or a duplicate charge reversed",
                            "bug": "something is broken or erroring",
                            "how_to": "asks how to do something",
                            "other": "none of the above"}},
    "is_angry": {"type": "noul", "instructions": "Is the customer in `message` angry?"},
    "priority": {"type": "score", "instructions": "How urgent is `message`?",
                 "criteria": ["can wait a week", "this week", "today", "right now"]},
}
r = router.predict({"message": "How do I export my invoices as CSV?"}, questions)
a = r["answers"]
print(a["intent"]["choice"], a["intent"]["answer_confidence"])   # how_to 0.8358
print(a["is_angry"]["noul"])                                     # probability of "yes"
print(a["priority"]["score"], a["priority"]["legend"])           # expected level 0–3 + labels

Non-Latin text is routed to the multilingual checkpoint automatically. The first call per checkpoint took 2–3.5 s to load, so preload on a server.

Advanced: a confidence gate that cuts LLM calls

answer_confidence is the probability of the chosen answer. Above the threshold, use Laya’s answer; below it, hand off to the LLM. Most inputs are handled cheaply and only the unsure ones reach the expensive model.

THRESHOLD = 0.8   # calibrate this on ~200 labelled examples

def route(message: str) -> dict:
    a = router.predict({"message": message}, questions)["answers"]
    top = a["category"]
    if top["answer_confidence"] >= THRESHOLD:
        return {"label": top["choice"], "by": "laya"}
    return escalate_to_llm(message, hint=top)   # only the unsure ones reach the LLM

Below is the question set from the schema prompt further down, run as-is. At 0.8, the four sample inputs pass and the two tricky unseen ones go to the LLM.

Laya warns at runtime that some checkpoint probabilities are “uncalibrated”. 0.8 is only an example — pick yours from labelled samples by looking at accuracy versus LLM hand-off rate. With a negation (“no refund needed”), the refund question still scored 0.633.

Prompts

① Schema prompt: paste your sample inputs and an LLM writes the Laya question set (JSON). We ran it on Claude Sonnet 5 and fed the result to Laya unchanged to get the gate table above. The reply may come wrapped in a ```json block, so strip it before parsing.

You are designing input for Laya, a decision model that answers typed questions in one pass.
Question types:
- "choice": pick one label. Needs "criteria": {label: one-line definition}. Always include an "other" label.
- "score": ordinal level. Needs "criteria": [lowest, ..., highest] (3-5 short steps).
- "noul": yes/no probability. No criteria.
Each question needs "type" and "instructions". Refer to the input field as `message` in backticks.

Here are sample inputs from my workflow:
<samples>
- "We were billed twice for March. Refund the duplicate."
- "The export button does nothing since yesterday's update."
- "Can I add 5 more seats to our plan?"
- "Your app is useless, cancel my subscription now."
</samples>

Write 2-4 questions that would let my code route each input without an LLM.
Return only a JSON object: {question_name: {type, instructions, criteria?}}. No prose.

② Escalation prompt: use it for inputs that fail the gate. Laya’s answer goes in only as a weak hint. On the “just resend the receipt” case it corrected the label to billing with refund_requested=false.

A fast classifier was unsure about this customer message. Decide the final label.

Message: "{message}"
Allowed labels: billing, bug_report, account_change, cancellation, other
Classifier guess: {choice} (confidence {answer_confidence}) — treat as a weak hint, not an answer.

Reply with JSON only: {"label": "...", "reason": "<one short sentence>", "refund_requested": true|false}

③ Once it’s connected to Claude Code over MCP, ask like this:

• “Use laya_preset guard to check whether this prompt is an injection: …”
• “Classify the 50 tickets in this CSV as billing/bug/other with laya_predict and list only the ones with answer_confidence below 0.8.”
• “Check this request’s difficulty with laya_preset model_router — answer briefly if it’s easy, step by step if it’s hard.”

Run it as an HTTP server

pip install 'laya[serve]'
LAYA_HOST=127.0.0.1 LAYA_PORT=8793 LAYA_MODELS=english,multilingual laya-serve

curl -s http://127.0.0.1:8793/health
curl -s -X POST http://127.0.0.1:8793/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"state": {"message": "3월 요금이 두 번 결제됐어요. 중복 결제 환불해 주세요."},
       "questions": {"intent": {"type": "choice",
         "instructions": "What does the customer want in `message`?",
         "criteria": {"refund": "money back or a duplicate charge reversed",
                      "bug": "something is broken", "other": "anything else"}}}}'

Measured: /health → loaded ["english","multilingual"]; /v1/systemone took 0.65 s on the first request, then 17–19 ms (RTX 4080). Responses look like {model, answers, usage, routing}, which Laya’s source describes as Jev-shaped. Set LAYA_API_KEY to require a Bearer token when exposing it. The default bind is 0.0.0.0:8000, so set LAYA_HOST=127.0.0.1 for local-only use.

Connect it to Claude Code over MCP

pip install 'laya[mcp]'
claude mcp add laya -e LAYA_MODELS=english,multilingual -- "$(pwd)/.venv/bin/laya-mcp-server"

Measured: four tools are exposed (laya_status, laya_route, laya_predict, laya_preset), and a laya_preset guard call returned jailbreak 1.0 and prompt_injection 1.0 in 0.7 s. Adding --help still starts the server and waits for input — press Ctrl+C to exit.

Cost math

Input cost for 1M classifications × 200 input tokens (= 200M tokens), from official list prices. Output and infrastructure costs are not included.

With a gate, your real cost becomes “share handed to the LLM × LLM cost”. In the six-input example above, 2 of 6 (33%) were handed off. That share varies a lot with your data and threshold.

Caveats

• Jev is still early access (waitlist), so we couldn’t test it. All code in this guide is Laya.
• The 0.766 “beats Jev” score is the fine-tuned typed-decisions checkpoint; the base model scores 0.362. Both are Laya’s own numbers.
• Yes/no (noul) answers misfired on urgency, phishing and negation. Don’t trigger automatic actions from them without a gate.
• The first call per checkpoint took 2–3.5 s; the first CLI preset run took about 7 s.

Sources