NandhaKishorM/laya instruction manual

1. Purpose, scope, and relationship to Claude Code

Laya is an Apache-2.0 Python decision engine, not a text generator. It evaluates a state—plain text, email, ticket, conversation, or JSON-like document—against typed questions in one encoder forward pass. The three primitives are `choice` for one label from a closed set, `score` for an expected position on an ordered rubric, and `noul` for P(true). The repository documents three checkpoints: `laya` for English, `laya-multilingual` for 100+ languages, and `laya-typed-decisions` for the fine-tuned typed-decisions workflows. `Router` selects a checkpoint by script/language or accepts an explicit override.

**Claude Code classification: unclassified.** The supplied first-party files do not name Claude Code or document a Claude Code plugin, hook, skill, command, settings entry, or installation path. Laya does provide standalone Python and TypeScript libraries, CLIs, HTTP services, an MCP extra, and framework integrations, but the evidence does not establish how any of these are wired specifically into Claude Code. Do not treat the MCP extra alone as proof of a Claude Code mechanism.

Install Python 3.10 or newer in an existing environment:

python -m pip install laya

Or create an isolated macOS/Linux environment and verify the installed version without letting a source checkout shadow it:

python3 -m venv .venv
.venv/bin/python -m pip install laya
.venv/bin/python -I -c "import laya; print(laya.__version__)"

On Windows PowerShell, the documented equivalent is:

py -3.11 -m venv .venv
.\.venv\Scripts\python.exe -m pip install laya
.\.venv\Scripts\python.exe -I -c "import laya; print(laya.__version__)"

The first checkpoint use requires Hugging Face access unless models are already local. Optional extras include `serve`, `mcp`, `langchain`, `llamaindex`, `crewai`, `onnx`, and `fast`.

2. Core prediction API and the three question types

The recommended API is `Router.predict(state, questions)`. A state may be text or structured data; a question mapping identifies each requested decision. A `choice` question maps labels to descriptions and returns the winning `choice`, per-option probabilities, and confidence. A `score` question supplies ordered descriptions and returns an expected numeric level plus its distribution. A `noul` question returns the probability that the proposition is true.

from laya import Router

router = Router()
state = "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
questions = {
    "department": {
        "type": "choice",
        "instructions": "Which department should handle this?",
        "criteria": {
            "billing": "invoices, payments, refunds",
            "technical": "bugs, outages, system errors",
            "other": "everything else",
        },
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this?",
        "criteria": ["not urgent", "soon", "blocking"],
    },
    "churn_risk": {
        "type": "noul",
        "instructions": "Does the user threaten to cancel or leave?",
    },
}

result = router.predict(state, questions)
print(result["answers"]["department"]["choice"])
print(result["answers"]["churn_risk"]["noul"])
print(result["routing"]["model"])

`noul` always means P(true). Its optional `criteria` can describe `false` and `true`; optional `labels` changes only the text shown to the model and must contain distinct, non-empty `false` and `true` strings. `option_order` changes presentation slots without changing the meaning or returned key order. Because documented position sensitivity can change answers, validate ordering on your data.

`laya.load(...)` provides direct single-checkpoint operation. `agent.predict` and its alias-compatible `system_one` return `answers` and `usage`; with no questions, they return empty answers without tokenizing or running the model. Use direct loading only when checkpoint selection is already known.

3. Routing, model residency, batches, and long documents

`Router.route(state, questions)` inspects routing without inference or checkpoint loading. Explicit `model`, `task`, or `lang` wins over detection. `lang_guess` accepts a known language code or callable; English-like codes select the English checkpoint, other language codes select multilingual, and uncertain values such as `None`, `und`, `C`, or `C.UTF-8` defer to built-in detection. Set `default="multilingual"` when ambiguous short Latin-script traffic is usually non-English.

from laya import Router

router = Router(default="multilingual")
print(router.route({"body": "Esqueci minha senha"}).model)
print(router.route({"body": "Please refund the duplicate charge"}).model)

`Router(preload=True)` loads all configured checkpoints. `preload([...])` loads selected models; `attach(name, agent)` reuses an existing agent; `unload()` frees resident models. `max_loaded` controls the LRU cache. The default keeps two checkpoints; `max_loaded=1` can cause a reload on every language switch.

`Agent.predict_batch(states, questions, batch_size=N, sort_by_length=True)` scores many states against one schema and preserves input order. `Router.predict_batch(requests, ...)` first routes heterogeneous requests, groups compatible checkpoint/schema/token-budget combinations, batches them, and restores order. `predict_many` aliases `predict_batch`; `route_batch` performs ordered routing only. Length sorting helps mixed-length workloads only when `1 < batch_size < len(states)` and can introduce small floating-point differences.

requests = [
    {"state": "Please refund invoice 1", "questions": questions},
    {"state": "تم خصم المبلغ مرتين", "questions": questions},
    {"state": "Please refund invoice 2", "questions": questions},
]
results = Router(max_loaded=1).predict_batch(requests, batch_size=8, sort_by_length=True)

`predict_long` scans an oversized state in overlapping windows. `noul` uses the strongest window; `choice` and `score` use the most-confident window. The result adds `usage["windows"]` and, when attribution remains valid, an answer `window` record. Its selected-window probability is not calibrated for the whole document. The multilingual model can use `max_len=8192`, but documented accuracy becomes variable on very long inputs.

4. Confidence, abstention, structured decisions, presets, and hooks

Use `answer_confidence` for action policies, not as proof of correctness. The supplied documentation says shipped checkpoints are over-confident and multilingual has no fitted temperatures, so fit calibration and validate thresholds on held-out workload data. `min_confidence` is opt-in on prediction and decision calls: low answers retain raw values but receive `low_confidence: True`; schema-driven `decide` returns `None` for a low-confidence field unless details are requested.

res = agent.predict(state, questions, min_confidence=0.85)
ans = res["answers"]["department"]
if ans.get("low_confidence"):
    escalate_to_human_agent(ans["choice"], reason=f"Low confidence ({ans['answer_confidence']:.2f})")
else:
    route_automatically(ans["choice"])

`decide(state, schema=...)` converts supported JSON Schema fields into questions: enums become choices, booleans become `noul`, and bounded integers become scores. `return_details=True` retains field-level probabilities and confidence. `decide_batch` plans one schema once and batches states; it exists on Agent, Router, and ONNXAgent, with `laya.decide_batch(runner, ...)` as a function form.

schema = {
    "type": "object",
    "properties": {
        "department": {"type": "string", "enum": ["billing", "support", "sales"]},
        "urgency": {"type": "integer", "minimum": 0, "maximum": 2},
        "needs_human": {"type": "boolean"},
    },
}
values = agent.decide("I was charged twice, refund me.", schema=schema)

Preset builders are `router_questions()`, `guard_questions()`, `moderation_questions()`, `triage_questions()`, and, as documented in examples/clients, `email_questions()`.

Prediction hooks may implement `on_predict_start`, `on_predict_end`, `on_route`, `on_load`, `on_evict`, and `on_error`. Start hooks can rewrite state/questions or call `ctx.skip(...)`; end hooks can replace results. Hooks support auditing, redaction, caching, metrics, gating, and routing overrides. They are opt-in; server requests reject callable hook fields because hooks must live in the server process.

5. Command line, local playground, and production HTTP service

The `laya` CLI can route without loading weights, predict with a checkpoint, force a language/model, use a preset or custom question JSON, process a file, read JSONL from stdin, or enter interactive mode.

laya "I was charged twice, please refund"
laya "Refactor this service" --predict
laya "Mein Konto wurde zweimal belastet" --lang de
laya "My payment failed twice" --preset triage
laya --batch tickets.txt --predict
cat tickets.txt | laya --batch - --predict --json
laya "Where is my card" --questions intents.json
laya

`--batch-size` bounds each pass; `--sort-by-length` groups similar lengths and requires a useful intermediate batch size. `--max-len` and `--head-max-len` control state and option budgets. High-cardinality choices can lose distinguishing option text; narrowing candidates first is safer than assuming a larger budget always improves accuracy.

For a development GUI and JSON API, install the serving extra and run the repository example:

pip install "laya[serve]"
python examples/server.py

It serves a playground at `http://127.0.0.1:8000`, plus `/predict`, `/predict/batch`, and `/models`. This example supports lazy loading with `--no-preload` and device pinning with `--device cuda|cpu|mps`.

For the documented Jev-compatible production surface, run:

pip install "laya[serve]"
LAYA_DEVICE=cuda LAYA_PRELOAD=1 laya-serve

The service exposes `POST /v1/systemone` and `POST /v1/systemone/batch`; batch responses preserve order and include aggregated usage. Configure it with environment variables including `LAYA_HOST`, `LAYA_PORT`, `LAYA_DEVICE`, `LAYA_PRELOAD`, `LAYA_MODELS`, `LAYA_THREADS`, `LAYA_AUTO_TASK`, `LAYA_MAX_LOADED`, `LAYA_ROOT_PATH`, and `LAYA_API_KEY`. When an API key is set, clients send Bearer authentication. The server caps a batch at 64 states and a choice at 100 options. Put TLS and browser-facing authentication at an application proxy or gateway.

6. Acceleration, ONNX, language calibration, and framework adapters

Device selection defaults through CUDA, MPS, then CPU. XPU requires a suitable Intel driver and XPU-enabled PyTorch wheel; `device="xpu"` can be passed to `laya.load` or `Router`. CUDA/CPU autocast policy can be adjusted with documented environment variables, but calibrate and evaluate in the same precision used for service because probabilities and boundary decisions may move.

The optional TileLang path installs with `laya[fast]`. `laya.load(..., fast=True)` or `agent.accelerate()` enables fused CUDA kernels; `agent.deaccelerate()` restores the stock path. It falls back on CPU/MPS or without TileLang. `compile=True` compiles graph shapes lazily; `agent.warmup()` runs synthetic shapes before traffic so live requests do not pay first-compile stalls.

agent = laya.load("convaiinnovations/laya", compile=True)
agent.warmup()

`ONNXAgent` documents `predict_batch`, `predict_long`, and `decide_batch` with contracts matching the PyTorch APIs. The release notes also document ONNX export quantization and `laya-evals run --onnx`, but no complete export command is supplied in the selected main documentation except the TypeScript-specific exporter covered later.

Per-language `lang_temperatures` can replace a checkpoint temperature or option-count bucket for matching languages. Language keys normalize to their base subtag. Fit such values on language-specific evaluation data rather than copying arbitrary values.

The LangChain/LangGraph extra provides `LayaRouter`, `LayaGuardrail`, and `LayaDecision`. `LayaRouter` is a conditional-edge router with confidence fallback, `LayaGuardrail` can raise `LayaGuardrailError`, and `LayaDecision` projects JSON Schema results. Their `batch()` and `abatch()` methods use Laya batch inference rather than LangChain’s one-call-per-input default. LlamaIndex selectors and CrewAI routing are named optional integrations, but no source-supported usage snippet for them is included in the supplied files, so consult the repository’s first-party integration documentation before use.

7. JavaScript and TypeScript: HTTP client versus local ONNX

There are two distinct JavaScript paths. `laya-client` is a dependency-free ESM/CommonJS HTTP client for Node.js 22+ and browsers; inference remains in a self-hosted Python `laya-serve`. `laya-ts` is ESM-only and runs split ONNX models directly in Node or a browser.

For the HTTP client, start the Python service, then build the checked-out SDK:

python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[serve]'
LAYA_HOST=127.0.0.1 LAYA_MODELS=english laya-serve

cd sdk/typescript
npm ci
npm run build
node examples/triage.mjs

`new Laya({baseURL, apiKey, timeoutMs, model, headers, fetch})` configures the client. `predict(state, questions, options)` supports cancellation with `AbortSignal` and per-call timeout. `health()` is separate and is not called by prediction. Preset functions mirror Python. Errors include API, timeout, validation, connection, abort, and response errors; requests are never automatically retried. Do not embed a shared server key in browser code.

For local ONNX inference, first export a checkpoint:

python laya-ts/scripts/export_onnx.py --model-dir <ckpt> --out-dir ./model

Then load it from Node:

import { Agent, Router } from "laya-ts";

const agent = await Agent.load("./model");
const router = new Router();
router.attach("english", agent);
const out = await router.predict({ body: "charged twice, refund please" }, {
  intent: { type: "choice", instructions: "What does the customer want?", criteria: { refund: "money back", other: "anything else" } },
});
console.log(out.answers.intent);

`laya-ts` also documents hooks, `decide`, truncation fields, `predictBatch`, `routeBatch`, `predictMany`, shortlist helpers, and per-language temperatures. Python’s `sort_by_length` is explicitly not ported. Browser loading tries WebGPU and falls back to WASM; the documented demo’s first model transfer is about 1.3 GB.

8. Evaluation, research harnesses, examples, and operating limits

The repository’s 41 Python examples form an eight-stage path from first prediction through primitives, state shapes, confidence, routing, presets, schema design, devices, batching, long documents, caching, monitoring, and a production triage service. Run one example or the complete smoke suite from the repository root:

python examples/01_first_call_minimal.py
./examples/run_all.sh

Each example is a separate process and loads real checkpoints; the suite is not a lightweight CI test.

For your own labelled JSONL, `laya-evals` validates data, runs evaluation, slices results, enforces minimum accuracy/maximum ECE, and compares against a baseline. Required row fields are `state`, `questions`, and `expected`; optional fields are `tags`, `language`, and `model`.

laya-evals validate research/evals/fixture.jsonl
laya-evals run data.jsonl --model english --min-accuracy 0.8 --max-ece 0.05 --slice language
laya-evals run data.jsonl --baseline baseline.json --tolerance choice_accuracy=0.02 --json out.json

The independent MASSIVE harness reports per-language accuracy, macro-F1, ECE, confidence, coverage, and every case:

pip install laya datasets
python research/eval/laya_eval.py --model convaiinnovations/laya --langs en,de,ro --out report.json

Research also documents presentation checks and metamorphic option-order/label-renaming diagnostics. These measure robustness, not correctness, and high-option-budget differences can confound comparisons. The Chinese benchmark archives are small synthetic diagnostics, not general rankings; their offline audits rederive stored results without model downloads.

Operational limits matter: route before inference because English can be confidently wrong on unreadable scripts; calibrate on held-out data; treat long-document selected-window probabilities cautiously; test option order and wording; avoid assuming `score` is as strong as other primitives; and evaluate high-cardinality choices for truncation/collisions. The documented fine-tuning notebook builds data, trains, fits calibration temperatures, evaluates, and pushes a result on Kaggle 2×T4. Its reported improvement is specific to a checkpoint fine-tuned on that benchmark’s training split, not a zero-shot guarantee.