Metamorphic robustness harness

Measures option-order and label-renaming stability, probability drift, budget confounding, and selective-prediction signals.

Parent repository GitHub

Type: research workflow
Repository: NandhaKishorM/laya
Readiness: Usable for evaluation and carefully validated deployments, but shipped calibration, multilingual task quality, option-order sensitivity, and workload-specific accuracy require local testing before automated actions.
Keywords: 6

Location

Repository path

research/eval/metamorphic.py

Invocation

python -m research.eval.metamorphic --model MODEL --langs LANGS --out FILE

Setup

Installation / activation

Python module execution

Keywords

metamorphicrobustnessoption orderlabel renamedriftevaluation

Repository context

NandhaKishorM/laya

Apache-2.0 multilingual decision-model toolkit that provides Python and TypeScript SDKs, local CLI and HTTP serving, ONNX runtimes, framework integrations, evaluation harnesses, and an optional MCP server for fast typed choice, ordinal-score, and yes/no decisions; no native Claude Code plugin, hook, skill, or Claude-specific installation path is evidenced.

machine learningnatural language processingclassificationdecision routingmultilingual inferenceguardrailscontent moderationsupport automationmodel routingevaluation

Open full repository research