Skip to main content
What it contains
Minimal autonomous LLM research loop: agent edits train.py, runs fixed five-minute experiments, checks val_bpb, and keeps/discards changes.
Typical uses
Let a coding agent explore model/optimizer/training changes on a bounded single-GPU setup while the human controls program.md.
Why people choose it
Inference: chosen for tightly scoped autonomous experiments with a simple metric and reviewable diffs.
Last meaningful updateUnknown
Production readinessprototype
Classification
Subject domains
LLM researchautonomous experimentation
Task categories
LLM researchautonomous experimentation
Project phases
researchimplementation
Secondary types
experiment harnessagent workflow
Select when
- Need autonomous small-LLM experiments on suitable NVIDIA GPU
Do not select when
- General SWE
- No suitable GPU
- Need cross-machine metric comparability
Implementation complexitymedium
Routing heuristic 71.0%
Security boundary clarity
5/5 First-party routing documentation
5/5 Strengths
None documented.
Poor-fit scenarios
- General SWE
- No suitable GPU
- Need cross-machine metric comparability
Limitations
None documented.
prepare.py
Fixed data/tokenizer/eval utilities.
datatokenizereval
train.py
Only file agent modifies.
modeloptimizertraining
program.md
Human-authored research-agent policy.
instructions
LanguagesPython, Markdown
RuntimesPython >=3.10, uv
Operating systemsNone documented
Installation methodsNone documented
InterfacesNone documented
Required credentialsNone documented
External servicesNone documented
Hardwaresingle NVIDIA GPU; tested H100
Major dependenciesNone documented
Compatibility notes
None documented.
One-sentence semantic summary
Minimal autonomous LLM research loop: agent edits train.py, runs fixed five-minute experiments, checks val_bpb, and keeps/discards changes.
Capability keywords
agent-edited train loopfixed-time experimentsvalidation metricexperiment log
User intent phrases
- Need autonomous small-LLM experiments on suitable NVIDIA GPU
Negative match phrases
- General SWE
- No suitable GPU
- Need cross-machine metric comparability
Differentiators
None documented.
Security considerations
- Run agent with restricted permissions; bound compute/cost.
Privacy considerations
None documented.
Uncertainty
Unresolved items
None documented.
Inference notes
None documented.
Repository usage guidance
installation summaryUnknown
basic usage summaryLet a coding agent explore model/optimizer/training changes on a bounded single-GPU setup while the human controls program.md.
documented entry pointsNone documented
key configuration filesNone documented
important directoriesNone documented
documentation pathsREADME.md
example pathsNone documented
Relationships
downstream
high
Autoresearch uses a simplified nanochat-derived training setup.
Recommended combinations
LLM research loop high
Each repository covers a distinct layer; avoid combining tools that duplicate lifecycle ownership.
Order of operations
- Understand/reproduce nanochat baseline
- Set up autoresearch on suitable GPU
- Define program.md policy
- Run bounded experiments and review logs
Routing rules
rule_040: Need autonomous small-LLM experiments on suitable NVIDIA GPU P960
Rationale
Minimal autonomous LLM research loop: agent edits train.py, runs fixed five-minute experiments, checks val_bpb, and keeps/discards changes.
Required conditions
None documented.
Preferred
karpathy/autoresearch
Fallback
None