Pi-AutoResearch
Plugin for Pi (Matthias 'badlogic' Müller's CLI coding agent) that extends Karpathy's AutoResearch pattern beyond ML training: give it a benchmarkable target (test speed, bundle size, Lighthouse score, memory allocations), the agent iterates overnight, proposes an optimization, measures, keeps the commit if it improves, reverts otherwise. Canonical proof: Tobi Lütke (CEO Shopify) used it on Liquid to gain 53% perf in 93 autonomous commits (120 experiments), surfaced by code_simple on Instagram.
Strengths
- Generalizes the AutoResearch pattern to any benchmarkable target, not limited to ML training
- Tobi Lütke / Shopify Liquid proof: 93 autonomous commits, 53% perf gain, 61% fewer allocations, the canonical 2026 reference
- Plugin for Pi (badlogic), lightweight terminal agent, no heavy harness setup needed
Limitations
- Requires a measurable and stable target (reproducible bench script), without it, the agent validates noise
- Continuous LLM cost: each experiment calls the agent, 100 experiments can get expensive
- Coupled to Pi (badlogic), not a standalone harness, assumes you use Pi as your CLI
Best for
- Perf optimization on prod code with a stable benchmark (Liquid, parsers, SQL hot path)
- Bundle size or Lighthouse score reduction with a measurable budget
- Tech leads who want to demonstrate autonomous agent value on application code (not just ML)