The loop is fixed. What you configure is how much of it to run, on which models, and how the next candidate is chosen.
One prompt set from the current best group.
A reflection model is shown a handful of that parent's failures — the inputs, the expected answer, what came back — and asked to rewrite one of the prompts so those failures stop. One prompt at a time, so that when something improves you know which change did it.
The rewrite is evaluated on the same small batch of rows the parent was judged on. If it does not beat its parent there, it is discarded. This is what keeps a full evaluation — the expensive part — for candidates that have shown something.
Survivors are evaluated on the full training set and join the tree. Everything is kept: the prompts, the diff against the parent, the rows the reflector saw, the scores.
Improve accuracy asks the rewriter to add whatever rule fixes the failures it was shown. Compress asks for a shorter prompt that keeps the same answers, and sets a negative template_tokens weight so the objective agrees.
Explore samples from the best set — cheap, robust, the default. Guided builds a model of past results and picks; it spends more reflection calls and earns them back under noisy scores or a weak reflection model, and not on flat tasks. The comparison at equal rollouts →
The rows the reflection model sees, and the rows a rewrite must beat its parent on. Small is the point. Downsample the evaluation set if you need cheaper rounds; never the check batch.
Rounds cap the work, max spend caps the money, and “stop after N rounds without improvement” stops a search that has plateaued. All three are live at once; whichever comes first wins.
Reflection is the one place where variety helps, so it has its own temperature. Evaluation is always temperature 0, which narrows — but does not close — the gap between two evaluations of the same prompt: several providers still answer differently at temperature 0. The pilot measures that gap so you can read a result against it.
A short written critique from a model that reads the trace (the default), a template you write, or just the expected answer and the scores. The critique costs a call per failure and usually earns it.
The search engine is bpto, open source: tree search over prompt rewrites, with GEPA-style reflective selection and Bayesian optimization over prompt embeddings as the two selectors. Its experiments, plots and standing conclusions live in that repository and are quoted on quantecarlo.com/findings. Impromptune is the studio around it — the canvas, the data, the validation, the versions and the traces.