Stage 1: Iterate Locally (Just `flopscope`)
Sourced from whest-starterkit @
aaa3882.
Stage 1: Iterate Locally (Just flopscope)
"Just flopscope" means: no whest CLI required. You run python estimator.py and the bundled local_engine.py constructs an MLP, calls your predict() inside a flopscope.BudgetContext, and sweeps Monte-Carlo sample counts to print a FLOPs-vs-MSE table. The whestbench.BaseEstimator and whestbench.MLP types you'll see imported are just the shared dataclasses — they don't pull in the harness.
Iterate here until predict() converges, then climb to Stage 2 to confirm the contract.
🚀 Run it
uv run python estimator.pyYou should see a table like:
--- Your estimator ---
MLP: width=256 depth=8 seed=0
n_samples | sampling_flops | estimator_flops | MSE
----------------------------------------------------------
10 | 10,551,808 | 0 | 0.706450
100 | 105,499,648 | 0 | 0.718928
1,000 | 1,054,978,048 | 0 | 0.739125
10,000 | 10,549,762,048 | 0 | 0.735092
100,000 | 105,497,602,048 | 0 | 0.737172The stub predict() returns all zeros, so estimator_flops is 0 and the MSE
plateaus at the variance of the true outputs — once you put real math in
predict(), both columns come alive and the MSE should shrink roughly as
1/sqrt(n_samples) (Monte Carlo converging to your estimator's answer).
Edit predict()
Open estimator.py. The body of predict() returns all zeros — replace it with your idea. The template already imports flopscope as flops and flopscope.numpy as fnp, so any array op you write through fnp (or via Python operators on fnp arrays) is FLOP-counted automatically. Re-run; the MSE column tells you how close you are, and estimator_flops shows what your math cost.
Compare against a baseline
uv run python estimator.py --baseline mean_propagationThis loads examples/02_mean_propagation.py and runs both estimators on the same MLP.
✅ Expected outcome
| Estimator | MSE on the default MLP | Status |
|---|---|---|
| Zeros template (default) | ~0.74 | floor — natural variance of the activations |
--baseline mean_propagation | ~0.0005 | ~1400x better; first-order analytical |
--baseline covariance_propagation | ~0.00002 | ~20x better than mean; tracks neuron correlations |
You're ready for Stage 2 once your estimator's MSE is comfortably below
the zeros floor and estimator_flops stays under whatever budget you'd
target downstream (Stage 3's grader default is 6.8e10).
✅ When you're ready
Move on to Stage 2: validate the contract.