EML Symbolic Regression — discover elementary formulas from data using a single operator.
Based on Odrzywolek (2026), "All elementary functions from a single operator":
the EML operator eml(x, y) = exp(x) - ln(y) (Exponent Minus Log), together with the constant 1, generates all standard
elementary functions. This makes S → 1 | x | eml(S, S) a complete, regular search space for
symbolic regression.
import numpy as np
from eml_sr import discover
x = np.linspace(0.5, 5.0, 50)
y = np.log(x) # secret formula
result = discover(x, y, max_depth=4, n_tries=8)
print(result["expr"]) # → ln(x)A full binary tree of depth n has 2ⁿ leaves and 2ⁿ−1 internal nodes. Each leaf soft-routes
between the constant 1 and the variable x. Each internal node computes eml(left, right),
and its gate 3-way-routes each child input between 1, x, and the child's own output — so
x can flow straight into any gate, not just through leaves.
Training (Adam + tau-annealing) pushes the soft weights toward a single hard choice per gate,
recovering an exact symbolic expression. When the generating law is elementary, the snapped
weights yield machine-epsilon RMSE. The final expression is then simplified by a recursive
AST rewriter that applies EML identities (e.g. eml(1, eml(eml(1, x), 1)) → ln(x)).
| Target | Depth | RMSE |
|---|---|---|
x (identity) |
0 | 0 |
1 (constant) |
0 | 0 |
exp(x) |
1 | 4.6e-16 |
e (constant) |
1 | 0 |
ln(x) |
3 | 7.3e-17 |
exp(x) - ln(x) |
1 | 3.2e-16 |
Depth 0 is a single leaf with no gates — the "passthrough" vocabulary
{1, x}. Without it, the identity y = x is unreachable until depth ≥ 4,
because every gated output is wrapped in an outer eml(·,·). The depth
ladder in discover() now starts at 0 to close that gap (issue #11).
At deep depths, random initialization becomes a needle-in-a-haystack search —
the paper reports 0% recovery at depth 6 from random init. discover_curriculum
grows the tree one leaf at a time as a warm start:
- Start at depth 1 (1 internal node, 2 leaves)
- Train to convergence
- If the fit isn't good enough, pick the leaf with the largest gradient
magnitude (the one most "wanting" to change) and split it — replacing it
with an
eml(x, 1) = exp(x)subtree, and biasing the parent gate to route through the new subtree - Continue training and repeat until
max_depthor exact recovery
from eml_sr import discover_curriculum
result = discover_curriculum(x, y, max_depth=6, n_tries=8)This is analogous to Net2Net / progressive growing in neural architecture
search. It solves nested compositions like exp(exp(exp(x))) several times
faster than random init, and lets shallow solutions seed deeper ones.
Run eml-sr against any two-column CSV from the command line:
python eml_sr.py csv data.csv --x-col time --y-col temperature \
--max-depth 5 --tries 16 --workers 8Options:
--method {discover,curriculum}— fixed-depth ladder vs growing tree--normalize {minmax,standard,none}— affine pre-scaling (EML overflows on un-scaled data;minmaxis the safest default)--workers N— parallel seed processes fordiscover(~linear speedup)
The discovered formula is reported in normalized coordinates along with
the affine transformation needed to convert it back. Use --normalize none
when your data is already in a numerically friendly range — that's the
only setting that yields a directly-readable formula in the original units.
EMLRegressor exposes the standard fit/predict API plus the model_
attribute SRBench (La Cava et al. 2021) reads to extract the symbolic
expression:
from eml_sr_sklearn import EMLRegressor
est = EMLRegressor(max_depth=4, n_tries=16, n_workers=8)
est.fit(X, y)
yhat = est.predict(X_test)
print(est.model_) # symbolic form in normalized coords
print(est.original_model_) # with the affine substitution stitched inWhen X has multiple columns, EMLRegressor projects onto the column with
the highest absolute Spearman correlation with y. This puts eml-sr on the
SRBench leaderboard for univariate / near-univariate problems, while
flagging cleanly that the engine is single-variable by construction.
A curated set of univariate slices from the AI Feynman dataset
(Udrescu & Tegmark 2020) lives in
benchmarks/feynman.py:
python -m benchmarks.feynman --workers 8 # quick (8 problems)
python -m benchmarks.feynman --all --method curriculum # full suiteEach problem reports recovery success, RMSE, depth used, and time to solution.
benchmarks/pysr_compare.py runs eml-sr and PySR
(Cranmer, MIT) on the same target set and writes a comparison table to
benchmarks/pysr_compare.md. It is gated behind PYSR_ENABLED=1 because
PySR needs a Julia toolchain — not something to impose on every CI run.
pip install pysr
python -c "import pysr; pysr.install()" # one-time Julia deps
PYSR_ENABLED=1 python -m benchmarks.pysr_compare # full run
PYSR_ENABLED=1 python -m benchmarks.pysr_compare --quick # small slice
pytest -m pysr # smoke testWithout PYSR_ENABLED, the script still runs and reports eml-sr columns
(PySR columns are marked "not run"). Useful as a plain eml-sr baseline.
The benchmark is a positioning tool, not a speed competition. eml-sr's
grammar uniformity (every node is eml) is its whole point; PySR's
heterogeneous AST over {+, -, *, /, exp, log, sqrt} has different
tradeoffs. The table makes the three axes concrete: grammar uniformity,
exact recovery vs Pareto front, and expression length (EML trees for
standard ops are long by design — multiplication is depth 8 in EML,
depth 1 in a conventional SR AST).
n_workers > 1:discover()accepts ann_workersargument that fans out per-depth seeds across worker processes. Each child process is pinned to a single torch thread to avoid oversubscription. Empirical speedup is near-linear up toos.cpu_count().- Normalization: pre-scaling with
Normalizer.fit(x, y, mode="minmax")prevents theexp(x)chain inside the EML tree from saturating to the 1e300 clamp, which would otherwise kill gradients on real-world data. - Curriculum: for deep formulas,
discover_curriculumwarm-starts by growing the tree leaf-by-leaf, dramatically beating random init past depth 4.
The search direction (data → formula) has a complement: given an elementary
formula, emit the pure EML tree that implements it. eml_compiler.py ports
the JS reference compiler from austegard.com/fun-and-games/eml-calc.html
to Python and exposes it as a first-class module.
from eml_compiler import compile_expr, eval_eml, to_string, tree_size, tree_depth
tree = compile_expr("ln(x) + exp(y)")
print(to_string(tree)) # eml(...) tree in eml_sr.to_expr format
print(tree_size(tree), tree_depth(tree))
eval_eml(tree, x=2.0, y=1.5).real # ≈ 5.1746...Or from the CLI:
$ python -m eml_compiler "ln(x) + exp(y)" --eval x=2 y=1.5
eml(eml(1, eml(eml(1, eml(1, eml(eml(1, x), 1))), 1)), eml(y, 1))
size: 29
depth: 7
value: (5.17463...+0j)
The paper's pure grammar is S → 1 | x | eml(S, S). The JS compiler and this
port both default to the pragmatic relaxation S → c | x | eml(S, S) where
arbitrary numeric constants and π ride as leaves rather than being
bootstrapped from {1} (e.g. π = −i·ln(−1) produces an enormous tree).
compile_expr("2*x", strict=True) fails with an explanatory error. Strict
mode also reroutes negation through eml_ln(1) = 0 so the compiled tree's
leaves are all either 1 or a named variable — paper-faithful.
- Ground-truth trees for the test suite and benchmarks: given a Feynman
target formula,
compile_expr(formula)yields the canonical EML tree whose size / depth become the recovery target. - Warm-start seeds for
discover_curriculum: compile a human guess, perturb weights around it, let gradient descent refine. - Round-trip with
EMLTree1D.to_expr(): a snapped tree's string form parses back into an equivalent EML tree (verified in the test suite).
Trig (sin, cos, tan) is out of scope per §4.1 — their EML trees are
enormous. Use exp/ln with complex arguments.
eml_sr_distill.py implements the KAN-style post-hoc extraction pipeline
proposed in issue #42. Train a small MLP, sample the per-feature univariate
partial-dependence curves, run discover_curriculum on each, and compose
the results additively. Surrogate quality is reported as R² against the
MLP's output; compression is the ratio of MLP parameters to EML tree
nodes.
from eml_sr_distill import train_mlp, distill
model = train_mlp(X, y, hidden_sizes=(8,), activation="tanh")
report = distill(model, X, y, method="curriculum", max_depth=5, n_tries=6)
print(report.summary())See benchmarks/distill_demo.py for a three-target demo (linear additive,
exp+log, and a non-additive product case showing the limits of additive
composition). Per-edge pre-activation sampling (include_edges=True) is
also provided for the first hidden layer.
Scope: 1–2 hidden layers; x², tanh, identity, relu activations.
Deeper networks and attention are documented out-of-scope in the issue.
| File | Description |
|---|---|
eml_sr.py |
Symbolic regression engine + CLI (the product) |
eml_sr_sklearn.py |
sklearn-style EMLRegressor for SRBench |
eml_sr_distill.py |
Per-edge distillation of trained MLPs (issue #42) |
eml_compiler.py |
Forward direction: elementary expression → EML tree |
benchmarks/feynman.py |
Univariate Feynman-equation benchmark |
benchmarks/pysr_compare.py |
Head-to-head vs PySR (opt-in via PYSR_ENABLED=1) |
benchmarks/distill_demo.py |
Demo of per-edge MLP distillation (issue #42) |
legacy/eml_executor.mojo |
Original parabolic-attention stack machine (archived) |
legacy/test_eml.mojo |
109-test bootstrap chain verification (archived) |
ternary/ |
§5 ternary-operator experiment (T(x,y,z)). See ternary/report.md for the VerifyBaseSet verdict. |
- Paper: arXiv:2603.21852
- Blog: Two buttons and a constant
- Blog: Two buttons, back row
- Demo: EML Calculator