Skip to content
oaustegardPublic

About

A compiled transformer executor whose only compute instruction is eml(x,y) = exp(x) − ln(y). Parabolic attention for memory, complex arithmetic over ℂ, 3-opcode ISA. Built in Mojo.

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

eml-sr

EML Symbolic Regression — discover elementary formulas from data using a single operator.

Based on Odrzywolek (2026), "All elementary functions from a single operator": the EML operator eml(x, y) = exp(x) - ln(y) (Exponent Minus Log), together with the constant 1, generates all standard elementary functions. This makes S → 1 | x | eml(S, S) a complete, regular search space for symbolic regression.

Quick start

import numpy as np
from eml_sr import discover

x = np.linspace(0.5, 5.0, 50)
y = np.log(x)  # secret formula

result = discover(x, y, max_depth=4, n_tries=8)
print(result["expr"])  # → ln(x)

How it works

A full binary tree of depth n has 2ⁿ leaves and 2ⁿ−1 internal nodes. Each leaf soft-routes between the constant 1 and the variable x. Each internal node computes eml(left, right), and its gate 3-way-routes each child input between 1, x, and the child's own output — so x can flow straight into any gate, not just through leaves.

Training (Adam + tau-annealing) pushes the soft weights toward a single hard choice per gate, recovering an exact symbolic expression. When the generating law is elementary, the snapped weights yield machine-epsilon RMSE. The final expression is then simplified by a recursive AST rewriter that applies EML identities (e.g. eml(1, eml(eml(1, x), 1)) → ln(x)).

Current recovery rates

Target Depth RMSE
x (identity) 0 0
1 (constant) 0 0
exp(x) 1 4.6e-16
e (constant) 1 0
ln(x) 3 7.3e-17
exp(x) - ln(x) 1 3.2e-16

Depth 0 is a single leaf with no gates — the "passthrough" vocabulary {1, x}. Without it, the identity y = x is unreachable until depth ≥ 4, because every gated output is wrapped in an outer eml(·,·). The depth ladder in discover() now starts at 0 to close that gap (issue #11).

Curriculum learning (growing trees)

At deep depths, random initialization becomes a needle-in-a-haystack search — the paper reports 0% recovery at depth 6 from random init. discover_curriculum grows the tree one leaf at a time as a warm start:

  1. Start at depth 1 (1 internal node, 2 leaves)
  2. Train to convergence
  3. If the fit isn't good enough, pick the leaf with the largest gradient magnitude (the one most "wanting" to change) and split it — replacing it with an eml(x, 1) = exp(x) subtree, and biasing the parent gate to route through the new subtree
  4. Continue training and repeat until max_depth or exact recovery
from eml_sr import discover_curriculum
result = discover_curriculum(x, y, max_depth=6, n_tries=8)

This is analogous to Net2Net / progressive growing in neural architecture search. It solves nested compositions like exp(exp(exp(x))) several times faster than random init, and lets shallow solutions seed deeper ones.

CSV ingestion

Run eml-sr against any two-column CSV from the command line:

python eml_sr.py csv data.csv --x-col time --y-col temperature \
    --max-depth 5 --tries 16 --workers 8

Options:

  • --method {discover,curriculum} — fixed-depth ladder vs growing tree
  • --normalize {minmax,standard,none} — affine pre-scaling (EML overflows on un-scaled data; minmax is the safest default)
  • --workers N — parallel seed processes for discover (~linear speedup)

The discovered formula is reported in normalized coordinates along with the affine transformation needed to convert it back. Use --normalize none when your data is already in a numerically friendly range — that's the only setting that yields a directly-readable formula in the original units.

sklearn interface (for SRBench)

EMLRegressor exposes the standard fit/predict API plus the model_ attribute SRBench (La Cava et al. 2021) reads to extract the symbolic expression:

from eml_sr_sklearn import EMLRegressor

est = EMLRegressor(max_depth=4, n_tries=16, n_workers=8)
est.fit(X, y)
yhat = est.predict(X_test)
print(est.model_)            # symbolic form in normalized coords
print(est.original_model_)   # with the affine substitution stitched in

When X has multiple columns, EMLRegressor projects onto the column with the highest absolute Spearman correlation with y. This puts eml-sr on the SRBench leaderboard for univariate / near-univariate problems, while flagging cleanly that the engine is single-variable by construction.

Feynman benchmark

A curated set of univariate slices from the AI Feynman dataset (Udrescu & Tegmark 2020) lives in benchmarks/feynman.py:

python -m benchmarks.feynman --workers 8                # quick (8 problems)
python -m benchmarks.feynman --all --method curriculum  # full suite

Each problem reports recovery success, RMSE, depth used, and time to solution.

Head-to-head vs PySR

benchmarks/pysr_compare.py runs eml-sr and PySR (Cranmer, MIT) on the same target set and writes a comparison table to benchmarks/pysr_compare.md. It is gated behind PYSR_ENABLED=1 because PySR needs a Julia toolchain — not something to impose on every CI run.

pip install pysr
python -c "import pysr; pysr.install()"          # one-time Julia deps
PYSR_ENABLED=1 python -m benchmarks.pysr_compare # full run
PYSR_ENABLED=1 python -m benchmarks.pysr_compare --quick  # small slice
pytest -m pysr                                   # smoke test

Without PYSR_ENABLED, the script still runs and reports eml-sr columns (PySR columns are marked "not run"). Useful as a plain eml-sr baseline.

The benchmark is a positioning tool, not a speed competition. eml-sr's grammar uniformity (every node is eml) is its whole point; PySR's heterogeneous AST over {+, -, *, /, exp, log, sqrt} has different tradeoffs. The table makes the three axes concrete: grammar uniformity, exact recovery vs Pareto front, and expression length (EML trees for standard ops are long by design — multiplication is depth 8 in EML, depth 1 in a conventional SR AST).

Performance

  • n_workers > 1: discover() accepts an n_workers argument that fans out per-depth seeds across worker processes. Each child process is pinned to a single torch thread to avoid oversubscription. Empirical speedup is near-linear up to os.cpu_count().
  • Normalization: pre-scaling with Normalizer.fit(x, y, mode="minmax") prevents the exp(x) chain inside the EML tree from saturating to the 1e300 clamp, which would otherwise kill gradients on real-world data.
  • Curriculum: for deep formulas, discover_curriculum warm-starts by growing the tree leaf-by-leaf, dramatically beating random init past depth 4.

Compiler: elementary expressions → EML trees

The search direction (data → formula) has a complement: given an elementary formula, emit the pure EML tree that implements it. eml_compiler.py ports the JS reference compiler from austegard.com/fun-and-games/eml-calc.html to Python and exposes it as a first-class module.

from eml_compiler import compile_expr, eval_eml, to_string, tree_size, tree_depth

tree = compile_expr("ln(x) + exp(y)")
print(to_string(tree))             # eml(...) tree in eml_sr.to_expr format
print(tree_size(tree), tree_depth(tree))
eval_eml(tree, x=2.0, y=1.5).real  # ≈ 5.1746...

Or from the CLI:

$ python -m eml_compiler "ln(x) + exp(y)" --eval x=2 y=1.5
eml(eml(1, eml(eml(1, eml(1, eml(eml(1, x), 1))), 1)), eml(y, 1))
size:  29
depth: 7
value: (5.17463...+0j)

Grammar relaxation

The paper's pure grammar is S → 1 | x | eml(S, S). The JS compiler and this port both default to the pragmatic relaxation S → c | x | eml(S, S) where arbitrary numeric constants and π ride as leaves rather than being bootstrapped from {1} (e.g. π = −i·ln(−1) produces an enormous tree).

compile_expr("2*x", strict=True) fails with an explanatory error. Strict mode also reroutes negation through eml_ln(1) = 0 so the compiled tree's leaves are all either 1 or a named variable — paper-faithful.

What it's for

  • Ground-truth trees for the test suite and benchmarks: given a Feynman target formula, compile_expr(formula) yields the canonical EML tree whose size / depth become the recovery target.
  • Warm-start seeds for discover_curriculum: compile a human guess, perturb weights around it, let gradient descent refine.
  • Round-trip with EMLTree1D.to_expr(): a snapped tree's string form parses back into an equivalent EML tree (verified in the test suite).

Trig (sin, cos, tan) is out of scope per §4.1 — their EML trees are enormous. Use exp/ln with complex arguments.

Distillation: MLP → EML expressions

eml_sr_distill.py implements the KAN-style post-hoc extraction pipeline proposed in issue #42. Train a small MLP, sample the per-feature univariate partial-dependence curves, run discover_curriculum on each, and compose the results additively. Surrogate quality is reported as R² against the MLP's output; compression is the ratio of MLP parameters to EML tree nodes.

from eml_sr_distill import train_mlp, distill

model = train_mlp(X, y, hidden_sizes=(8,), activation="tanh")
report = distill(model, X, y, method="curriculum", max_depth=5, n_tries=6)
print(report.summary())

See benchmarks/distill_demo.py for a three-target demo (linear additive, exp+log, and a non-additive product case showing the limits of additive composition). Per-edge pre-activation sampling (include_edges=True) is also provided for the first hidden layer.

Scope: 1–2 hidden layers; x², tanh, identity, relu activations. Deeper networks and attention are documented out-of-scope in the issue.

Files

File Description
eml_sr.py Symbolic regression engine + CLI (the product)
eml_sr_sklearn.py sklearn-style EMLRegressor for SRBench
eml_sr_distill.py Per-edge distillation of trained MLPs (issue #42)
eml_compiler.py Forward direction: elementary expression → EML tree
benchmarks/feynman.py Univariate Feynman-equation benchmark
benchmarks/pysr_compare.py Head-to-head vs PySR (opt-in via PYSR_ENABLED=1)
benchmarks/distill_demo.py Demo of per-edge MLP distillation (issue #42)
legacy/eml_executor.mojo Original parabolic-attention stack machine (archived)
legacy/test_eml.mojo 109-test bootstrap chain verification (archived)
ternary/ §5 ternary-operator experiment (T(x,y,z)). See ternary/report.md for the VerifyBaseSet verdict.

References

About

A compiled transformer executor whose only compute instruction is eml(x,y) = exp(x) − ln(y). Parabolic attention for memory, complex arithmetic over ℂ, 3-opcode ISA. Built in Mojo.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages