Project: A Neural Net That Finds the Symmetry

The brute-force engine dies on a two-variable mystery whose formula costs eight tokens: its budget stops at six, and after two million candidates the best it can offer is 20% wrong. This page is the AI Feynman move that rescues it — train a neural network on the same samples, then interrogate the network for a symmetry the data never states out loud. The network is not the answer; it is a witness. Everything below — the failed search, the MLP and its backprop, the probes, the winning re-search — is one Knot program compiled to 49 KB of WebAssembly, running on your machine.

Why a neural network, when it explains nothing?

The dataset is 225 fixed points. You cannot ask a dataset "what happens if I nudge both inputs by the same amount?" — the nudged point isn't in it. A network trained to rms 0.003 on those points can be asked: it interpolates, so counterfactual queries come for free. That is the entire role of the MLP here — a differentiable stand-in for $f$ that turns a frozen table into something you can poke.

# probe kinds: 1 shift both (+a,+a) · 2 shift x only · 3 scale both · 4 shift opposite
def probe(kind, n) {
    a = 0.35
    for t to n {
        x = (rand01() - 0.5) * 1.6          # points the dataset never contained
        y = (rand01() - 0.5) * 1.6
        g0 = nn_forward(x, y)
        g1 = 0.0
        if kind == 1.0 { g1 = nn_forward(x + a, y + a) }
        if kind == 2.0 { g1 = nn_forward(x + a, y) }
        if kind == 3.0 { g1 = nn_forward(1.3 * x, 1.3 * y) }
        if kind == 4.0 { g1 = nn_forward(x + a, y - a) }
        pv[t] = abs(g1 - g0)
    }
    return median_of(n)
}

Four pokes, each a hypothesis about structure. If $f(x+a,y+a)=f(x,y)$, then $f$ can only depend on the difference $x-y$; if scaling both inputs changes nothing, it lives on the ratio; if the opposite shift is invisible, it lives on the sum. The trained network answers with a 300× gap between "shift both" and everything else. One substitution $u=x-y$ later, the mystery is a one-variable problem, the eight-token formula costs five, and the same enumeration budget that just failed now lands on $\exp (-u^2)$ with zero error.

The accounting is the lesson. The brute-force engine never improved — same alphabet, same budget, same scoring. What changed is the problem: each discovered symmetry deletes a variable, and deleting a variable shrinks the search space exponentially. That is the entire architecture of AI Feynman: dimensional analysis, separability, compositionality — every module is a differently-shaped probe whose only job is to hand the dumb enumerator a smaller problem.

Everything runs inside the one Knot source (tab ① above): the RPN enumerator shared with the symbolic-regression page, a 2→12→12→1 MLP with backprop written out by hand — the same flat-array style as the reverse-mode autodiff engine — a Park–Miller generator for the weights (chosen because 16807·s stays under 2⁵³, so the modular arithmetic is exact in a language whose only number is a double), and an insertion sort for the probe medians.

Reproduce it

scripts/build_nns_wasm.sh rebuilds the pipeline. The native binary runs all four acts in 0.55 s. Together with the enumeration engine and the Meijer-G metamodeler this completes the trilogy: search, relax, and shrink — the three ways a machine gets a formula out of data.