“Know how to solve every problem that has been solved.” “What I cannot create, I do not understand.” — Richard Feynman

Stress testing: catch the WA before the judge does

Learn

Lesson 11 of 13 standard ~7 min

You have a clever solution, a Wrong Answer verdict, and no idea which of 10⁵ hidden cases breaks it. Re-reading the code rarely finds the bug — your eyes skip the same blind spot every pass. Stress testing finds it mechanically, while you watch: pit your clever solution against a dumb one you trust, on thousands of random inputs, until they disagree.

It takes three small pieces. A brute force that is obviously correct and too slow to ever submit — the slow, dead-simple way you would never get wrong. A generator that prints a random input. And a driver that loops: generate, run both, diff the outputs, stop the instant they differ.

for i in $(seq 1 10000); do
    ./gen  $i        > in.txt      # random input, seeded by i
    ./brute < in.txt > out_brute.txt
    ./fast  < in.txt > out_fast.txt
    if ! diff -q out_brute.txt out_fast.txt > /dev/null; then
        echo "MISMATCH on test $i:"; cat in.txt
        break
    fi
done
The whole harness — a few lines of shell around three programs.

The first mismatch is a minimal failing case — and minimality is why the generator must produce small inputs. A bug that surfaces on an array of length 3 you can debug by hand in a minute; the identical bug on length 10⁵ is a wall of numbers. Bound the generator small (say n ≤ 8, values ≤ 5) and bias it toward the cases that break things: empty, single element, all-equal, the extremes.

standardMultiple choice

Your clever solution gets Wrong Answer on a hidden test, somewhere among 10⁵ cases you cannot see. What is the most reliable way to find the failing case?

The mindset underneath: trust the brute, distrust the clever. The brute force is your ground truth precisely because it is too simple to be wrong. Once the harness hands you a three-element counterexample, the bug that hid in 10⁵ cases is usually obvious in thirty seconds.

standardMultiple choice

In a stress-testing harness, why should the random generator be bounded to produce SMALL inputs (e.g. n ≤ 8) rather than full-size ones?