“Know how to solve every problem that has been solved.” “What I cannot create, I do not understand.” — Richard Feynman

Picard Iteration

Differential Equations

The Picard iteration is an iterative procedure for solving differential equations. It's mostly used in proofs of existence and uniqueness for ODEs rather than as a numerical method — it's not particularly fast, and the integrals get unwieldy in a hurry. But the construction itself is striking on simple examples, and worth seeing once.

Introduction

Given a differential equation of the following form

\frac{d y}{d x} = f (x, y (x)) y (x_{0}) = y_{0}

We may write this in integral form

y (x) - y (x_{0}) = \int_{x_{0}}^{x} f (t, y (t)) d t

Rearranging the formula:

y (x) = y (x_{0}) + \int_{x_{0}}^{x} f (t, y (t)) d t

We can conveniently change this into a fixed point iteration.

y_{n + 1} = y_{0} + \int_{x_{0}}^{x} f (t, y_{n} (t)) d t

An Application

Let's apply this to the simplest nontrivial linear ODE:

\frac{d y}{d x} = y, y (0) = 1.

The same example shows up on the Adomian Decomposition page, but I don't mind repeating it here. Starting from $y_{0} = 1$ and feeding each iterate back through the integral:

y_{0} = 1

y_{1} = 1 + \int_{0}^{x} d x = 1 + x

y_{2} = 1 + \int_{0}^{x} (1 + x) d x = 1 + x + \frac{x ^{2}}{2 !}

y_{3} = 1 + \int_{0}^{x} (1 + x + \frac{x ^{2}}{2}) d x = 1 + x + \frac{x ^{2}}{2 !} + \frac{x ^{3}}{3 !}

That's the Taylor series for $e^{x}$ . The Picard iteration is literally constructing the exponential function one term at a time out of polynomial integrations — which, on reflection, is a rather nice thing to watch happen. In the limit:

y_{n} = k = 0 \sum n \frac{x ^{k}}{k !} ⟶ e^{x} .

That's clean for a trivial linear ODE. Make the right-hand side a little more complicated and the method starts to fight you: the iterates take many more steps to match what a first-order Euler scheme gives you for free, and for some perfectly innocent-looking linear problems the integrals stop being expressible in elementary functions, at which point you're back to numerical integration anyway.

Symbolic

A computer algebra system handles the iteration symbolically and gives you exact polynomial answers as long as the right-hand side is well-behaved. Nonlinearities are where the wheels come off — many of the resulting integrals stop being expressible in elementary functions. You can Taylor expand the nonlinearity and keep going, but the radius of convergence may be finite, and what you get back is a local series rather than a global solution.