Mean-Variance Portfolio Optimization

Finance

Mean-variance portfolio optimization, due to Harry Markowitz (1952), is the canonical applied portfolio-construction technique and the foundation of modern portfolio theory. The idea: given a set of investable assets with expected returns and a covariance matrix of those returns, find the portfolio weights that DELIVER A TARGET EXPECTED RETURN AT MINIMUM VARIANCE — or equivalently, MAXIMIZE EXPECTED RETURN AT A GIVEN VARIANCE. The locus of all such optima as the target sweeps is the EFFICIENT FRONTIER. Every textbook on portfolio management starts here; every modern allocation method (Black-Litterman, risk parity, factor models) either extends it or pushes back against it.

The conceptual move is to treat a portfolio's RETURN-RISK profile as a two-dimensional point — expected return on one axis, standard deviation (or variance) on the other — and then optimize within that plane. Diversification appears automatically: as long as asset returns aren't perfectly correlated, mixing them reduces variance below the variance of any single asset, sometimes dramatically. Markowitz's insight was to formalize this as a quadratic program whose solution is closed-form.

The mathematical statement

Given expected return vector $μ \in R^{n}$ and covariance matrix $Σ \in R^{n \times n}$ (symmetric positive definite), choose weights $w \in R^{n}$ with $1^{T} w = 1$ (fully invested) to solve

w min \frac{1}{2} w^{T} Σ w subject to w^{T} μ = μ_{p}, 1^{T} w = 1.

Form the Lagrangian

L = \frac{1}{2} w^{T} Σ w - λ (w^{T} μ - μ_{p}) - γ (1^{T} w - 1) .

Setting $\partial L / \partial w = 0$ gives $w^{*} = λ Σ^{- 1} μ + γ Σ^{- 1} 1$ . Applying the two constraints yields two linear equations for $(λ, γ)$ , which are solved explicitly. Defining

A = 1^{T} Σ^{- 1} 1, B = 1^{T} Σ^{- 1} μ, C = μ^{T} Σ^{- 1} μ, D = A C - B^{2},

one gets

w^{*} = \frac{( C - B μ _{p} ) Σ ^{- 1} 1 + ( A μ _{p} - B ) Σ ^{- 1} μ}{D} .

The variance of this optimal portfolio is $σ_{p}^{2} = (A μ_{p}^{2} - 2 B μ_{p} + C) / D$ — a parabola in $μ_{p}$ opening upward. Plotting $μ_{p}$ against $σ_{p} = σ_{p}^{2}$ gives the EFFICIENT FRONTIER: a hyperbola in mean-std space. The minimum-variance portfolio sits at the vertex ( $μ_{p} = B / A$ ); the upper branch (higher return per unit risk than the lower branch) is the efficient frontier proper.

The tangency portfolio

If a RISK-FREE ASSET is available with return $r_{f}$ , the picture simplifies: investors can hold any combination of the risk-free asset and ONE specific risky portfolio (the TANGENCY PORTFOLIO), and the set of attainable portfolios is the CAPITAL ALLOCATION LINE — a straight line tangent to the efficient frontier. The tangency portfolio maximizes the SHARPE RATIO

Sharpe = \frac{w ^{T} μ - r _{f}}{w ^{T} Σ w} .

Closed form:

w_{T} \propto Σ^{- 1} (μ - r_{f} 1),

normalized so the weights sum to 1. The Sharpe ratio at the tangency portfolio is the maximum achievable Sharpe with the given assets: $(μ - r_{f} 1)^{T} Σ^{- 1} (μ - r_{f} 1)$ under the square root. The Capital Asset Pricing Model (CAPM) builds on this: in equilibrium, the tangency portfolio EQUALS the market portfolio, and expected excess returns of individual assets are proportional to their covariance with the market (beta).

Code

# Markowitz mean-variance optimization on five fictional assets.
# Find: (a) minimum-variance portfolio, (b) efficient frontier,
# (c) tangency portfolio (max Sharpe at risk-free rf).

import numpy as np

# Expected annual returns and standard deviations
mu        = np.array([0.08, 0.10, 0.12, 0.07, 0.09])
sigma_vec = np.array([0.15, 0.20, 0.25, 0.10, 0.18])
corr = np.array([
    [1.0, 0.3, 0.2, 0.4, 0.5],
    [0.3, 1.0, 0.6, 0.2, 0.4],
    [0.2, 0.6, 1.0, 0.1, 0.3],
    [0.4, 0.2, 0.1, 1.0, 0.3],
    [0.5, 0.4, 0.3, 0.3, 1.0],
])
Sigma = np.outer(sigma_vec, sigma_vec) * corr

# (a) Minimum-variance portfolio: w* = Σ⁻¹·1 / (1ᵀ Σ⁻¹·1)
def min_var_portfolio(Sigma):
    n = Sigma.shape[0]
    ones = np.ones(n)
    inv_S = np.linalg.solve(Sigma, ones)
    return inv_S / (ones @ inv_S)

w_min = min_var_portfolio(Sigma)
print("Minimum-variance portfolio:")
print(f"  Weights = {np.round(w_min, 4)}  (sum = {w_min.sum():.4f})")
print(f"  Return  = {w_min @ mu:.4f}, Std = {np.sqrt(w_min @ Sigma @ w_min):.4f}")

# (b) Efficient frontier: minimum variance subject to target return.
# Lagrangian gives a closed form:
#   w = lam * Σ⁻¹·1 + gam * Σ⁻¹·μ
# with lam, gam chosen so that wᵀ1 = 1 and wᵀμ = target.
def efficient_portfolio(mu, Sigma, target):
    n = len(mu)
    ones = np.ones(n)
    Sinv_one = np.linalg.solve(Sigma, ones)
    Sinv_mu  = np.linalg.solve(Sigma, mu)
    A = ones @ Sinv_one
    B = ones @ Sinv_mu
    C = mu @ Sinv_mu
    det = A*C - B*B
    lam = (C - B*target) / det
    gam = (A*target - B) / det
    return lam * Sinv_one + gam * Sinv_mu

print("\nEfficient frontier:")
print(f"  {'target':>8s}  {'std':>8s}  {'leverage':>10s}")
for t in [0.07, 0.08, 0.09, 0.10, 0.11, 0.12]:
    w = efficient_portfolio(mu, Sigma, t)
    print(f"  {t:>8.4f}  {np.sqrt(w @ Sigma @ w):>8.4f}  {np.abs(w).sum():>10.4f}")

# (c) Tangency portfolio (max Sharpe ratio with risk-free rate rf).
def tangency_portfolio(mu, Sigma, rf):
    excess = mu - rf
    Sinv_ex = np.linalg.solve(Sigma, excess)
    return Sinv_ex / np.sum(Sinv_ex)

w_t = tangency_portfolio(mu, Sigma, rf=0.02)
sharpe = (w_t @ mu - 0.02) / np.sqrt(w_t @ Sigma @ w_t)
print("\nTangency portfolio (rf = 2%):")
print(f"  Weights = {np.round(w_t, 4)}")
print(f"  Return  = {w_t @ mu:.4f}, Std = {np.sqrt(w_t @ Sigma @ w_t):.4f}")
print(f"  Sharpe  = {sharpe:.4f}")

Output:

Minimum-variance portfolio:
  Weights = [0.1188 0.0555 0.0572 0.723  0.0455]  (sum = 1.0000)
  Return  = 0.0766, Std = 0.0925

Efficient frontier:
    target       std    leverage
    0.0700    0.0982      1.5046
    0.0800    0.0940      1.0810
    0.0900    0.1142      1.3128
    0.1000    0.1491      1.7942
    0.1100    0.1909      2.4276
    0.1200    0.2359      3.0823

Tangency portfolio (rf = 2%):
  Weights = [0.1194 0.0747 0.1501 0.5699 0.0859]
  Return  = 0.0904, Std = 0.1093
  Sharpe  = 0.6441

Three things to read off. (1) The minimum-variance portfolio puts 72% in asset D — the lowest-volatility asset (10% std) — and small weights in the others, achieving overall std 9.25%, BELOW the lowest single-asset std. That's diversification reducing risk below any constituent. (2) Walking up the efficient frontier (target return from 7% to 12%), the standard deviation grows superlinearly, and the LEVERAGE (sum of absolute weights) grows from 1.08 to 3.08 — to chase higher return, the optimizer takes increasingly extreme long/short positions in the highest-return assets. (3) The tangency portfolio (Sharpe 0.64) sits between the min-variance and high-return targets, balancing the two via the risk-free rate.

The leverage problem

The closed-form Markowitz solver allows weights to be ANY real numbers — positive (long), negative (short), or unboundedly large. As we saw above, the unconstrained efficient frontier produces leverage that grows without bound as the target return increases. In real applications this is unacceptable; the constraints that matter are LONG-ONLY ( $w_{i} \geq 0$ ), POSITION LIMITS ( $∣ w_{i} ∣ \leq c$ ), and TURNOVER LIMITS ( $∥ w - w_{prev} ∥_{1} \leq T$ ). With constraints, the problem becomes a quadratic program with linear inequality constraints — no closed form, but well within reach of any QP solver (cvxpy, OSQP, MOSEK).

The estimation problem

Markowitz works perfectly when $μ$ and $Σ$ are KNOWN. They never are. In practice they must be estimated from historical returns, and the estimates are very noisy — especially $μ$ , where you typically need DECADES of data to get a few percent precision. The Markowitz solution is HIGHLY SENSITIVE to $μ$ : tiny changes in expected returns produce wildly different optimal portfolios. This is the famous CRITIQUE of mean-variance:

Sample-mean returns are noisy. Annualised return estimated from 5 years of monthly data has standard error $σ / T \approx σ /2$ — so the noise in $\overset{μ}{^}$ can be comparable to its size.
The covariance matrix is also noisy. With $N$ assets and $T$ time observations, $\hat{Σ}$ has rank $min (N, T - 1)$ ; when $N \sim T$ , the smallest eigenvalues are essentially random.
Markowitz is a small-eigenvalue amplifier: the optimizer puts large weights along the directions of smallest $Σ$ -eigenvalues, which are exactly the ones most polluted by estimation noise. Garbage in, garbage out.

Standard cures: SHRINKAGE estimators for $Σ$ (Ledoit-Wolf 2004), pulling toward a structured target like a constant correlation matrix; FACTOR MODELS (replace the sample covariance with one built from a few common factors); BLACK-LITTERMAN (start from an equilibrium portfolio and only deviate when you have specific views); ROBUST OPTIMIZATION (find the portfolio that's good under WORST-CASE perturbations of $μ$ and $Σ$ ); and RISK PARITY (don't try to use $μ$ at all; equalize risk contributions across assets).

What this enables in practice

Equity portfolio construction: large-cap equity strategies (institutional, pension funds, mutual funds) start from a factor-model-based $Σ$ and apply mean-variance optimization with constraints to tilt toward desired exposures while limiting tracking error.
Multi-asset allocation: the classic 60/40 stock/bond portfolio is, structurally, the output of a tangency-portfolio calculation with a specific (historical) $(μ, Σ)$ .
Risk budgeting: assign each asset a fixed contribution to total portfolio variance (risk parity), which is a different optimization but related to Markowitz inversion.
Index construction: optimization-based indices (low-volatility, minimum-variance, equal-risk-contribution) compete with market-cap-weighted indices.

Value at Risk — the risk-management complement to mean-variance.
Black-Scholes model — option pricing assumes log-normal returns, similar Gaussian structure.
Condition number — the small-eigenvalue amplification problem is exactly the ill-conditioned-matrix story applied to portfolio optimization.
Singular value decomposition — used for robust covariance estimation and factor analysis.