Backtesting of VaR Models

1Learning roadmap

A bank forecasts a one-day 99% Value at Risk of ₹10 million. Tomorrow it loses ₹12 million. Has the model failed? One loss cannot answer that question. A backtest examines a sequence of forecasts and realised outcomes to ask whether exceptions occur at a reasonable frequency and whether they cluster.

By the end, you will be able to build an exception series, distinguish exact binomial and likelihood-ratio tests, interpret coverage and independence, and implement a clean out-of-sample workflow in Excel. The examples are teaching illustrations, not estimates from a live portfolio.

2VaR, losses and exceptions

VaR is a quantile of the loss distribution at a stated horizon and confidence level. It is not the maximum possible loss and does not describe the average loss beyond the threshold.

Here VaRₜ is the forecast made before day t, using information available through t−1. Losses are positive and profits are negative losses. Equality is not an exception under this strict convention; document your treatment of ties.

3Design a genuine out-of-sample backtest

1. Fix the specification: portfolio, currency, horizon, confidence level, model and estimation window.
2. Estimate using past information: for a 250-day rolling window, use observations t−250 through t−1 to forecast day t.
3. Archive the forecast: retain parameters, data version and production timestamp.
4. Observe a comparable P&L: match the portfolio definition, valuation time and horizon.
5. Move forward one day: repeat without incorporating the day being forecast.
6. Evaluate a separate test period: investigate exceptions and document model changes.

Actual P&L reflects realised trading activity and other included income or costs. Hypothetical P&L revalues a fixed prior-day portfolio and helps isolate market-risk forecasting. Definitions matter; neither series should be silently substituted for the other.

4Expected exceptions and the binomial model

Under independent, correctly calibrated hits with constant probability p, X follows a Binomial(T,p) distribution. For 250 days at 99% confidence, the expected count is 2.5 and its standard deviation is about 1.573. A realised count must be an integer; the expectation need not be.

5Exact binomial testing

For the question “Are there too many exceptions?”, use an upper-tail test: H₀ has exception probability p; H₁ has a larger probability. The exact p-value includes the observed count and every more extreme count.

In Excel, use =IF(x=0,1,1-BINOM.DIST(x-1,T,p,TRUE)). By contrast, BINOM.DIST(x,T,p,FALSE) gives the probability of exactly x, not a tail p-value. For two-sided calibration, specify a two-sided procedure: an exact probability-ordering test is not generally obtained by blindly doubling the upper tail.

Reject when the selected p-value is below the pre-specified significance level. Otherwise, fail to reject; the evidence may be too weak to detect a defective model.

6Kupiec: unconditional coverage

The Kupiec proportion-of-failures test compares the likelihood under the target exception probability with the likelihood at the observed rate p̂ = x/T. It can detect both too many and too few exceptions, but ignores their order.

Under standard large-sample conditions, compare LRuc with χ² with one degree of freedom. The 5% critical value is 3.841. Excel’s CHISQ.DIST.RT(LRuc,1) gives the approximate p-value. Evaluate log-likelihoods directly to avoid multiplying tiny probabilities. Define zero-count terms as zero: 0 ln(0) contributes 0 by continuity.

7Worked examples: why conclusions can differ

Six exceptions in 250 days: the observed rate is 2.4%, above the 1% target. The exact upper-tail p-value is about 0.0412, so an upper-tail test rejects at 5%. Kupiec’s approximate p-value is about 0.0594, so that procedure does not reject at 5%. Report both faithfully: their alternatives and finite-sample behaviour differ.

Eight exceptions: both tests provide stronger evidence of inadequate coverage. Ten exceptions at 97.5% confidence: the target count is 6.25, not 2.5; confidence changes the reference distribution. 252 days: expected exceptions are 2.52, and the 250-day traffic-light table should not be copied over unchanged.

8Timing matters: independence and clustering

Two models may record six exceptions over the same period. One spreads them across the year; another records all six in one week. Coverage counts alone cannot distinguish them. Clustering may indicate delayed volatility adjustment, changing correlations or an omitted risk factor.

For T daily indicators there are T−1 adjacent transitions. Define π₀₁ = n₀₁/(n₀₀+n₀₁), π₁₁ = n₁₁/(n₁₀+n₁₁), and π = (n₀₁+n₁₁)/(T−1). A much larger π₁₁ than π₀₁ is a clustering signal.

The usual reference distribution is χ²(1). With no exceptions or missing transition states, conditional rates may be undefined and the approximation unreliable. Do not interpret a convenient zero statistic as proof of independence. This first-order test also does not detect every form of serial dependence.

9Conditional coverage and model comparison

Conditional coverage combines frequency and independence; the 5% χ²(2) cutoff is about 5.991. Use a consistent likelihood convention, including treatment of the first observation, when combining results. Christoffersen’s interval-forecast framework explains why unconditional coverage alone is insufficient.

Compare models on the same dates, portfolio and P&L definition. An excessively large VaR may suppress breaches while being economically unhelpful. Quantile loss balances calibration and sharpness:

Here q is the VaR forecast at confidence c. Lower average quantile loss is better on comparable data, but formal forecast comparisons should account for dependence. Also examine breach magnitude, time between exceptions and stress-period performance.

10Basel traffic lights: scope and interpretation

The familiar 99%, one-day, 250-observation traffic-light count classifies results as follows. It is a supervisory framework, not a universal statistical acceptance test.

Six exceptions are yellow even though different formal tests can disagree at 5%. Green does not certify independence or eliminate model risk. The table is not applicable unchanged at 95%, 97.5%, or a different sample length.

The Basel market-risk backtesting framework distinguishes bank-wide and trading-desk requirements and actual versus hypothetical P&L. Expected Shortfall capital measurement does not remove the role of VaR backtesting. Check the applicable local implementation before making a regulatory capital calculation; this guide does not calculate a capital multiplier.

11Interactive exception-count laboratory

Change the sample size, VaR confidence and observed exceptions. The calculator reports the exact upper-tail binomial test and approximate Kupiec test separately. It cannot assess independence without dated exceptions.

12Excel implementation: a reproducible blueprint

Use one row per test day. Column A is the date, B is realised P&L, C is positive-loss VaR forecast before that day, D is loss, E is the hit and F is the cumulative count. For 250 observations use rows 2–251.

Copy D2:F2 down. Change the two state criteria to obtain the other transition counts. For a longer dataset, =SUM(E2:E251) gives the first full 250-day rolling count; move both endpoints down one row for each subsequent window. Validate missing dates, duplicate rows and non-numeric values before applying formulas.

13Diagnosing a failed backtest

Record each exception’s date, VaR, realised loss, exceedance amount, market context, data findings and action owner. Keep original results when correcting data or modifying the model. Repeatedly retuning until a test passes makes the reported test less informative; evaluate changes on a fresh holdout period.

14Monte Carlo VaR must also be backtested

Monte Carlo simulation generates a forecast distribution. Backtesting compares its archived risk forecasts with later outcomes. More simulated paths reduce simulation noise but do not repair a wrong distribution, omitted risk factor or stale parameter.

This geometric Brownian motion specification assumes constant drift and volatility over the step. It implies normally distributed log returns and positive prices. Other Monte Carlo models can incorporate different dynamics; simulation itself does not guarantee realistic tails.

For simulated losses, 99% VaR is the 99th percentile. For signed P&L, it is minus the 1st percentile. For terminal portfolio values, take the initial value minus the 1st percentile terminal value. Record path count and random seed, and assess numerical stability before producing daily forecasts.

15VaR backtesting versus Expected Shortfall

VaR identifies a threshold; Expected Shortfall measures average tail loss. A model can get the frequency of VaR breaches approximately right while badly underestimating how severe those losses are.

Track exceedance magnitudes and complement VaR validation with an appropriate ES or joint VaR–ES backtest. Simply counting VaR exceptions is not an ES backtest. A small tail sample makes severity estimates noisy, so pair formal tests with stress scenarios and model-risk investigation.

16Common errors to avoid

17Knowledge check

18Further study and source notes

This article adapts the Academy’s Session 15 learning materials: Backtesting of VaR Models Video Guide (sections 1–15), 14. Back Testing VAR (binomial and solved-problem worksheets), and Session 15 — VaR Backtesting & Monte Carlo Simulation (backtesting and simulation sections). The additional Backtesting of VAR guide (28 July 2025, pp. 2–7) supplies the metals portfolio case study in Section 19. The worked statistics above were recalculated; illustrative classroom shortcuts are qualified in the text.

• Basel Framework MAR32 — backtesting and P&L attribution requirements.
• Basel Framework MAR99 — backtesting application guidance.
• Peter Christoffersen (1998), Evaluating Interval Forecasts.
• Continue learning: Value at Risk and Expected Shortfall.
• Return to Financial Analytics topics.

19Metals portfolio case study: Gold, Silver and Copper

The additional Academy guide, Backtesting of VAR by Prof. V. Ravichandran (28 July 2025), reports an equally weighted Gold (GC=F), Silver (SI=F) and Copper (HG=F) futures-price study covering 4 May 2020 to 28 July 2025. Its reported sample contains 1,316 daily observations. The figures below are taken from that guide; the original price series was not supplied with it, so the prices, sample construction and VaR estimates have not been independently reproduced.

Read volatility and correlation together

The PDF reports daily log-return standard deviations of 0.995% for Gold, 2.035% for Silver and 1.584% for Copper. Its Gold–Silver correlation is 0.753, Gold–Copper 0.338 and Silver–Copper 0.416. Silver has the highest standalone volatility in this sample; exact portfolio risk contributions also depend on weights and covariances. Lower reported correlations with Copper suggest diversification potential, but do not guarantee the same behaviour in a future stress period.

Recalculate the coverage and independence statistics

Using the reported count and transition table gives the following arithmetic. These calculations validate the reported statistical summaries, not the underlying market-data pipeline.

*The combined statistic reproduces the guide’s convention: full-sample LRuc plus transition-based LRind. For a formal conditional-likelihood implementation, align treatment of the first observation across both components, as discussed in Section 9.

Turn the illustration into a forecasting exercise

1. Audit the futures series: specify contract selection, rolls, trading calendars and currency. Investigate roll-related jumps and stale quotes.
2. Avoid future-data filling: backward-filling missing prices uses a later observation and can leak information. Forward-filling may create artificial zero returns. Align valid trading observations and document exclusions.
3. Define portfolio returns correctly: weighted log returns are a convenient approximation, not the exact simple return of a rebalanced portfolio. For simple returns, use rₚ,ₜ = Σwᵢ,ₜ₋₁rᵢ,ₜ. For actual futures P&L, use signed contract counts × contract multipliers × price changes, with an explicit capital or notional denominator.
4. Separate estimation from testing: for example, use the first 250 observations to estimate the next day’s VaR, then roll forward. If all 1,316 observations are valid returns, this leaves 1,066 one-step forecasts, not 1,316.
5. Re-estimate both thresholds daily: use past returns only for historical quantiles and normal-model parameters. Log returns are not automatically normal; inspect skewness and tails.
6. Record and test each model separately: calculate its own hits, transition counts, coverage, independence and loss severity. Plot realised outcomes against the dated VaR forecasts, rather than a full-sample horizontal line.

Try it: enter T = 1,316, confidence = 97.5% and x = 33 in the interactive laboratory. Then ask why a nearly perfect in-sample hit rate may change after genuine rolling forecasts replace the fixed full-sample threshold.