Elicitability of risk measures
Property of a risk measure for which there is a scoring function whose expectation is minimized by the true value, so forecasts can be compared; VaR has it, expected shortfall on its own does not.
A risk measure is elicitable if there is a strictly consistent scoring function S(x, y) whose expectation is minimized by the true value of the measure and by it alone: forecasting correctly becomes the only way to minimize the average score. Tilmann Gneiting set out this framework in "Making and evaluating point forecasts," Journal of the American Statistical Association, vol. 106, no. 494, 2011, pp. 746-762. Means, ratios of means and quantiles are elicitable. VaR, being a quantile, is elicited by the pinball loss, S(x, y) = (1{y ≤ x} - α)(x - y). Expected shortfall is not, because an elicitable functional has convex level sets and ES does not. The result landed as the Basel Committee was preparing the Fundamental Review of the Trading Book (FRTB) and the move from 99% VaR to 97.5% ES: how do you validate a measure that no score can rank? Tobias Fissler and Johanna Ziegel answered in "Higher order elicitability and Osband's principle," Annals of Statistics, vol. 44, no. 4, 2016, pp. 1680-1707: the pair (VaR, ES) is jointly elicitable, under mild conditions that hold in risk management practice. A forecast of ES alone cannot be scored; the pair can. The Basel framework kept the two apart. Internal model capital is computed on ES, but desk-level backtesting compares one-day VaR at 97.5% and 99%; in the consolidated version (MAR32), a desk with more than 12 exceptions at 99% or 30 at 97.5% over twelve months goes back to the standardized approach. Elicitability and backtestability remain two separate properties: Carlo Acerbi and Balázs Székely (2014) proposed ES tests that need no scoring function, and an exception count tests one model without ranking two.
Textbook case: a desk backtests its 99% VaR over 250 days. If the model is right, it expects 250 × 0.01 = 2.5 exceptions, and the binomial distribution gives a 10.8% probability of seeing at least 5 by pure chance. Two models A and B show the same 99% VaR, 10 million euros, and so the same 3 exceptions, with losses of 18, 22 and 25 million. The count does not separate them. A forecasts an ES of 14 million, B of 20. The average loss beyond VaR is (18 + 22 + 25) / 3 = 21.67 million, or 21.67 / 14 = 1.55 times A's forecast and 21.67 / 20 = 1.08 times B's. Only a method that looks at the pair (VaR, ES) sees that A underestimates the tail.
elicitability, elicitable risk measure, is expected shortfall elicitable, joint elicitability VaR ES, strictly consistent scoring function, pinball loss, Fissler Ziegel scoring function, backtesting expected shortfall, FRTB backtesting expected shortfall
Follow the research
One letter a week. First, the week's underwriting note, when one is published: a dated, argued decision, signed by Paul Bernardski. Then what the observatory published on cyber risk, political risk and insurance. No account, no tracking.
No spam, one-click unsubscribe, no data ever sold.