Verification of internal risk measure estimates

Mark H. A. Davis

doi:10.1515/strm-2015-0007

Artikel

Verification of internal risk measure estimates

Mark H. A. Davis

Veröffentlicht/Copyright: 14. Januar 2016

Veröffentlicht von

Veröffentlichen auch Sie bei De Gruyter Brill

Manuskript einreichen Informationen für Autor*innen

Aus der Zeitschrift Statistics & Risk Modeling Band 33 Heft 3-4

Abstract

This paper concerns sequential computation of risk measures for financial data and asks how, given a risk measurement procedure, we can tell whether the answers it produces are ‘correct’. We draw the distinction between ‘external’ and ‘internal’ risk measures and concentrate on the latter, where we observe data in real time, make predictions and observe outcomes. It is argued that evaluation of such procedures is best addressed from the point of view of probability forecasting or Dawid’s theory of ‘prequential statistics’ [12]. We introduce a concept of ‘calibration’ of a risk measure in a dynamic setting, following the precepts of Dawid’s weak and strong prequential principles, and examine its application to quantile forecasting (VaR – value at risk) and to mean estimation (applicable to CVaR – expected shortfall). The relationship between these ideas and ‘elicitability’ [24] is examined. We show in particular that VaR has special properties not shared by any other risk measure. Turning to CVaR we argue that its main deficiency is the unquantifiable tail dependence of estimators. In a final section we show that a simple data-driven feedback algorithm can produce VaR estimates on financial data that easily pass both the consistency test and a further newly-introduced statistical test for independence of a binary sequence.

Keywords: Risk measures; probability forecasting; prequential statistics; quantile and mean forecasting; calibration of estimates

MSC 2010: 62A01; 91B30; 91G70

A The Markov chain model

In model ℌμ,θ the transition probabilities of the chain are as shown in Table 4, where f=(1-μ)/μ≤1. The table also indicates the notation ni, i=1,2,3,4 we use for the number of occurrences of the four pairs 00,11,01,10 in a sample of size n. It should be clear that n3 and n4 play no real role in the problem, since algebraically it must be the case that |n3-n4|≤1, so for a large sample n3≈n4≈12⁢(n-n1-n2).

Table 4

Markov chain transition probabilities, f=(1-μ)/μ. The sample size is n=n1+n2+n3+n4.

xk-1	xk	pθ(xk\|xk-1)	pμ(xk\|xk-1)	# in sample
0	0	1-θ	1-μ	n1
1	1	1-θ⁢f	μ	n2
0	1	θ	μ	n3
1	0	θ⁢f	1-μ	n4

For any k the probability mass distribution of Yk is m⁢(x)=1-μ+(2⁢μ-1)⁢x, x=0,1, while for n>0 the distribution of (Y0,…,Yn) is

pnθ(x0,…,xn)=m(x0)∏k=1npθ(xk|xk-1).

When θ=μ the Yk are i.i.d. with joint distribution pnμ⁢(x0,…,xn)=∏k=0nm⁢(xk) so, referring to Table 4, the likelihood ratio LRn=pnθ/pnμ is given by

LRnθ⁡(x0,…,xn)=(1-θ)n1⁢(1-θ⁢f)n2⁢θn3⁢(θ⁢f)n4⁢(1-μ)-(n1+n4)⁢μ-(n3+n2)

={(1-θ)n1⁢(1-θ⁢f)n2⁢θ(n3+n4)}⁢{fn4⁢(1-μ)-(n1+n4)⁢μ-(n3+n2)}

={(1-θ)n1⁢(1-θ⁢f)n2⁢θn-n1-n2}⁢{fn4⁢(1-μ)-(n1+n4)⁢μ-(n3+n2)}

and of course LRnμ≡1. The log likelihood ratio is therefore

LLRθ⁡(n1,…,n4)=Lθ⁢(n1,n2)+M⁢(n1,…,n4)

where

(A.1)Lθ⁢(n1,n2)=n1⁢log⁡(1-θ)+n2⁢log⁡(1-θ⁢f)+(n-n1-n2)⁢log⁡θ

and M=LLRθ-Lθ does not depend on θ.

Proposition A.1

For β≥12, the maximum likelihood estimator is given by

(A.2)θ^⁢(n¯1,n¯2)=12⁢f⁢(n¯-2+f⁢n¯-1-(f-c1)2+4⁢f⁢(c1-c2))

where n¯-i=(n-ni)/n=1-n¯i.

Proof.

To compute the maximum likelihood estimate we maximise Lθ over θ. We have

∂⁡Lθ∂⁡θ=-n1(1-θ)-f⁢n2(1-θ⁢f)+n-n1-n2θ=Q⁢(θ)θ⁢(1-θ)⁢(1-θ⁢f)

where

(A.3)Q⁢(θ)=f⁢n⁢θ2-(n-n2+(n-n1)⁢f)⁢θ+(n-n1-n2).

The discriminant of Q is

D=(n-n2+(n-n1)⁢f)2-4⁢f⁢n⁢(n-n1-n2)=n2⁢[(f+c1)2-4⁢f⁢c2],

where

c1=1-n2+f⁢n1n,c2=1-n1+n2n.

Under our standing assumption μ≥12 we have f≤1 and hence c1≥c2. Now D can be expressed as

D=n2⁢[(f-c1)2+4⁢f⁢(c1-c2)],

showing that D≥0 whatever the values of n1,n2. Taking into account that Q⁢(0), Q⁢(1/f)>0 and Q⁢(1)<0 we easily see that Q has a root in each of the intervals (0,1), (1,1/f) so the maximising θ^∈(0,1) is the smaller of the two roots, which is given by (A.2). ∎

Proposition A.2

In any model Hμ,θ0 with μ∈[12,1] and θ0∈[0,1], the estimator θ^ is consistent, that is, θ^⁢(n¯1,n¯2)→θ0 a.s. as n→∞.

Proof.

Associated to the chain Yk is the 4-state Markov chain Yk, k=1,2,… where Yk takes the values 1,2,3,4 respectively when (Yk-1,Yk)=(0,0),(1,1),(0,1),(1,0). The transition matrix for this chain is

𝐏=[1-θ00θ000θ0′01-θ0′0θ0′01-θ0′1-θ00θ00].

Consider first the case θ0∈(0,1). Then the chain Yk is irreducible and recurrent and consequently has a unique stationary distribution 𝐦 characterised by the property that 𝐦′⁢(𝐈-𝐏)=0, where 𝐈 is the 4×4 identity matrix. This system of equations is readily solved to give

𝐦=[(1-θ0)⁢(1-μ)μ-(1-μ)⁢θ0θ0⁢(1-μ)θ0⁢(1-μ)],

where we have substituted for θ0′ from (5.4). The numbers ni introduced above are simply the numbers of visits to state i by the chain Y in a sample of length n. Since Y is recurrent,

(A.4)limn→∞⁡nin=𝐦i a.s., i=1,2,3,4.

The quadratic form Q of (A.3) can be written as

1n⁢Q⁢(θ)=f⁢θ2-(1-n¯2+(1-n¯1)⁢f)⁢θ+(1-n¯1-n¯2).

If we substitute n¯i=𝐦i, i=1,2 we find Q⁢(θ0)=0 and hence that θ^⁢(𝐦1,𝐦2)=θ0. Now θ^⁢(n¯1,n¯2) is a continuous function of the two parameters, and hence in view of (A.4) we have limn→∞⁡θ^⁢(n¯1,n¯2)=θ0 a.s.

We now consider the cases θ0=0,1. When θ0=0, Yk=Y0 for all k, so either n1=n, n2=0 or n1=0, n2=n giving, from (A.1), values of Lθ equal to n⁢log⁡(1-θ) or n⁢log⁡(1-θ⁢f) respectively. In either case, Lθ is maximised at θ=0=θ0.

The case θ0=1 is a little more tricky. Here Yk-1=0 implies Yk=1, so n1≡0. The sample path consists of strings of ones separated by single zeros. The probability of flipping from 1 to 0 is f, so the mean length of a string of ones is 1/f. Each flip from 1 to 0 and back adds 1 to n3 and to n4, and each string of ones of length m adds m-1 to n2. So the mean growth rates in n3+n4 and in n2 are in the ratio 2:(1/f)-1=(2⁢μ-1)/(1-μ), implying that, loosely speaking, the fraction of the time spent growing n2 is ((2⁢μ-1)/(1-μ))/((2⁢μ-1)/(1-μ)+2)=2⁢μ-1. We conclude that

limn→∞⁡n¯2=2⁢μ-1 a.s.

At the limiting value,

1n⁢Lθ=(2⁢μ-1)⁢log⁡(1-θ⁢f)+2⁢(1-μ)⁢log⁡θ

and the derivative with respect to θ is

-μ⁢(1-f)⁢f1-θ⁢f+2⁢(1-μ)θ.

This is equal to +∞ at θ=0 and is finite and decreasing for θ>0. Its value at θ=1 is 1-μ>0, and we conclude that the maximum occurs at θ=1. A simple continuity argument now shows that

limn→∞⁡θ^⁢(0,n¯2)=1=q0 a.s.∎

Acknowledgements

The author thanks Paul Embrechts for introducing him to this topic, Phil Dawid for discussions of prequential statistics, and Johanna Ziegel for extremely helpful comments and advice. He is also grateful to two anonymous referees for their careful reading of the paper and many constructive suggestions for improvements. Some of this work was carried out at the Hausdorff Research Institute for Mathematics at the University of Bonn, where the author was a visitor during the Trimester Program Stochastic Dynamics in Economics and Finance in August 2013. Discussions there have been very helpful in clarifying the content of this paper.

References

[1] Abdous B. and Remillard B., Relating quantiles and expectiles under weighted-symmetry, Ann. Inst. Statist. Math. 47 (1995), 371–384. 10.1007/BF00773468Suche in Google Scholar

[2] Acerbi C. and Szekely B., Backtesting expected shortfall, Risk Magazine 16 (2014). Suche in Google Scholar

[3] Artzner P., Delbaen F., Eber M. and Heath D., Coherent measures of risk, Math. Finance 9 (1999), 203–228. 10.1017/CBO9780511615337.007Suche in Google Scholar

[4] Basel Committee on Banking Supervision, Fundamental review of the trading book, a revised market risk framework, 2nd consultative paper, Bank for International Settlements, Basel, 2013. Suche in Google Scholar

[5] Bollerslev T., Generalized autoregressive conditional heteroskedacicity, Econometrics 51 (1986), 307–327. 10.1016/0304-4076(86)90063-1Suche in Google Scholar

[6] Campbell J., Lo A. and MacKinley A., The Econometrics of Financial Markets, Princeton University Press, Princeton, 1990. Suche in Google Scholar

[7] Cervera J. and Muñoz J., Proper scoring rules for fractiles, Bayesian Statistics, Vol 5, Oxford University Press, Oxford (1996), 513–519. 10.1093/oso/9780198523567.003.0029Suche in Google Scholar

[8] Christoffersen P., Evaluating interval forecasts, Int. Econ. Rev. 39 (1998), 841–862. 10.2307/2527341Suche in Google Scholar

[9] Cont R., Empirical properties of asset returns: Stylized facts and statistical issues, Quant. Finance 1 (2001), 223–236. 10.1080/713665670Suche in Google Scholar

[10] Cont R., Deguest R. and Scandolo G., Robustness and sensitivity analysis of risk measurement procedures, Quant. Finance 10 (2010), 593–606. 10.1080/14697681003685597Suche in Google Scholar

[11] Cox D., Hinkley D. and Barndorff-Nielsen O. (Eds.), Times Series Models in Econometrics, Finance and Other Fields, Chapman & Hall, Boca Raton, 1996. 10.1007/978-1-4899-2879-5Suche in Google Scholar

[12] Dawid A. P., Present position and potential developments: Some personal views. Statistical theory: The prequential approach (with discussion), J. Roy. Statist. Soc. A 147 (1984), 278–292. 10.2307/2981683Suche in Google Scholar

[13] Dawid A. P., Probability forecasting, Encyclopaedia of Statistical Sciences, vol. 7, Wiley, New York (1986), 210–218. Suche in Google Scholar

[14] Dawid A. P. and Vovk V., Prequential probability: Principles and properties, Bernoulli 5 (1999), 125–162. 10.2307/3318616Suche in Google Scholar

[15] Dempster M. A. H. and Leemans V., An automated FX trading system using adaptive reinforcement learning, Expert Syst. Appl. 30 (2006), 543–552. 10.1016/j.eswa.2005.10.012Suche in Google Scholar

[16] Diebold F., Gunther T. and Tay A., Evaluating density forecasts with applications to financial risk management, Int. Econ. Rev. 39 (1998), 863–883. 10.2307/2527342Suche in Google Scholar

[17] Diebold F. X. and Mariano R., Comparing predictive accuracy, J. Bus. Econom. Statist. 13 (1995), 253–263. 10.3386/t0169Suche in Google Scholar

[18] Dudley R., Real Analysis and Probability, Wadsworth & Brooks/Cole, Pacific Grove, 1989. Suche in Google Scholar

[19] Embrechts P. and Hofert M., Statistics and quantitative risk management for banking and insurance, Annu. Rev. Statist. Appl. 1 (2014), 493–514. 10.1146/annurev-statistics-022513-115631Suche in Google Scholar

[20] Embrechts P., Klüppelberg C. and Mikosch T., Modelling Extremal Events: For Insurance and Finance, Springer, Berlin, 1997. 10.1007/978-3-642-33483-2Suche in Google Scholar

[21] Engle R. and Manganelli S., CAViaR, J. Bus. Econom. Statist. 22 (2004), 367–381. 10.1198/073500104000000370Suche in Google Scholar

[22] Fissler T. and Ziegel J. F., Higher-order elicitability and Osband’s principle, preprint 2015, https://arxiv.org/abs/1503.08123. 10.1214/16-AOS1439Suche in Google Scholar

[23] Gibbons J. D. and Chakraborti S., Nonparametric Statistical Inference, 5th ed., Chapman & Hall/CRC, Boca Raton, 2010. 10.1201/9781439896129Suche in Google Scholar

[24] Gneiting T., Making and evaluating point forecasts, J. Amer. Statist. Assoc. 106 (2011), 746–762. 10.1198/jasa.2011.r10138Suche in Google Scholar

[25] Gneiting T., Balabdaoui F. and Raftery A., Probabilistic forecasts, calibration and sharpness, J. R. Statist. Soc. B 69 (2007), 243–268. 10.21236/ADA454827Suche in Google Scholar

[26] Gneiting T. and Raftery A., Strictly proper scoring rules, prediction and estimation, J. Amer. Statist. Assoc. 102 (2007), 359–378. 10.1198/016214506000001437Suche in Google Scholar

[27] Gripenberg G. and Norros I., On the prediction of fractional Brownian motion, J. Applied Prob. 33 (1996), 400–410. 10.1017/S0021900200099812Suche in Google Scholar

[28] Hall P. and Heyde C. C., Martingale Limit Theory and its Application, Academic Press, New York, 1980. Suche in Google Scholar

[29] Haldane A. G., The dog and the frisbee, speech 2012, www.bankofengland.co.uk/publications/documents/speeches/2012/speech596.pdf. Suche in Google Scholar

[30] Holzmann H. and Eulert M., The role of the information set for forecasting. With applications to risk management, Ann. Appl. Stat. 8 (2014), 595–621. 10.1214/13-AOAS709Suche in Google Scholar

[31] Joliffe I. and Stephenson D. (Eds.), Forecast Verification: A Practitioner’s Guide, Wiley, New York, 2003. Suche in Google Scholar

[32] Kou S., Peng X. and Heyde C. C., External risk measures and Basel accords, Math. Oper. Res. 38 (2013), 393–417. 10.1287/moor.1120.0577Suche in Google Scholar

[33] Kuester K., Mittnik S. and Paolella M., Value-at-risk prediction: A comparison of alternative strategies, J. Finan. Econom. 4 (2006), 53–89. 10.1093/jjfinec/nbj002Suche in Google Scholar

[34] Lai T. L., Shen D. and Gross S., Evaluating probability forecasts, Ann. Statist. 39 (2011), 2356–2382. 10.1214/11-AOS902Suche in Google Scholar

[35] Lambert N., Pennock D. and Shoham Y., Eliciting properties of probability distributions, Proceedings of the 9th ACM Conference on Electronic Commerce (Chicago 2008), ACM, New York (2008), 129–138. 10.1145/1386790.1386813Suche in Google Scholar

[36] Mandelbrot B., Fractals and Scaling in Finance: Discontinuity, Concentration, Risk, Springer, New York, 1997. 10.1007/978-1-4757-2763-0Suche in Google Scholar

[37] Mandelbrot B. and Taylor H., On the distribution of stock price differences, Oper. Res. 15 (1967), 1057–1062. 10.1287/opre.15.6.1057Suche in Google Scholar

[38] Mitchell J. and Wallace K., Evaluating density forecasts: Forecast combinations, model mixtures, calibration and sharpness, J. Appl. Econometrics 26 (2011), 1023–1040. 10.1002/jae.1192Suche in Google Scholar

[39] Osband K. and Reichelstein S., Information-eliciting compensation schemes, J. Public Econ. 27 (1985), 107–115. 10.1016/0047-2727(85)90031-3Suche in Google Scholar

[40] Popper K., The Logic of Scientific Discovery, 2nd ed., Routledge, London, 2002; originally published as Logik der Forschung. Zur Erkenntnistheorie der modernen Naturwissenschaft, Springer, Vienna, 1935. Suche in Google Scholar

[41] Rockafellar R. T. and Uryasev S., Optimization of conditional value-at-risk, Risk 2 (2000), 21–41. 10.21314/JOR.2000.038Suche in Google Scholar

[42] Rockafellar R. T. and Uryasev S., Conditional value-at-risk for general loss distributions, J. Banking Finance 26 (2002), 1443–1471. 10.1016/S0378-4266(02)00271-6Suche in Google Scholar

[43] Rosenblatt M., Remarks on a multivariate transformation, Ann. Math. Stat. 23 (1952), 470–472. 10.1214/aoms/1177729394Suche in Google Scholar

[44] Savage L., Elicitation of personal probabilities and expectations, J. Amer. Statist. Assoc. 66 (1971), 783–801. 10.1080/01621459.1971.10482346Suche in Google Scholar

[45] Smith R. C., Uncertainty Quantification: Theory, Implementation, and Applications, SIAM, Philadelphia, 2014. 10.1137/1.9781611973228Suche in Google Scholar

[46] Steinwart I., Pasin C., Williamson R. and Zhang S., Elicitation and identification of properties, J. Mach. Learn. Res. 35 (2014), 482–526. Suche in Google Scholar

[47] Warner T. T., Numerical Weather and Climate Prediction, Cambridge University Press, Cambridge, 2010. 10.1017/CBO9780511763243Suche in Google Scholar

[48] Williams D., Probability with Martingales, Cambridge University Press, Cambridge, 1991. 10.1017/CBO9780511813658Suche in Google Scholar

[49] Ziegel J. F., Coherence and elicitability, Math. Finance 26 (2016), 901–918. 10.1111/mafi.12080Suche in Google Scholar

Received: 2015-3-15

Revised: 2015-9-17

Accepted: 2015-11-18

Published Online: 2016-1-14

Published in Print: 2016-12-1

Sie haben derzeit keinen Zugang zu diesem Inhalt.

Artikel in diesem Heft

https://doi.org/10.1515/strm-2015-0007

Schlagwörter für diesen Artikel

Risk measures; probability forecasting; prequential statistics; quantile and mean forecasting; calibration of estimates