Calibration of the scoring model with censored data

Abstract

In connection with the introduction of the Basel II Agreement and IFRS 9, the question of a more accurate assessment of bank credit risk is becoming increasingly important. In accordance with this provision, banks independently calculate the credit risk assessment, which is most often based on a historical sample in the form of a scoring model. The problem that arises when building a model is the evaluation of credit agreements that they stop functioning before the date on which the model forecast was built, that is, these loans are eliminated from the observation before the end date of the study. These loans are called censored, and in the context of the study there is censorship on the right. At the same time, the influence of such credit agreements on the level of bank default is significant, and, therefore, the value that serves as the basis for calibrating the scoring model also influences the value of the calibration coefficient. The purpose of this study is to solve the problem of accounting for censored data when calibrating a scoring model at the validation stage. The article discusses various ways of accounting for censored data, namely, 1) accounting for censored loans as “good”, 2) excluding censored loans from the sample, 3) Kaplan-Meier method, 4) weighting method. At the same time, attention is paid to the currently relevant issue of several estimates of the share of defaults itself, taking into account censored contracts, and the use of censored data when adjusting the model risk assessment during the validation of the scoring model. In this article, for each of the censored data accounting methods, the influence of the calibration coefficient on the ratio of the model number of defaults to the actual is analyzed, for which three methods of model calibration are considered: Linear calibration values from the probabilities, Linear calibration values from the odds, Logarithmic calibration values from the odds. According to the results of the research, the conclusion is drawn on the dependence of the method of accounting for censored data on the policy of a credit institution. A regional retail bank experience is taken to bring an example for calculation. According to the results of the study, it is concluded that the method of accounting for censored data depends on the policy of a credit institution: for organizations with a low-risk appetite, it is necessary to use the method of eliminating censored loans, for organizations with a high-risk appetite, consider censored loans as “good”, and to obtain more accurate forecast with adequate risk appetite use the methods of weighing censored data and Kaplan-Meyer. Further studies in the field will consider censored data not only at the stage of validation of the scoring model, but also at the initial stage of its construction.

Keywords

credit risk, commercial bank, probability of default, a scoring model, validation of a scoring model, censored data, methods for the accounting of censored data, calibration coefficient, calibration methods, risk appetite

For citation

Shirobokova M.A. Calibration of the scoring model with censored data. Perm University Herald. Economy, 2019, vol. 14, no. 3, pp. 406–420. DOI 10.17072/1994-9960-2019-3-406-420

References

1. Aleskerov F.T., Solodkov V.M., Chelnokova D.S. Dinamicheskii analiz patternov povedeniya kommercheskikh bankov Rossii [A dynamic analysis of behavioral patterns of Russian commercial banks]. Ekonomicheskii zhurnal Vysshei shkoly ekonomiki [Higher School of Economics. Economic Journal], 2006, vol. 10, no. 1, pp. 48–61. (In Russian).
2. Aleskerov F.T., Belousova V.Yu., Serdyuk M.Yu., Solodkov V.M. Stereotipy povedeniya rossiiskikh bankov [Stereotypes of behavior of Russian banks]. Bankovskoe delo [Banking], 2008, no. 7, pp. 44–50. (In Russian).
3. Aleskerov F.T., Andrievskaya I.K., Penikas G.I., Solodkov V.M. Analiz matematicheskikh modelei Bazel' II. 2-e izd., ispr [Analysis of mathematical models Basel II. 2nd ed. corr.]. Moscow, Fizmatlit Publ., 2013. 295 p. (In Russian).
4. Kudryavtseva M. Identifikatsiya znachimykh riskov: metodicheskie podkhody i prakticheskie rezul'taty [Identification of significant risks: Methodological approaches and practical results]. Risk-menedzhment v kreditnoi organizatsii [Risk Management in a Credit Institution], 2017, no. 3, pp. 92–102.
5. Shirobokova M.A., Letchikov A.V. Sravnenie metodov kalibrovki skoringovoi modeli pri prognozirovanii logisticheskoi regressiei [Comparison of calibration methods of the scoring model based on the logistic regression]. Vestnik Udmurtskogo universiteta. Seriya ekonomika i pravo [Bulletin of Udmurt University. Series Economics and Law], 2017, no. 2, pp. 74–79. (In Russian).
6. Koks D.R., Ouks D. Analiz dannykh tipa vremeni zhizni. Per. s angl. O.V. Selezneva [Analysis of survival data. Transl. from Engl. O.V. Selezneva]. Moscow, Finansy i statistika Publ., 1988. 191 p. (In Russian).
7. Letchikov A.V., Matveev R.Yu., Shirobokova M.A. Reshenie problemy tsenzurirovannykh dannykh pri modelirovanii otsenki individual'nogo kreditnogo riska [Solving the problem of censored data in modeling the individual credit risk estimation]. Vestnik Udmurtskogo universiteta. Seriya ekonomika i pravo [Bulletin of Udmurt University. Series Economics and Law], 2019, no. 1, pp. 34–41. (In Russian).
8. Gruzdev A.V. Prognoznoe modelirovanie v IBM SPSS Statistics, R i Python: metod derev'ev reshenii i sluchainyi les [Predictive modeling in IBM SPSS Statistics, R and Python: Decision tree method and random forest]. Moscow, DMK Press Publ., 2018. 642 p. (In Russian).
9. Bannykh A.A., Letchikov A.V. Metodika otsenki kreditnogo riska zaemshchika s primeneniem skoringa byuro kreditnykh istorii [Method of credit risk assessment with the use of credit bureau scoring]. Vestnik Udmurtskogo universiteta. Seriya ekonomika i pravo [Bulletin of Udmurt University. Series Economics and Law], 2013, no. 4, pp. 5–9. (In Russian).
10. Bannykh A.A., Letchikov A.V. Metodika rascheta ekonomicheskogo kapitala na pokrytie nepredvidennykh poter' po portfelyu potrebitel'skikh kreditov [Calculation of economic capital for covering unexpected credit losses under consumer loans portfolio]. Vestnik Udmurtskogo universiteta. Seriya ekonomika i pravo [Bulletin of Udmurt University. Series Economics and Law], 2015, no. 1, pp. 18–24. (In Russian).
11. Letchikov A.V. Raschet individual'nogo kreditnogo riska s primeneniem modeli geometricheskogo raspredeleniya [Calculation of individual credit risk by the geometric distribution model]. Vestnik Udmurtskogo universiteta. Seriya ekonomika i pravo [Bulletin of Udmurt University. Series Economics and Law], 2018, no. 2, pp. 208–213. (In Russian).
12. Dirick L., Claeskens G., Baesens B. Time to default in credit scoring using survival analysis: A benchmark study. Journal of the Operational Research Society, 2017, vol. 68, iss. 6, pp. 652–665.
13. Kaplan E.L., Meier P. Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 1958, vol. 53, no. 282, pp. 457–481.
14. Diez D. Survival analysis in R. Available at: https://folk.ntnu.no/bo/TMA4275/Download/ R.tutorialDiez.pdf. (accessed: 31.05.2019).
15. Ishwaran H. The effect of splitting on random forests. Machine Learning, 2015, vol. 99, no. 1, pp. 75–118. doi: 10.1007/s10994-014-5451-2.
16. Mogensen U.B., Ishwaran H., Gerds T.A. Evaluating random forests for survival analysis using prediction error curves. University of Copenhagen, 2012. Available at: https://ifsv.sund.ku.dk/biostat/ annualreport/images/4/4d/Research_Report_10-8.pdf (accessed 31.05.2019).
17. Man R. Survival analysis in credit scoring: A framework for PD estimation. University of Twente, 2014. Available at: https://pdfs.semanticscholar.org/b4e3/ee5a66e180ba6d3cc7174ee2327
99 cfd1831.pdf (accessed 31.05.2019).
18. Shirobokova M.A. Model' otsenki riska defolta na vsem protyazhenii zhizni kredita [Model of evaluating the default credit risk throughout the whole life of the loan]. Vestnik Udmurtskogo universiteta. Seriya ekonomika i pravo [Bulletin of Udmurt University. Series Economics and Law], 2018, no. 2, pp. 228–233. (In Russian).
19. Philosophov L. Assessing validity and accuracy of the Basel II Model in measuring credit risks of individual borrowers and credit portfolios. SSRN Electronic Journal. 2012. doi:10.2139/ssrn.655205.
20. Letchikov A.V., Markova A.A. Prognoznaya otsenka ubytkov po kreditnomu portfelyu na osnove migratsionnoi modeli [Predictive assessment of losses on the loan portfolio based on the migration model]. Matematicheskie metody i intellektual'nye sistemy v ekonomike i obrazovanii: materialy vserossiiskoi zaochnoi nauchno-prakticheskoi konferentsii [Mathematical methods and intelligent systems in Economics and education. Proceedings of Russian correspondence scientific and practical conference]. Izhevsk, IEiU UdGU Publ., 2015, pp. 10–12. (In Russian).

Show full text