Quotation Philipp, Michel, Rusch, Thomas, Strobl, Carolin, Hornik, Kurt. 2018. Measuring the Stability of Supervised Statistical Learning Results. Journal of Computational and Graphical Statistics. 27 (4), 685-700.




Stability is a major requirement to draw reliable conclusions when interpreting results from supervised statistical learning. In this article, we present a general framework for assessing and comparing the stability of results, which can be used in real-world statistical learning applications as well as in simulation and benchmark studies. We use the framework to show that stability is a property of both the algorithm and the data-generating process. In particular, we demonstrate that unstable algorithms (such as recursive partitioning) can produce stable results when the functional form of the relationship between the predictors and the response matches the algorithm. Typical uses of the framework in practical data analysis would be to compare the stability of results generated by different candidate algorithms for a dataset at hand or to assess the stability of algorithms in a benchmark study. Code to perform the stability analyses is provided in the form of an R package. Supplementary material for this article is available online.


Press 'enter' for creating the tag

Publication's profile

Status of publication Published
Affiliation WU
Type of publication Journal article
Journal Journal of Computational and Graphical Statistics
Citation Index SCI
WU-Journal-Rating new FIN-A, VW-C
Language English
Title Measuring the Stability of Supervised Statistical Learning Results
Volume 27
Number 4
Year 2018
Page from 685
Page to 700
Reviewed? Y
URL https://amstat.tandfonline.com/doi/full/10.1080/10618600.2018.1473779
DOI https://doi.org/10.1080/10618600.2018.1473779
Open Access N


Rusch, Thomas (Details)
Hornik, Kurt (Details)
Philipp, Michel (Universität Zürich, Switzerland)
Strobl, Carolin (Universität Zürich, Switzerland)
Competence Center for Empirical Research Methods WE (Details)
Research areas (ÖSTAT Classification 'Statistik Austria')
1162 Statistics (Details)
5509 Psychological methodology (Details)
5701 Applied statistics (Details)
5704 Social statistics (Details)
5912 Social sciences (interdisciplinary) (Details)
Google Scholar: Search