Investigating experiential effects in online chess using a hierarchical Bayesian analysis
-
Adam Gee
Abstract
The presence or absence of winner-loser effects is a widely discussed phenomenon across both sports and psychology research. Investigation of such effects is often hampered by the limited availability of data. Online chess has exploded in popularity in recent years and provides vast amounts of data which can be used to explore this question. With a hierarchical Bayesian regression model, we carefully investigate the presence of such experiential effects in online chess. Using a large quantity of online chess data, we see little evidence for experiential effects that are consistent across all players, with some individual players showing some evidence for such effects. Given the challenging temporal nature of this data, we discuss several methods for assessing the suitability of our model and carefully check its validity.
Funding source: Natural Sciences and Engineering Research Council of Canada
Award Identifier / Grant number: RGPIN-2023-05335
-
Research ethics: Not applicable.
-
Informed consent: Not applicable.
-
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission. SS collected the data, SS, JC, OW developed initial model. AG SS and OW implemented all models. All authors contributed to writing and editing.
-
Use of Large Language Models, AI and Machine Learning Tools: None declared.
-
Conflict of interest: The authors state no conflict of interest.
-
Research funding: AG supported by an NSERC Canada Graduate Scholarships – Master’s program. OGW supported by NSERC-DG RGPIN-2023-05335.
-
Data availability: Data available on request from the authors and will be made available following publication in a public data repository.
Appendix: Additional analyses
Here we include some additional model fitting results which were omitted from the main text. In particular, we show selected model estimates and model diagnostics for the other cohorts considered along with an additional model checking procedure and simulation study. We also investigate the effect of using varying numbers of games for the historic influence.
A.1 Examining winner-/loser-effects across cohorts
In Figure A1 we show the posterior distributions of individual β parameters for each player in the 2000–2200 and 2300–2500 cohorts. In each case we see similar results to those shown in the main text. The majority of posterior distributions for β j contain zero, with a small number of players showing positive or negative estimates. This behaviour is common across all ranges of rating we considered.
The estimated change in win probability for the other two cohorts we consider. (a) Players in the 2000–2200 cohort. (b) Players in the 2300–2500 cohort.
A.2 Posterior predictive checking for other cohorts
In Figure A2 we demonstrate further posterior predictive checks using players from cohorts not included in the main text. We again construct the posterior predictive distributions of the Glicko-2 rating evolution. We show sample players in the 1700–1900, 2000–2200 and 2300–2500 cohorts. In each case we see that the predictive distribution appears to match the general pattern over future games, indicating our model is able to predict the temporal nature of the ranking evolution.
Posterior predictive distribution of Glicko-2 ratings for 3 players in different rating cohorts. We show the true Lichess rating in red, with 4,000 draws of the predicted rating evolution. The mean of these draws is shown in the solid black line. (a) Posterior predictive distribution for “APlayerfromEarth” (1700–1900 cohort). (b) Posterior predictive distribution for “Rerereggie” (2000–2200 cohort). (c) Posterior predictive distribution for “Lobovanvliet” (2,300–2,500 cohort).
A.3 Investigating the role of n
In the main text we fit our model with n = 1 and n = 10, considering the previous performance over the previous game and the previous 10 games respectively. Given that many playing sessions contain less than 10 games, a natural question is whether n = 10 is too long a time scale for potential experiential effects to persist. In Figure A3 we show the key global parameters γ1, γ2, μ β for the 1700–1900 and GM cohorts, for n = 1, 5, 10. In each case we see that the estimates of these parameters are essentially identical, which is also the case for the individual parameters β j .
Posterior estimates for the key global parameters γ1, γ2, μ β as we vary n in our fitted model. (a) 1700–1900 cohort. (b) GM cohort.
A.4 Permuting the game results
As an alternative approach to assessing the robustness of our model, we consider permuting the data. A similar approach was considered by Ding et al. (2025), to examine evidence for a hot hand effect in hockey. For computational considerations, we perform this analysis with a subset of the complete bullet data. We use 20 players from each of the 1700–1900 and GM cohorts, taking the final 1,000 games from each. We randomly permute the results within each focal player’s data 1,000 times, giving us 1,000 permuted datasets. For each of these we fit our proposed model with n = 1. We then examine the distribution of the posterior means of the experiential effects that are generated from each of these model fits.
Figure A4 shows these “permutation distributions” for 20 players in the 1700–1900 cohort and 20 players in the GM cohort, with the central 66 % and 95 % intervals for each estimate of β j under these permutation distributions. For all players the true estimated experiential effects lie within these 95 % permutation intervals. This aligns with our previous results, indicating that individual experiential effects are small and close to zero. We note that when we repeat this analysis for the fixed effects γ1 and γ2 the distribution of these effects under the permutations is clearly different from the estimated effects, for all cohorts, as would be expected.
Posterior mean estimates for winner/loser-effects after permuting the game data. For each the true posterior mean estimate is shown in red, with the intervals representing 66 % and 95 % of the estimates obtained under the permutation distributions. (a) Permutation estimates for 20 players in 1700–1900 cohort. (b) Permutation estimates for 20 players in the GM cohort.
A.5 Simulation study
Our previous analyses reveal that the experiential effects that we are interested in, if they exist, are likely small. In this case, even the large number of games analysed here may not be sufficient to detect such small effects. To investigate this further we construct a simulation study, examining when these effects can be identified from a cohort of 10 simulated focal players. To do this, we simulate game outcomes from the model proposed in Section 3. The values of the parameters in this model are given by:
γ1 = 0.091
γ2 = 0.0038
α j ∼ N(−0.075, 0.025). This distribution approximates the estimates from Figure A6a.
β j ∼ N(μ β , 0.01). This variance is close to the posterior mean for τ2 estimated from the 1700–1900 cohort.
The rating difference for any given game is sampled from a Laplace distribution with mean 0 and scale 30. This closely resembles the actual rating differences in the 1700–1900 cohort.
For any given game, the simulated player had 0.5 probability of being white and 0.5 probability of being black.
We choose to only look at the previous game for history here (n = 1). To investigate what effects we can detect we vary the true value of μ β in this simulation along with the number of games played by each player, using μ β = (−0.5, 0.12, 0.5) and nper player = (1,000, 5,000, 20,000) we chose two large global experiential effects in different directions and one small positive experiential effect, giving a range of possible true effects. Note that μ β = 0.12 equates to an approximate average boost in win percentage by 3 % when coming from a win for all players in the dataset. We chose a maximum of 20,000 games per player to be somewhat larger than the average number of games per player in the 1700–1900 cohort. We show the posterior estimates of μ β and β j for each of these 9 simulation settings in Figure A5.
66 % and 95 % credible intervals for estimated effects for μ β and β j . In both figures the red vertical lines represent the true values of each respective parameter. (a) The estimated values of μ β for 3 different known effect sizes. (b) The estimated experiential effects for 10 simulated players for known μ β = 0.12.
In each simulation study, the 95 % credible intervals contain the true parameter values for both μ β and each β j . We note that when estimating individual β j , we need a large number of games to get posterior intervals which do not contain 0 when the true value is non-zero. While a large number of games is required to detect these small effects, as shown in our model checking procedures, these small effects have little importance in describing the outcomes of these games.
A.6 Player effects
Finally, we show some of the estimated player effects which we have excluded from the main text. In Figure A6 we show the posterior estimates for each α j for the 1700–1900 and GM cohort. For the lower ability players many of these effects appear to be negative. This agrees with intuition that these players are likely to win less than half of their games. For the GM cohort we see some strong positive player effects, including for Magnus Carlsen.
Estimated player effects for the 1700–1900 and GM cohorts. (a) Player effects for the 1700–1900 cohort. (b) Player effects for the GM cohort.
References
Abe, T., Tada, C., and Nagayama, T. (2021). Winner and loser effects of juvenile cricket gryllus bimaculatus. J. Ethol. 39: 47–54, https://doi.org/10.1007/s10164-020-00671-1.Suche in Google Scholar
Adler, P. (1981). Momentum, a theory of social action. Sage Publications, Beverly Hills, CA.Suche in Google Scholar
Bakker, T.C.M. (1986). Aggressiveness in sticklebacks (Gasterosteus aculeatus L.): a behaviour-genetic study. Behaviour 98: 1–144, https://doi.org/10.1163/156853986x00937.Suche in Google Scholar
Bakker, T.C. and Sevenster, P. (1983). Determinants of dominance in male sticklebacks (Gasterosteus aculeatus L.). Behaviour 86: 55–71, https://doi.org/10.1163/156853983x00561.Suche in Google Scholar
Benelli, G., Desneux, N., Romano, D., Conte, G., Messing, R.H., and Canale, A. (2015). Contest experience enhances aggressive behaviour in a fly: when losers learn to win. Proc. R. Soc. London, Ser. B: Biol. Sci. 5: 9347, https://doi.org/10.1038/srep09347.Suche in Google Scholar PubMed PubMed Central
Bergman, D.A. and Moore, P.A. (2003). Field observations of intraspecific agonistic behavior of two crayfish species, Orconectes rusticus and Orconectes virilis, in different habitats. Biol. Bull. 205: 26–35, https://doi.org/10.2307/1543442.Suche in Google Scholar PubMed
Briffa, M. and Lane, S.M. (2017). The role of skill in animal contests: a neglected component of fighting ability. Proc. R. Soc. B 284: 20171596, https://doi.org/10.1098/rspb.2017.1596.Suche in Google Scholar PubMed PubMed Central
Cohen-Zada, D., Krumer, A., and Shtudiner, Z.E. (2017a). Psychological momentum and gender. J. Econ. Behav. Organ. 135: 66–81, https://doi.org/10.1016/j.jebo.2017.01.009.Suche in Google Scholar
Cohen-Zada, D., Krumer, A., Rosenboim, M., and Shapir, O.M. (2017b). Choking under pressure and gender: evidence from professional tennis. J. Econ. Psychol. 61: 176–190, https://doi.org/10.1016/j.joep.2017.04.005.Suche in Google Scholar
Dijkstra, P.D., Schaafsma, S.M., Hofmann, H.A., and Groothuis, T.G. (2012). ‘Winner effect’ without winning: unresolved social conflicts increase the probability of winning a subsequent contest in a cichlid fish. Physiol. Behav. 105: 489–492, https://doi.org/10.1016/j.physbeh.2011.08.029.Suche in Google Scholar PubMed
Ding, L., Cribben, I., Ingolfsson, A., and Tran, M. (2025). Do goalies get hot in the playoffs? A multilevel logistic regression analysis. J. Sports Anal. 11: 22150218251314737, https://doi.org/10.1177/22150218251314737.Suche in Google Scholar
Dugatkin, L.A. (1997). Winner and loser effects and the structure of dominance hierarchies. Behav. Ecol. 8: 583–587, https://doi.org/10.1093/beheco/8.6.583.Suche in Google Scholar
Dugatkin, L.A. and Druen, M. (2004). The social implications of winner and loser effects. Proc. R. Soc. London, Ser. B: Biol. Sci. 271: S488–S489, https://doi.org/10.1098/rsbl.2004.0235.Suche in Google Scholar PubMed PubMed Central
Dugatkin, L.A. and Earley, R.L. (2004). Individual recognition, dominance hierarchies and winner and loser effects. Proc. R. Soc. London, Ser. B: Biol. Sci. 271: 1537–1540, https://doi.org/10.1098/rspb.2004.2777.Suche in Google Scholar PubMed PubMed Central
Eldakar, O.T., Buckwold, N., and Gallup, A.C. (2022). Carpe diem: winner and loser effects are constrained to same-day competitions in collegiate baseball. J. Ethol. 40: 97–101, https://doi.org/10.1007/s10164-021-00728-9.Suche in Google Scholar
Fawcett, T.W. and Johnstone, R.A. (2010). Learning your own strength: winner and loser effects should change with age and experience. Proc. R. Soc. B: Biol. Sci. 277: 1427–1434, https://doi.org/10.1098/rspb.2009.2088.Suche in Google Scholar PubMed PubMed Central
Franz, M., McLean, E., Tung, J., Altmann, J., and Alberts, S.C. (2015). Self-organizing dominance hierarchies in a wild primate population. Proc. R. Soc. B: Biol. Sci. 282: 20151512, https://doi.org/10.1098/rspb.2015.1512.Suche in Google Scholar PubMed PubMed Central
Gabry, J., Češnovar, R., Johnson, A., and Bronder, S. (2025). cmdstanr: R interface to ‘CmdStan’, R package version 0.9.0, Available at: https://mc-stan.org/cmdstanr/.Suche in Google Scholar
Gallup, A.C., Eldakar, O.T., Schonning, M., and Yanchus, M. (2018). Winner and loser effects in collegiate baseball and softball doubleheaders. Adapt. Human Behav. Physiol. 4: 108–120, https://doi.org/10.1007/s40750-017-0082-z.Suche in Google Scholar
Gauriot, R. and Page, L. (2018). Psychological momentum in contests: the case of scoring before half-time in football. J. Econ. Behav. Organ. 149: 137–168, https://doi.org/10.1016/j.jebo.2018.02.015.Suche in Google Scholar
Gauriot, R. and Page, L. (2019). Does success breed success? A quasi-experiment on strategic momentum in dynamic contests. Econ J. 129: 3107–3136, https://doi.org/10.1093/ej/uez040.Suche in Google Scholar
Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., and Rubin, D.B. (2013). Bayesian data analysis. Third. Chapman & Hall/CRC Texts in Statistical Science Series. CRC, Boca Raton, Florida.10.1201/b16018Suche in Google Scholar
Glickman, M.E. (1999). Parameter estimation in large dynamic paired comparison experiments. J. R. Stat. Soc. Ser. C: Appl. Stat. 48: 377–394, https://doi.org/10.1111/1467-9876.00159.Suche in Google Scholar
Glickman, M.E. (2001). Dynamic paired comparison models with stochastic variances. J. Appl. Stat. 28: 673–689, https://doi.org/10.1080/02664760120059219.Suche in Google Scholar
Glickman, M.E. (2012). Example of the Glicko-2 system, Vol. 28. Boston University, Boston.Suche in Google Scholar
Glickman, M.E. and Jones, A.C. (2024). Models and rating systems for head-to-head competition. Annu. Rev. Stat. Appl. 12, https://doi.org/10.1146/annurev-statistics-040722-061813.Suche in Google Scholar
Hsu, Y. and Wolf, L.L. (1999). The winner and loser effect: integrating multiple experiences. Anim. Behav. 57: 903–910, https://doi.org/10.1006/anbe.1998.1049.Suche in Google Scholar PubMed
Kasumovic, M.M., Elias, D.O., Sivalinghem, S., Mason, A.C., and Andrade, M.C. (2010). Examination of prior contest experience and the retention of winner and loser effects. Behav. Ecol. 21: 404–409, https://doi.org/10.1093/beheco/arp204.Suche in Google Scholar PubMed PubMed Central
Kaufman, L. (2021). Chess board options: a memoir of players, games and engines, New in Chess.Suche in Google Scholar
Kura, K., Broom, M., and Kandler, A. (2015). Modelling dominance hierarchies under winner and loser effects. Bull. Math. Biol. 77: 927–952, https://doi.org/10.1007/s11538-015-0070-z.Suche in Google Scholar PubMed
Lan, Y.T. and Hsu, Y. (2011). Prior contest experience exerts a long-term influence on subsequent winner and loser effects. Front. Zool. 8: 1–12, https://doi.org/10.1186/1742-9994-8-28.Suche in Google Scholar PubMed PubMed Central
Laskowski, K.L., Wolf, M., and Bierbach, D. (2016). The making of winners (and losers): how early dominance interactions determine adult social structure in a clonal fish. Proc. R. Soc. London, Ser. B: Biol. Sci. 283: 20160183, https://doi.org/10.1098/rspb.2016.0183.Suche in Google Scholar PubMed PubMed Central
Lehner, S.R., Rutte, C., and Taborsky, M. (2011). Rats benefit from winner and loser effects. Ethology 117: 949–960, https://doi.org/10.1111/j.1439-0310.2011.01962.x.Suche in Google Scholar
Lerena, D.A., Antunes, D.F., and Taborsky, B. (2021). The interplay between winner – loser effects and social rank in cooperatively breeding vertebrates. Anim. Behav. 177: 19–29, https://doi.org/10.1016/j.anbehav.2021.04.011.Suche in Google Scholar
Lewandowski, D., Kurowicka, D., and Joe, H. (2009). Generating random correlation matrices based on vines and extended onion method. J. Multivar. Anal. 100: 1989–2001, https://doi.org/10.1016/j.jmva.2009.04.008.Suche in Google Scholar
Li, C.Y., Pan, C.Y., and Hsu, Y. (2023). Age-dependent winner – loser effects in a mangrove rivulus fish, Kryptolebias marmoratus. Anim. Cogn. 26: 1477–1488, https://doi.org/10.1007/s10071-023-01797-8.Suche in Google Scholar PubMed
Miller, J.B. and Sanjurjo, A. (2018). Surprised by the hot hand fallacy? A truth in the law of small numbers. Econometrica 86: 2019–2047, https://doi.org/10.3982/ecta14943.Suche in Google Scholar
Oldham, L., Camerlink, I., Arnott, G., Doeschl-Wilson, A., Farish, M., and Turner, S.P. (2020). Winner – loser effects overrule aggressiveness during the early stages of contests between pigs. Sci. Rep. 10: 13338, https://doi.org/10.1038/s41598-020-69664-x.Suche in Google Scholar PubMed PubMed Central
Oliveira, R.F., Silva, A., and Canario, A.V. (2009). Why do winners keep winning? Androgen mediation of winner but not loser effects in cichlid fish. Proc. R. Soc. B: Biol. Sci. 276: 2249–2256, https://doi.org/10.1098/rspb.2009.0132.Suche in Google Scholar PubMed PubMed Central
Oliveira, R.F., Silva, J.F., and Simoes, J.M. (2011). Fighting zebrafish: characterization of aggressive behavior and winner–loser effects. Zebrafish 8: 73–81, https://doi.org/10.1089/zeb.2011.0690.Suche in Google Scholar PubMed
Ötting et al., 2020 Ötting, M., Langrock, R., Deutscher, C., and Leos-Barajas, V. (2020). The hot hand in professional darts. J. R. Stat. Soc. Ser. A: Stat. Soc. 183: 565–580, https://doi.org/10.1111/rssa.12527.Suche in Google Scholar
Page, L. and Coates, J. (2017). Winner and loser effects in human competitions. Evidence from equally matched tennis players. Evol. Hum. Behav. 38: 530–535, https://doi.org/10.1016/j.evolhumbehav.2017.02.003.Suche in Google Scholar
Pelechrinis, K. and Winston, W. (2022). The hot hand in the wild. PLoS One 17: e0261890, https://doi.org/10.1371/journal.pone.0261890.Suche in Google Scholar PubMed PubMed Central
R Core Team. (2024). R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, Available at: https://www.R-project.org/.Suche in Google Scholar
Schuett, G.W. (1997). Body size and agonistic experience affect dominance and mating success in male copperheads. Anim. Behav. 54: 213–224, https://doi.org/10.1006/anbe.1996.0417.Suche in Google Scholar PubMed
Schwartzer, J.J., Ricci, L.A., and Melloni, R.H.Jr. (2013). Prior fighting experience increases aggression in Syrian hamsters: implications for a role of dopamine in the winner effect. Aggressive Behav. 39: 290–300, https://doi.org/10.1002/ab.21476.Suche in Google Scholar PubMed
Smith, N.M.T. and Dukas, R. (2024). Winner and loser effects in humans: evidence from randomized trials. Anim. Behav. 207: 101–107, https://doi.org/10.1016/j.anbehav.2023.10.017.Suche in Google Scholar
Stan Development Team. (2024). Stan modeling language users guide and reference manual, 2.35, Available at: https://mc-stan.org.Suche in Google Scholar
Steeger, G.M., Dulin, J.L., and Gonzalez, G.O. (2021). Winning and losing streaks in the National Hockey League: are teams experiencing momentum or are games a sequence of random events? J. Quant. Anal. Sports 17: 155–170, https://doi.org/10.1515/jqas-2020-0077.Suche in Google Scholar
Stevenson, P.A. and Rillich, J. (2013). Isolation associated aggression – a consequence of recovery from defeat in a territorial animal. PLoS One 8: e74965, https://doi.org/10.1371/journal.pone.0074965.Suche in Google Scholar PubMed PubMed Central
Trannoy, S. and Kravitz, E.A. (2017). Strategy changes in subsequent fights as consequences of winning and losing in fruit fly fights. Fly 11: 129–138, https://doi.org/10.1080/19336934.2016.1259041.Suche in Google Scholar PubMed PubMed Central
Zhao, Y. and Zhang, H. (2023). Does success breed success? An investigation of momentum in elite recurve archery. Psychol. Sport Exerc. 66: 102397, https://doi.org/10.1016/j.psychsport.2023.102397.Suche in Google Scholar PubMed
© 2025 Walter de Gruyter GmbH, Berlin/Boston
