Improving the aggregation and evaluation of NBA mock drafts
-
Jared D. Fisher
and
Colin Montague
Abstract
If professional teams can accurately predict the order of their league’s draft, they would have a competitive advantage when using or trading their draft picks. Many experts and enthusiasts publish forecasts of the order players are drafted into professional sports leagues, known as mock drafts. Using a novel dataset of mock drafts for the National Basketball Association (NBA), we explore mock drafts’ ability to forecast the actual draft. We analyze authors’ mock draft accuracy over time and ask how we can reasonably aggregate information from multiple authors. For both tasks, mock drafts are usually analyzed as ranked lists, and in this paper, we propose ways to improve on these methods. We propose that rank-biased distance is the appropriate error metric for measuring accuracy of mock drafts as ranked lists. To best combine information from multiple mock drafts into a single consensus mock draft, we also propose a combination method based on the ideas of ranked-choice voting. We show that this method provides improved forecasts over the standard Borda count combination method used for most similar analyses in sports, and that either combination method provides a more accurate forecast across seasons than any single author.
Funding source: National Science Foundation
Award Identifier / Grant number: 1745640
Acknowledgments
The authors thank Richard Yu for his work on the early phases of this project. They also thank Nathan Sandholtz, David Grimsman, Chris Archibald, and Nate Hawkins for providing feedback on early versions of this manuscript.
-
Research ethics: Not applicable.
-
Author contributions: The views expressed in this manuscript are those of the authors and do not necessarily reflect the views of their employers. The authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Competing interests: The authors state no conflict of interest.
-
Research funding: This work was partially supported by the U.S. National Science Foundation DMS RTG Grant #1745640.
-
Data availability: The full dataset is proprietary, but individual mock drafts can be found widely on the internet.
Appendix A: Detailed RCA demonstration
To demonstrate how ranked-choice aggregation works, we now show how to aggregate the GOAT mock drafts of Table 1 using Algorithm 1. The first set of ballots are simply the top ranked players from each of the mocks (we can just read across row 1), such that there are four votes for Michael Jordan and one vote for Robert Horry, so that Jordan is clearly the first pick of the aggregate mock and is removed from the list of available players. We can then visualize the rankings as
| Order | Mock A | Mock B | Mock C | Mock D | Mock E |
|---|---|---|---|---|---|
| 1 | L. James | L. James | L. James | L. James | R. Horry |
| 2 | K. Abdul-Jabbar | K. Abdul-Jabbar | B. Russell | B. Russell | K. Abdul-Jabbar |
| 3 | B. Russell | B. Russell | K. Abdul-Jabbar | K. Bryant | S. Pippen |
| 4 | K. Bryant | E. Johnson | E. Johnson | K. Abdul-Jabbar | K. Bryant |
and the ballots for the second pick are counted with four votes for LeBron James and one vote for Robert Horry. Thus, LeBron James is clearly the second pick of the aggregate mock and is removed for the list of available players:
| Order | Mock A | Mock B | Mock C | Mock D | Mock E |
|---|---|---|---|---|---|
| 1 | K. Abdul-Jabbar | K. Abdul-Jabbar | B. Russell | B. Russell | R. Horry |
| 2 | B. Russell | B. Russell | K. Abdul-Jabbar | K. Bryant | K. Abdul-Jabbar |
| 3 | K. Bryant | E. Johnson | E. Johnson | K. Abdul-Jabbar | S. Pippen |
| 4 | K. Bryant |
For the third aggregate RCA pick, the ballots yield two votes for Kareem Abdul-Jabbar, two votes for Bill Russell, and one vote for Robert Horry. As no player has a majority, we drop the player with the fewest positive number of votes from the list of eligible players, which here is Robert Horry.
| Order | Mock A | Mock B | Mock C | Mock D | Mock E |
|---|---|---|---|---|---|
| 1 | K. Abdul-Jabbar | K. Abdul-Jabbar | B. Russell | B. Russell |
|
| 2 | B. Russell | B. Russell | K. Abdul-Jabbar | K. Bryant | K. Abdul-Jabbar |
| 3 | K. Bryant | E. Johnson | E. Johnson | K. Abdul-Jabbar | S. Pippen |
| 4 | K. Bryant |
Now recounting the ballots (i.e. each author’s top eligible player), we have three votes for Kareem Abdul Jabbar, which is a majority, making him the RCA aggregate mock’s third pick and removing him from the available list. This leads to the ballots for the fourth pick
| Order | Mock A | Mock B | Mock C | Mock D | Mock E |
|---|---|---|---|---|---|
| 1 | B. Russell | B. Russell | B. Russell | B. Russell | R. Horry |
| 2 | K. Bryant | E. Johnson | E. Johnson | K. Bryant | S. Pippen |
| 3 | K. Bryant |
| Order | Mock A | Mock B | Mock C | Mock D | Mock E |
|---|---|---|---|---|---|
| 1 | K. Bryant | E. Johnson | E. Johnson | K. Bryant | R. Horry |
| 2 | S. Pippen | ||||
| 3 | K. Bryant |
| Order | Mock A | Mock B | Mock C | Mock D | Mock E |
|---|---|---|---|---|---|
| 1 | K. Bryant | E. Johnson | E. Johnson | K. Bryant |
|
| 2 | S. Pippen | ||||
| 3 | K. Bryant |
There is still not an immediate majority, with Kobe Bryant and Magic Johnson each receiving two votes and Scottie Pippen receiving one. Now Scottie Pippen is dropped from the eligible list
| Order | Mock A | Mock B | Mock C | Mock D | Mock E |
|---|---|---|---|---|---|
| 1 | K. Bryant | E. Johnson | E. Johnson | K. Bryant |
|
| 2 |
|
||||
| 3 | K. Bryant |
and we see that Kobe has three votes and Magic has two, earning Kobe the nod for the fifth pick of our aggregate mock draft. At this point, two of the mocks have used all of their listed picks, such that the next round of voting sees Magic Johnson with two votes and Robert Horry with one, giving Magic the sixth pick. That leaves one mock remaining, such that we simply follow its order. Hence, our RCA aggregate mock is as follows.
Michael Jordan
LeBron James
Kareem Abdul-Jabbar
Bill Russell
Kobe Bryant
Earvin “Magic” Johnson
Robert Horry
Scottie Pippen
References
Boehmer, N., Bredereck, R., and Peters, D. (2023). Rank aggregation using scoring rules. Proc. AAAI Conf. Artif. Intell. 37: 5515–5523. https://doi.org/10.1609/aaai.v37i5.25685.Search in Google Scholar
Borda, J.-C.D. (1781). M’emoire sur les’ elections au scrutin, 102. Histoire de l’Acad’emie Royale des Sciences, Paris, pp. 657–666.Search in Google Scholar
Caron, F. and Teh, Y. (2012). Bayesian nonparametric models for ranked data. Adv. Neural Inf. Process. Syst. 25.Search in Google Scholar
Caudill, S.B.JR., Mixon, F.G., and Mixon, C.P. (2014). NFL draftnikology: euclidean metrics and other approaches to scoring ranking predictions. Commun. Stat. Simulat. Comput. 43: 237–248. https://doi.org/10.1080/03610918.2012.700363.Search in Google Scholar
Etesamipour, B. and Hammell II, R.J. (2020). Investigation of ranking methods within the military value of information (VoI) problem domain. In: Lesot, M.-J., Vieira, S., Reformat, M.Z., Carvalho, J.P., Wilbik, A., Bouchon-Meunier, B., and Yager, R.R. (Eds.), Information processing and management of uncertainty in knowledge-based systems. Springer International Publishing, Cham, pp. 129–142.Search in Google Scholar
Fraenkel, J. and Grofman, B. (2014). The Borda count and its real-world alternatives: comparing scoring rules in Nauru and Slovenia. Aust. J. Polit. Sci. 49: 186–205. https://doi.org/10.1080/10361146.2014.900530.Search in Google Scholar
Lavey, F. (2014). Mocking mock drafts, Available at: https://harvardsportsanalysis.org/2014/10/mocking-mock-drafts/ (Accessed 12 July 2023).Search in Google Scholar
Lin, S. (2010). Rank aggregation methods. WIREs Comput. Stat. 2: 555–570. https://doi.org/10.1002/wics.111.Search in Google Scholar
Luce, R.D. (1959). Individual choice behavior: a theoretical analysis. Wiley, New York.Search in Google Scholar
Manfred, T. (2011). The ultimate 2011 NBA mock draft scorecard: which hoops forecaster did the best last night? Available at: https://www.businessinsider.com/nba-mock-drafts-2011-6 (Accessed 12 July 2023).Search in Google Scholar
Marden, J.I. (1996). Analyzing and modeling rank data. CRC Press, Boca Raton.Search in Google Scholar
McKeough, K. and Glickman, M. (2024). Plackett–Luce modeling with trajectory models for measuring athlete strength. J. Quant. Anal. Sports 20: 21–35. https://doi.org/10.1515/jqas-2021-0034.Search in Google Scholar
Mulford, M. (2023). Mavs GM Nico Harrison on draft night trades: ‘we’re not done at all, Available at: https://www.si.com/nba/mavericks/news/dallas-mavs-nico-harrison-nba-draft-dereck-lively-olivier-maxence-prosper-trade-sign-free-agency-rumors (Accessed 15 March 2024).Search in Google Scholar
National Basketball Association (2023). 2023 NBA-NBPA collective bargaining agreement, Available at: https://ak-static.cms.nba.com/wp-content/uploads/sites/4/2023/06/2023-NBA-Collective-Bargaining-Agreement.pdf (Accessed 28 March 2024).Search in Google Scholar
Plackett, R.L. (1975). The analysis of permutations. J. Roy. Stat. Soc. C Appl. Stat. 24: 193–202. https://doi.org/10.2307/2346567.Search in Google Scholar
Pros, F. (2023). NFL mock draft accuracy methodology, Available at: https://www.fantasypros.com/nfl-mock-draft-accuracy-methodology/ (Accessed 12 July 2023).Search in Google Scholar
Rivera, J. (2022). Grading the 2022 NFL mock drafts: how did Mel Kiper, Daniel Jeremiah, Todd McShay, Peter King & other experts fare? Available at: https://www.sportingnews.com/us/nfl/news/nfl-mock-draft-2022-grades/p1rgccjqoeicodbh1pnzyori (Accessed 12 July 2023).Search in Google Scholar
Robinson, B. (2020). The value of mock drafts, Available at: https://www.footballoutsiders.com/stat-analysis/2020/value-mock-drafts (Accessed 12 July 2023).Search in Google Scholar
Shieh, G.S. (1998). A weighted Kendall’s tau statistic. Stat. Probab. Lett. 39: 17–24. https://doi.org/10.1016/s0167-7152(98)00006-6.Search in Google Scholar
Tice, N. (2018). A review of NBA mock drafts, Available at: https://www.samford.edu/sports-analytics/fans/2018/A-Review-of-NBA-Mock-Drafts (Accessed 12 July 2023).Search in Google Scholar
Webber, W., Moffat, A., and Zobel, J. (2010). A similarity measure for indefinite rankings. ACM Trans. Inf. Syst. 28. https://doi.org/10.1145/1852102.1852106.Search in Google Scholar
Zubatko, J. (2024). NBA win shares, Available at: https://www.basketball-reference.com/about/ws.html (Accessed 15 March 2024).Search in Google Scholar
© 2024 Walter de Gruyter GmbH, Berlin/Boston
