PEP: a tackle value measuring the prevention of expected points
-
Robert Bajons
,
Jan-Ole Koslik
Abstract
Traditional assessments of tackling in American Football often only consider the number of tackles made, without adequately accounting for their context and importance for the game. Aiming for improvement, we develop a metric that quantifies the value of a tackle in terms of the prevented expected points (PEP). Specifically, we compare the real end-of-play yard line of tackles with the predicted yard line given the hypothetical situation that the tackle had been missed. For this, we use high-resolution tracking data, that capture the position and velocity of players, and a random forest to account for uncertainty and multi-modality in yard-line prediction. Moreover, we acknowledge the difference in the importance of tackles by assigning an expected points value to each individual tree prediction of the random forest. Finally, to relate the value of tackles to a player’s ability to tackle, we fit a suitable mixed-effect model to the PEP values. Our approach contributes to a deeper understanding of defensive performances in American football and offers valuable insights for coaches and analysts.
Acknowledgments
We would like to thank the organizers of the NFL Big Data Bowl 2024 for setting up this competition and providing access to the data.
-
Research ethics: Not applicable.
-
Informed consent: Not applicable.
-
Author contributions: The author have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Conflict of interests: The authors state no conflict of interest.
-
Research funding: None declared.
-
Data availability: The raw data can be obtained from https://www.kaggle.com/competitions/nfl-big-data-bowl-2024/data.
A Further results
A.1 Cumulative PEP values
Table 3 displays the top 20 players based on their cumulative PEP values. To account for uncertainty in the evaluation of the sum of PEP values, we bootstrapped the dataset 1000 times, obtaining a distribution of cumulative PEP values. The players in the table are ranked based on the median of the cumulative PEP values obtained from this bootstrap density. Intuitively, the results from this procedure seem reasonable. A simple sanity check is to compare them with conventional tackle rankings as, e.g., provided by the NFL via their 2022 NFL tackles leaderboard. All of the well performing linebacker (six in total) are also found in the top 20 of this leaderboard (based on combined tackles), even though our data contains only the first 9 weeks of the season. While this result is to some extent reassuring, it further indicates the shortcomings of using cumulative PEP values as indicators for tackle value. In the top 20, we find mostly linebackers and safeties, whereas not a single defensive liner is present. Thus, this metric fails to account for the ability of defensive linemen to consistently stop forward movement in critical situations.
A.2 Mixed effects model results for all positions
In addition to the results presented in Section 4.3, we provide further results on the effect distribution for players in other position groups. Figure 10 is in the pendant to Figure 8 and displays the distribution of the mixed effect model estimates for the top 10 players in the remaining position groups. Note that we do not observe more than five nominal middle linebackers (MLB) with more than 10 tackles, hence only five players are shown. In principle, the results are similar to the observations of 4.3.
Distribution of the top 10 (if available) tackler effects for the remaining position groups. Players are ordered with respect to the median of the bootstrap distribution, represented by the solid line.
A.3 Rushing versus passing plays
As pointed out in the discussion (Section 5), the type of play (i.e. pass vs. run) affects our final estimate of player strength. While it would be possible to account for the play type in our mixed model specification, it is not (at least not directly) possible to characterize a cornerback’s ability. That is, we have no way of determining whether the cornerback allowed a catch that he should have already stopped earlier. Thus it is questionable, whether passing plays should be taken into account when analyzing PEP values. In this section, we briefly address this issue. To this end, we filtered tackles resulting from run plays as identified by the play type variable from play-by-play data (leaving us with 5889 tackles to analyze) and refit the model.
Table 4 presents the result of this analysis. Similar to Figure 9, it shows the top 20 players ranked by the median of the varying intercept from mixed models fitted to 1000 bootstrap samples of the data comprised solely of rushing plays. Interestingly, a safety and a cornerback pop up on top of our table. However, in comparison to Figure 9, we observe fewer cornerbacks in the top spots. We again stress that looking only at run plays reduces the number of tackles in our dataset and therefore also the number of tackles of each individual. In order to be consistent with the previous results, we displayed only players, who were able to tackle more than 10 times within run plays. Doing so excludes, for example, Dexter Lawrence (the top player in our full dataset, see Figure 9), for whom we observed exactly ten run-play-tackles.
Top 20 players considering only run plays.
| Rank | Player | Position | Mixed model intercept | Sum PEP | Avg PEP | N tackles | ||
|---|---|---|---|---|---|---|---|---|
| MM median | MM Q−2.5 % | MM Q−97.5 % | ||||||
| 1 | Vonn Bell | SS | 0.078 | −0.005 | 0.138 | 2.749 | 0.25 | 11 |
| 2 | Jeff Okudah | CB | 0.078 | −0.031 | 0.193 | 4.381 | 0.292 | 15 |
| 3 | Kareem Jackson | SS | 0.072 | 0.017 | 0.124 | 2.591 | 0.162 | 16 |
| 4 | Grady Jarrett | DT | 0.066 | −0.023 | 0.129 | −0.264 | −0.019 | 14 |
| 5 | Marcus Maye | FS | 0.066 | 0.002 | 0.162 | 2.49 | 0.192 | 13 |
| 6 | Samson Ebukam | DE | 0.056 | 0.015 | 0.095 | 1.52 | 0.101 | 15 |
| 7 | Jeffery Simmons | DT | 0.056 | 0.003 | 0.116 | 0.605 | 0.043 | 14 |
| 8 | Kenny Moore | CB | 0.049 | −0.01 | 0.129 | 0.86 | 0.066 | 13 |
| 9 | Leonard Floyd | DE | 0.048 | 0.002 | 0.089 | −0.975 | −0.075 | 13 |
| 10 | Roquan Smith | ILB | 0.048 | −0.003 | 0.103 | 2.62 | 0.06 | 44 |
| 11 | Broderick Washington | DT | 0.047 | −0.03 | 0.106 | 0.585 | 0.045 | 13 |
| 12 | Brandon Jones | SS | 0.047 | 0.011 | 0.093 | 1.741 | 0.158 | 11 |
| 13 | Zaire Franklin | OLB | 0.047 | 0.001 | 0.097 | 4.279 | 0.138 | 31 |
| 14 | Alex Anzalone | ILB | 0.04 | −0.015 | 0.093 | 1.731 | 0.082 | 21 |
| 15 | Armon Watts | NT | 0.038 | 0.003 | 0.077 | 0.805 | 0.045 | 18 |
| 16 | Demarcus Lawrence | DE | 0.038 | −0.013 | 0.092 | 2.782 | 0.199 | 14 |
| 17 | Nick Scott | SS | 0.038 | −0.025 | 0.134 | 5.693 | 0.474 | 12 |
| 18 | Christian Wilkins | DT | 0.036 | −0.017 | 0.087 | −0.908 | −0.039 | 23 |
| 19 | Rasheem Green | DE | 0.035 | −0.039 | 0.099 | 0.801 | 0.062 | 13 |
| 20 | Cameron Heyward | DT | 0.034 | −0.031 | 0.093 | 0.311 | 0.015 | 21 |
A.4 Adding missed tackles
Quantifying the value of missed tackles is an important aspect when analyzing a player’s tackling ability. As mentioned in the discussion, it is possible to extend our framework to analyzing missed tackles. To this end, we could treat missed tackles as tackles, predict the EOPY, and obtain a value for this hypothetical tackle on the EP scale. This value could again be compared to the real outcome allowing us to derive a missed tackle PEP value. However, this relies on accurately identifying tackle opportunities respectively missed tackles, which is not an easy task. The big data bowl provides information on missed tackles – these have been obtained from the data provider PFF – within the timeframe of our data. Compared to observed tackles (11,313), the number of missed tackles in the data is substantially lower (1669). Thus, we believe that solely analyzing missed tackles with this small data set is inappropriate. However, we can combine the PEP values from missed tackles and real tackles, refit the mixed effects model for the PEP values, and analyze the varying intercepts for the tacklers. In general, the results from adding missed tackles are similar to the ones obtained without them. Figures 11 and 12 provide a visual confirmation of that. However, since identifying missed tackles is intricate, it is unclear whether the missed tackles distributions with respect to players and positions in our data are accurate and reflect the true missed tackles events distribution. Therefore, we refrain from adding them to the main analysis in this work.
Relationship between mixed model tackler effect estimates with and without missed tackles. A strong linear correlation (r = 0.7473) is observable.
Distribution of the top and bottom 5 inside linebackers (ILB, left) and defensive tackles (DT, right) when adding missed tackles. Results are similar to results from Figure 8.
References
Adam, T., Ötting, M., and Michels, R. (2024). Markov-switching decision trees. AStA Adv. Stat. Anal.: 1–16.10.1007/s10182-024-00501-6Search in Google Scholar
Boehmke, B. and Greenwell, B. (2019). Hands-On Machine Learning with R. Chapman & Hall/CRC The R Series. CRC Press, New York, NY, USA.10.1201/9780367816377Search in Google Scholar
Breiman, L. (2001). Random forests. Mach. Learn. 45: 5–32.10.1023/A:1010933404324Search in Google Scholar
Brill, R.S., Yurko, R., and Wyner, A.J. (2024). Analytics, have some humility: a statistical view of fourth-down decision making. arXiv:2311.03490.10.1080/00031305.2025.2475801Search in Google Scholar
Buuren, S.V. and Fredriks, M. (2001). Worm plot: a simple diagnostic device for modelling growth reference curves. Stat. Med. 20: 1259–1277, https://doi.org/10.1002/sim.746.Search in Google Scholar PubMed
Carl, S. and Baldwin, B. (2023). nflfastR: functions to efficiently access NFL play by play data, https://github.com/nflverse/nflfastR.Search in Google Scholar
Chen, T. and Guestrin, C. (2016). XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA.10.1145/2939672.2939785Search in Google Scholar
Chu, D., Reyers, M., Thomson, J., and Wu, L.Y. (2020). Route identification in the national football league. J. Quant. Anal. Sports 16: 121–132, https://doi.org/10.1515/jqas-2019-0047.Search in Google Scholar
Curth, A., Jeffares, A., and van der Schaar, M. (2024). Why do random forests work? Understanding tree ensembles as self-regularizing adaptive smoothers. arXiv 2402: 01502.Search in Google Scholar
Deshpande, S.K. and Evans, K. (2020). Expected hypothetical completion probability. J. Quant. Anal. Sports 16: 85–94, https://doi.org/10.1515/jqas-2019-0050.Search in Google Scholar
DFL. (2024). The source of all – the official match data, https://www.dfl.de/en/topics/match-data/official-match-data/.Search in Google Scholar
Duan, T., Anand, A., Ding, D.Y., Thai, K.K., Basu, S., Ng, A., and Schuler, A. (2020). Ngboost: natural gradient boosting for probabilistic prediction. In: International Conference on machine learning. PMLR, pp. 2690–2700.Search in Google Scholar
Dutta, R., Yurko, R., and Ventura, S.L. (2020). Unsupervised methods for identifying pass coverage among defensive backs with NFL player tracking data. J. Quant. Anal. Sports 16: 143–161, https://doi.org/10.1515/jqas-2020-0017.Search in Google Scholar
Eager, E. and Seth, T. (2023). Investigating trade-offs made by American football linebackers using tracking data. J. Quant. Anal. Sports 19: 171–185, https://doi.org/10.1515/jqas-2022-0091.Search in Google Scholar
Fernandes, C.J., Yakubov, R., Li, Y., Prasad, A.K., and Chan, T.C. (2020). Predicting plays in the national football league. J. Sports Anal. 6: 35–43, https://doi.org/10.3233/jsa-190348.Search in Google Scholar
Forcher, L., Altmann, S., Forcher, L., Jekauc, D., and Kempe, M. (2022). The use of player tracking data to analyze defensive play in professional soccer-a scoping review. Int. J. Sports Sci. Coach. 17: 1567–1592, https://doi.org/10.1177/17479541221075734.Search in Google Scholar
Goes, F.R., Brink, M.S., Elferink-Gemser, M.T., Kempe, M., and Lemmink, K.A. (2021). The tactics of successful attacks in professional association football: large-scale spatiotemporal analysis of dynamic subgroups using position tracking data. J. Sports Sci. 39: 523–532, https://doi.org/10.1080/02640414.2020.1834689.Search in Google Scholar PubMed
Heiny, E.L. and Blevins, D. (2011). Predicting the Atlanta Falcons play-calling using discriminant analysis. J. Quant. Anal. Sports 7: 2, https://doi.org/10.2202/1559-0410.1230.Search in Google Scholar
Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural Comput. 9: 1735–1780, https://doi.org/10.1162/neco.1997.9.8.1735.Search in Google Scholar PubMed
Izbicki, R. and Lee, A.B. (2017). Converting high-dimensional regression to high-dimensional conditional density estimation. Elec. J. Stat. 11: 2800–2831, https://doi.org/10.1214/17-ejs1302.Search in Google Scholar
Kovalchik, S.A. (2023). Player tracking data in sports. Ann. Rev. Stat. Appl. 10: 677–697, https://doi.org/10.1146/annurev-statistics-033021-110117.Search in Google Scholar
Lopez, M., Bliss, T., Blake, A., Patton, A., McWilliams, J., Howard, A., and Cukierski, W. (2023). NFL big data bowl 2024. Kaggle. https://kaggle.com/competitions/nfl-big-data-bowl-2024.Search in Google Scholar
Michels, R. and Langrock, R. (2023). Nonparametric estimation of multivariate hidden markov models using tensor-product B-splines. arXiv preprint arXiv:2302.06510.Search in Google Scholar
Müller, O., Caron, M., Döring, M., Heuwinkel, T., and Baumeister, J. (2021). PIVOT: a parsimonious end-to-end learning framework for valuing player actions in handball using tracking data. In: International Workshop on Machine Learning and Data Mining for Sports Analytics. Springer, Cham, Switzerland, pp. 116–128.10.1007/978-3-031-02044-5_10Search in Google Scholar
Nguyen, Q., Yurko, R., and Matthews, G. J. (2023). Here comes the STRAIN: analyzing defensive pass rush in American football with player tracking data. Am. Stat. 78: 199–208, https://doi.org/10.1080/00031305.2023.2242442.Search in Google Scholar
Nguyen, Q., Jiang, R., Ellingwood, M., and Yurko, R. (2024). Fractional tackles: leveraging player tracking data for within-play tackling evaluation in American football. arXiv preprint arXiv:2403.14769.10.1038/s41598-025-85993-1Search in Google Scholar PubMed PubMed Central
Ötting and Karlis, 2023 Ötting, M. and Karlis, D. (2023). Football tracking data: a copula-based hidden Markov model for classification of tactics in football. Ann. Oper. Res. 325: 167–183, https://doi.org/10.1007/s10479-022-04660-0.Search in Google Scholar
Pospisil, T. (2024). RFCDE: random forests for conditional density estimation. R package version 0.3.1.Search in Google Scholar
Pospisil, T. and Lee, A.B. (2018). RFCDE: random forests for conditional density estimation. arXiv preprint arXiv:1804.05753.Search in Google Scholar
R Core Team. (2024). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.Search in Google Scholar
Reyers, M. and Swartz, T.B. (2023). Quarterback evaluation in the National Football League using tracking data. AStA Adv. Stat. Anal. 107: 327–342, https://doi.org/10.1007/s10182-021-00406-8.Search in Google Scholar
Rigby, R.A. and Stasinopoulos, D.M. (2005). Generalized additive models for location, scale and shape. J. Roy. Stat. Soc.: Ser. C (Appl. Stat.) 54: 507–554, https://doi.org/10.1111/j.1467-9876.2005.00510.x.Search in Google Scholar
Rigby, R., Stasinopoulos, M., Heller, G., and De Bastiani, F. (2019). Distributions for Modeling Location, Scale, and Shape: Using GAMLSS in R. Chapman & Hall/CRC The R Series. CRC Press, New York, NY, USA.10.1201/9780429298547Search in Google Scholar
Schlosser, L., Hothorn, T., Stauffer, R., and Zeileis, A. (2018). Distributional regression forests for probabilistic precipitation forecasting in complex terrain. Ann. Appl. Stat. 13, https://doi.org/10.1214/19-aoas1247.Search in Google Scholar
Stasinopoulos, M. and Rigby, R. (2007). Generalized additive models for location scale and shape (gamlss) in R. J. Stat. Software 23: 1–46, https://doi.org/10.18637/jss.v023.i07.Search in Google Scholar
Van Haaren, J. (2021). Industry-leaders in football’s use of data intelligence, https://www.scisports.com/state-of-the-football-analytics-industry-in-2021/.Search in Google Scholar
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Proc. Syst. 30.Search in Google Scholar
Yam, D.R. and Lopez, M.J. (2019). What was lost? A causal Estimate of fourth down behavior in the National Football League. J. Sports Anal. 5: 153–167, https://doi.org/10.3233/jsa-190294.Search in Google Scholar
Yurko, R., Ventura, S., and Horowitz, M. (2019). nflWAR: a reproducible method for offensive player evaluation in football. J. Quant. Anal. Sports 15: 163–183, https://doi.org/10.1515/jqas-2018-0010.Search in Google Scholar
Yurko, R., Matano, F., Richardson, L.F., Granered, N., Pospisil, T., Pelechrinis, K., and Ventura, S.L. (2020). Going deep: models for continuous-time within-play valuation of game outcomes in American football with tracking data. J. Quant. Anal. Sports 16: 163–182, https://doi.org/10.1515/jqas-2019-0056.Search in Google Scholar
Yurko, R., Nguyen, Q., and Pelechrinis, K. (2024). Nfl ghosts: a framework for evaluating defender positioning with conditional density estimation. arXiv preprint arXiv:2406.17220.Search in Google Scholar
© 2025 Walter de Gruyter GmbH, Berlin/Boston
