26 Aug 2026
How to backtest a football prediction method using historical data
Learn how to test football predictions without confusing past results with genuine performance. This guide explains train-test splits, calibration, closing odds, sample size, yield and data leakage errors.
Define the prediction and metric before looking at the results
Specify the market, the timing of the prediction and the decision rule. For example: estimate the probability of a home win before the match and compare that estimate with the available odds. Also record the return from each bet, net profit, number of bets and strike rate. Yield is net profit divided by the total amount staked, multiplied by 100. If the method selects only certain matches, report the selection rate, because excluding matches without a defined criterion can inflate the results.
Separate training, validation and testing in chronological order
In football, future data must not influence decisions about past matches. Use a time-based split: older data for training, a subsequent period for validation and the final segment for testing. The test set should be kept untouched until the end. A model trained on matches from 2019 to 2023 and evaluated on 2024 better reflects real-world use than a random split. To assess several seasons, you can also use walk-forward validation: train on the past, test on the next block and move the window forward.
Calibrate probabilities and compare them with a benchmark
A 60% prediction should win close to 60% of the time when grouped with other similar predictions. That is the idea of calibration. Group predictions into ranges, such as 50% to 59%, and compare the average probability with the observed frequency. Use metrics such as log loss and the Brier score, which assess the quality of the probability rather than just whether the pick was correct. Compare the method with simple benchmarks, such as the implied probability from the closing odds or a model based only on home advantage. A good strike rate alone does not prove that the probabilities are useful.
Use closing odds carefully
Closing odds are the prices available shortly before kick-off. They are a strong pricing benchmark, but they are not perfect truth: they include the bookmaker’s margin and can vary between sources. Convert decimal odds into implied probability by dividing 1 by the odds, and adjust for the margin when comparing probabilities across all outcomes in a market. In a backtest, record the odds that would genuinely have been available at the defined time. Using the lowest odds available throughout the afternoon or a price published after kick-off produces an unrealistic result. It is also useful to compare the odds at the time of the prediction with the closing odds to see whether the method found a better price than the market, without treating this alone as proof of profit.
Check sample size and yield uncertainty
Small samples produce unstable results. A high yield over 30 matches can disappear in a larger sample, especially in markets with high odds. Break results down by season, league, market and odds range, but avoid creating so many groups that each becomes too small. Calculate confidence intervals for returns and conduct sensitivity analyses, such as removing the best and worst result. The conclusion should state the total number of matches, amount staked, profit, yield and spread of results. No past performance guarantees future repetition.
Avoid data leakage in the code and data collection
Leakage occurs when information that was only available after the match enters the prediction. Examples include using the final league table, injuries confirmed after the prediction time, a line-up announced later, closing odds when the model was supposed to operate in the morning, or statistics updated with the match’s own result. When calculating moving averages, use only previous matches and shift the data by one row. Record the time of each piece of information, freeze the test set, and replicate costs, odds margins, cancellations and liquidity limits. Finally, run the method over a completely out-of-sample period and keep a report of every decision, including those that did not generate a bet.
Related pre-match analysis
Keep reading
Analysis: PK Sport · our methodology
Analysis based on public data and market signals. For analysis only — not betting advice.