Tuesday, January 24, 2006

How the Forecasters Stack Up

So here are the results and how various forecasts matched them. As you can see, The Truth Hurts did very well compared to other national forecasts. If you are aware of any other forecast sites I am more than prepared to add them to this comparison chart. If I find the time I will do the same comparisons at the subnational level.

I found through the election that my model's weakness, though it was not that bad, was with smaller parties. The final results reflect that as well as no other predictor had a lower error rate for the two largest parties combined and only The Truth Hurts predicted one of those two parties exactly. There is also some irony that the forecast with the absolutely worst results was from a professional polling firm. All of the others were reasonably close to one another.



CANADA






LIBERAL CONSERVATIVE BLOC NDP GREEN OTHER

103 124 51 29 0 1 Actual
TOTAL WRONG 99 124 60 25 0 0 The Truth Hurts
18 4 0 9 4 0 1 Difference

104 118 56 29 0 1 Election Prediction
12 1 6 5 0 0 0 Difference

94 128 56 29 0 1 Democratic Space
18 9 4 5 0 0 0 Difference

64 150 58
36
0 0 IPSOS
80
39 26 7
7
0 1 Difference

93 127 54 33 0 1 UBC
20 10 3 3 4 0 0 Difference


I should point out that I have adjusted my forecast for the NDP from 27 down to 25 because of a data entry error. That actually increases the error rate for The Truth Hurts in the chart above, but it better reflects the accuracy of the model because the error was caused by a data entry mistake not a mistake in the model (SES Research had the NDP at 12% in Montreal and I mistakenly entered that as 22%). But feel free to call this mistake differently than I have if you like.