How to Get an A* in A-Level Maths

Washington Post have posted statistics which "leave very little room for reasonable doubt" that the Iranian election was rigged. In summary, they look at the final two digits of the vote counts and compare them with what you would expect if they were random.

The statistics applied to the problem are admirable and (as far as I can make out) follow the hypothesis test methods which can be found in the A-level Statistics modules - the article is very much worth a read. The results show significant results at 5% level, suggesting that the number of votes in the Iranian election may have been made up. I'd like to get the opportunity to check the numbers in more detail, though I have no reason to disbelieve what they've written.

However - this is not the end of the story...

Considering the weight of feeling amongst Western people that the Iranian elections are rigged, people are prepared to seek out any evidence anything which agrees with that conclusion. This is a human bias known as confirmation bias - the act of searching for information that agrees with what you believe - a very dangerous source of bias. Generally we must make a serious effort to be as neutral as possible.

So let's concede that political scientists Beber and Scacco were genuine in their investigation - that they honestly set up the test with no preconceptions, were not looking for evidence unduly, and were prepared to accept the result whichever way it landed - with no confirmation bias.
Even so, we might imagine that perhaps hundreds of similar scientiists have recently done hundreds of similar tests. For every hundred similar tests, at a 5% significance level, around 5 tests will generate a significant result incorrectly - even if the election was perfectly fair, 5% of the time we will conclude that the election was not fair.

We don't know exactly how many of these tests have been performed, that's true - but under these circumstances, we would easily be able to argue that 5% is not a strong enough significance level. If we were to make the significance level any stricter - say 1% - then we'd have to conclude that there's not enough evidence to say the election was rigged. (in some sciences, a significance level of 0.000001% is not unusual)
This is often a consideration with medical statistics and drug trials in particular. It's important that we don't allow companies to repeat hypothesis tests until they get the result that they want or do multiple parallel tests without admitting that and correcting for it.

Then we have another source of bias I'm afraid! The Washington Post are very unlikely to publish a result that said "Iranian elections might be fair" - and are much more likely to publish "Iranian elections might be rigged" (as they have here). So there's a bias which comes from the way news is reported.

I would add a final point - a hypothesis test is always better if the conditions are prepared before the results are released. If it were an international standard that elections were checked by this method, then a 5% significance level might be very reasonable. On the other hand, if that were the international standard and everyone knew it, it would be very easy to cheat the system. Oh dear!

Conclusion: Always take statistics with a pinch of salt! In trials involving humans, we must be very careful to eliminate as much bias as possible - but often it will be impossible! In the real world, we'll have to apply statistics with consideration for the situation.

In terms of the Iranian elections - I can't say whether the elections are rigged or not, but these particular statistics are not really conclusive and can't fairly be described as "strong evidence".

Hope this wasn't too ranty! Happy statistics!