I have no particularly deep commentary on the Gerber and Malhotra study discussed by Kieran and here by Kevin Drum. Bitterly fought battles over the relative value of different political science methodologies are a phenomenon that I am happy to consign to my misspent youth. Nonetheless, some attention is appropriate.
There are a lot of studies that just barely show significant results, and there are hardly any that fall just barely short of significance. There’s a pretty obvious conclusion here, and it has nothing to do with publication bias: data is being massaged on wide scale. A lot of researchers who almost find significant results are fiddling with the data to get themselves just over the line into significance.
This is probably correct, as there are lots of ways to massage and finesse data such that p < .05. There is also tremendous incentive to do so, as a single publication in APSR or AJPS may prove critical in tenure and hiring decisions. When the survival of your career depends on whether APSR accepts your article, and when you know that APSR doesn't publish negative results, it can hardly be surprising that fudges happen. Moreover, given that the journals don't have the time to rigorously analyze the data on their own, and given that there are enough disputes about the appropriateness of certain models and particular ways of handling data that "finesse" decisions are almost always defensible on some grounds, it's not even as if scholars are "cheating" in the traditional sense of the term. Were this still 1999, I might add that this problem and others like it tend to undermine the “hard science” claims of those who vigorously argue for the superiority of quantitative over qualitative methods. It’s not 1999, though, so I’ll just let that lie. Anyway, I made up, like, six countries for my dissertation, so who am I to complain?