Volume 231, Issue 5-6

This paper deals with the demonstration in Leipzig on October 9, 1989, an important episode in the history of the East German Revolution. It is generally held that 70,000 demonstrators participated. This paper shows that this number is clearly wrong. The paper describes briefly the results of a survey that were inconsistent with this number and how the authors of the study proceeded to make a new estimate. The paper further outlines how the original estimate was made and found its way into the media and historical accounts. Finally, some general lessons are drawn from the case. The case study this paper focuses on is not an example of the faking of data, but rather of negligent data handling. However, it is argued that the lessons from this case discussed in the final section hold for faked data as well.

For decades, there has been a heated debate about whether or not nuclear power plants contribute to childhood cancer in their respective neighbourhoods, with statisticians testifying on both sides. The present paper points to some flaws in the pro-arguments, taking a recent study prepared for the political party “Bündnis 90 /Grüne” as a specimen. Typical mistakes include an understatement of the size of tests of significance, disregard of important covariates and extreme reliance on very few selected data points.

Krämer and Arminger in a preceding article in this volume insinuated that in a meta-analysis on childhood leukaemia in the vicinity of nuclear power plants (NPP) in five countries gross methodological errors had led to falsified statistics. Major assumptions were a) arbitrary exclusion of publications with nil results, and b) publication bias in conduct of the meta-analysis. It is demonstrated that all appropriate publications providing data on incident cases of leukaemia and on the underlying population or rates of incidence with confidence intervals had been included. In addition it is demonstrated that all publications excluded from the meta-analysis either did not provide sufficient data on NPPs or cases of these publications had been already included into the meta-analysis from other publications.

In a provocative article Ioannidis (2005) argues that, in disciplines employing statistical tests of significance, professional journals report more wrong than true significant results. This short note sketches the argument and explores under what conditions the assertion holds. The “positive predictive value” (PPV) is lower than 1/2 if the a priori probability of the truth of a hypothesis is low. However, computation of the PPV includes only significant results. If both significant and non-significant results are taken into account the “total error ratio” (TER) will not exceed 1/2 provided no extremely large publication bias is present. Moreover, it is shown that theory-driven research may reduce the proportion of errors. Also, the role of replications is emphasized; replication studies of original research are so important because they drastically decrease the error ratio.

Significance tests were originally developed to enable more objective evaluations of research results. Yet the strong orientation towards statistical significance encourages biased results, a phenomenon termed “publication bias”. Publication bias occurs whenever the likelihood or time-lag of publication, or the prominence, language, impact factor of journal space or the citation rate of studies depend on the direction and significance of research findings. Although there is much evidence concerning the existence of publication bias in all scientific disciplines and although its detrimental consequences for the progress of the sciences have been known for a long time, all attempts to eliminate the bias have failed. The present article reviews the history and logic of significance testing, the state of research on publication bias, and existing practical recommendations. After demonstrating that more systematical research on the risk factors of publication bias is needed, the paper suggests two new directions for publication bias research. First, a more comprehensive theoretical model based on theories of rational choice and economics as well as on the sociology of science is sketched out. Publication bias is recognized as the outcome of a social dilemma that cannot be overcome by moral pleas alone. Second, detection methods for publication bias going beyond meta-analysis, ones that are more suitable for testing causal hypotheses, are discussed. In particular, the “caliper test” seems well-suited for conducting theoretically motivated comparisons across heterogeneous research fields like sociology. Its potential is demonstrated by testing hypotheses on (a) the relevance of explicitly vs. implicitly stated research propositions and on (b) the relevance of the number of authors on incidence rates of publication bias in 50 papers published in leading German sociology journals.

Systematic research reviews have become essential in all empirical sciences. However, the validity of research syntheses is threatened by the fact that not all studies on a given topic can be summarized. Research reviews may suffer from missing data, and this is especially crucial in those cases where the selectivity of studies and their findings affects the summarized result. So-called publication bias is a type of missing data and a phenomenon that jeopardizes the validity of systematic or quantitative, as well as narrative, reviews. Publication bias exists if the preparation, submission or publication of research findings depend on characteristics of just these research results, e. g. their direction or statistical significance. This article describes methods to identify publication bias in the context of meta-analysis. It also reviews empirical studies on the prevalence of publication bias, especially in the social and economic sciences, where publication bias also seems to be prevalent. Several proposals to prevent publication bias are discussed.

This paper focuses on fraud detection in surveys using Socio-Economic Panel (SOEP) data as an example for testing newly methods proposed here. A statistical theorem referred to as Benford’s Law states that in many sets of numerical data, the significant digits are not uniformly distributed, as one might expect, but adhere to a certain logarithmic probability function. In order to detect fraud, we derive several requirements that should, according to this law, be fulfilled in the case of survey data.We show that in several SOEP subsamples, Benford’s Law holds for the available continuous data. For this analysis, we developed a measure that reflects the plausibility of the digit distribution in interviewer clusters. We are thus able to demonstrate that several interviews that were known to have been fabricated and therefore deleted in the original user data set can now be detected using this method. Furthermore, in one subsample, we use this method to identify a case of an interviewer falsifying ten interviews not previously detected by the fieldwork organization.

Detecting election fraud with a simple statistical method and minimal information makes the application of Benford’s Law quite promising for a wide range of researchers. Whilst its specific form, the Second-Digit Benford’s Law (2BL)-test, is increasingly applied to fraud suspected elections, concerns about the validity of its test results have been raised. One important caveat of this kind of research is that the 2BL-test has been applied mostly to fraud suspected elections. Therefore, this article will apply the test to the 2009 German Federal Parliamentary Election against which no serious allegation of fraud has been raised. Surprisingly, the test results indicate that there should be electoral fraud in a number of constituencies. These counter intuitive results might be due to the naive application of the 2BL-test which is based on the conventional χ 2 distribution. If we use an alternative distribution based on simulated election data, the 2BL-test indicates no significant deviation. Using the simulated election data, we also identified under which circumstances the naive application of the 2BL-test is inappropriate. Accordingly, constituencies with homogeneous precincts and a specific range of vote counts tend to have a higher value for the 2BL statistic.

The occurrence of scientific fraud damages the credibility of science. An instrument to discover deceit was proposed with Benford’s law, a distribution which describes the probability of significant digits in many empirical observations. If Benford-distributed digits are expected and empirical observations deviate from this law, the difference yields evidence for fraud. This article analyses the practicability and capability of the digit distribution to investigate scientific counterfeit. In our context, capability means that little data is required to discover forgery. Furthermore, we present a Benford-based method which is more effective in detecting deceit and can also be extended to several other fields of digit analysis. We also restrict this article to the research area of non-standardized regressions. The results reproduce and extend the finding that non-standardized regression coefficients follow Benford’s law. Moreover, the data show that investigating regressions from different subjects demands more observations and hence is less effective than investigating regressions from single persons. Consequently, the digit distribution can discover indications for fraud, but only if the percentage of forgery in the data is large. With a decreasing proportion of fabricated values, the number of required cases to detect a significant difference between real and fraudulent regressions rises. Under the condition that only few scientists forge results, the investigation method becomes ineffective and inapplicable.

This article evaluates three different questioning techniques for measuring the prevalence of plagiarism in student papers: the randomized response technique (RRT), the item count technique (ICT), and the crosswise model (CM). In three independent experimental surveys with Swiss and German university students as subjects (two web surveys and a survey using paper and- pencil questionnaires in a classroom setting), each of the three techniques is compared to direct questioning and evaluated based on the “more-is-better” assumption. According to our results the RRT and the ICT failed to reduce social desirability bias in self-reports of plagiarism. In contrast, the CM was more successful in eliciting a significantly higher rate of reported sensitive behavior than direct questioning. One reason for the success of the CM, we believe, is that it overcomes the “self-protective no” bias known from the RRT (and which may also be a potential problem in the ICT).We find rates of up to 22 percent of students who declared that they ever intentionally adopted a passage from someone else’s work without citing it. Severe plagiarism such as handing in someone else’s paper as one’s own, however, seems to be less frequent with rates of about 1 to 2 percent.

Acquiescence can be the source of a serious response bias in international comparative research. We demonstrate this by referring to an example taken from environmental sociology. The effect of wealth on individuals’ willingness to pay for environmental protection is controversially discussed in the literature. Studies analyzing the International Social Survey Programme (ISSP) report that individuals in wealthier nations are more concerned about the environment, while studies using the World Values Survey (WVS) or the European Values Study (EVS) come up with the opposite finding. The puzzle is resolved when the different levels of acquiescence are taken into consideration. As it turns out, respondents in poorer nations in Asia and Eastern Europe have higher levels of acquiescence than respondents in richer Western nations. Thus, acquiescence conceals the wealth effect of studies analyzing the WVS or EVS and the issue is resolved when acquiescence is properly controlled for in multivariate statistical models.