Band 15, Heft 1 - Empirical Research in Accounting and Social Sciences: Elephants in the Room

Attendees of accounting empirical research seminars all too often come to view the conclusions presented in the papers as non-persuasive. This disappointing situation indicates that researchers employ data analysis methodologies which inherently support conclusions they are looking for. Such issues are rarely discussed because many participants have relied on the same methodologies – thus they have firsthand knowledge about the inherent deficiencies. The mantra becomes: “We are all aware of uncomfortable aspects of the methodologies used in our research, so why dwell on it?” Because these potential questions tend to be outside normal and acceptable bounds, I term them “elephants in the room”. Five such cases are delineated to illustrate incontrovertible problems therein. To sum it up, the elephants highlight that the purported substantive contents of most published papers will be taken with a grain of salt for the foreseeable future.

Many empirical studies in social sciences including accounting, economics and finance apply a mathematical model to fit data in view to infer association between variables, or predict further serial values. Restricted by normal distributions and linear regression analysis, many studies neglect to address (i) the conceptual frame of reference and analysis overarching scientific endeavour (design); and (ii) the relationship between data and the phenomenon under investigation (morphology). This note discusses some consequences of this neglect of design and morphology, by pointing to accounting systems that stand behind data, and the conceptual framework which is needed to back and ground scientific research.

This essay advances the possibly startling notion that as a matter of largely conceptual understanding, the vast majority of accounting research’s empirical efforts concern tests of null hypotheses that are, a priori , false. Figuratively speaking, our journals specialize in providing compelling evidence that ‘the earth is not flat’. It further presents thoughts on the causes and consequences of this state of affairs. Moreover, while other fields also suffer from this ‘flat earth’ condition, it argues that as a field that specializes in issues surrounding the conveyance of useful information, the accounting academy should step up and take leadership on dealing with the problem. The essay closes by discussing how the field might venture forth from this sheltered approach to empirical inference and thereby enhance the discovery content of its empirical inquiries.

The problems with p -values have been extensively discussed recently, but there is little work about the broader aspects of scientific inference of which p -values are but one part. This article explains how scientific inference can be characterized as Bayesian inference to the best explanation, which involves developing and assessing theories based on their fit with background facts and their ability to explain the observed data better than competing theories can. These factors translate into prior odds and Bayes factor respectively, which determine posterior odds under Bayesian inference. I provide examples from accounting research to illustrate how attention to these points makes for better research designs and stronger justification for inferences.

Ohlson (2025. Empirical accounting seminars: Elephants in the room. Accounting, Economics, and Law: A Convivium 15 , 1–8) draws on his experience in empirical accounting seminars to identify five “elephants in the room”. I interpret each of these elephants as either a variant or a symptom of p-hacking. I provide evidence of the prevalence of p-hacking in accounting research that complements the observations made by Ohlson (2025. Empirical accounting seminars: Elephants in the room. Accounting, Economics, and Law: A Convivium 15 , 1–8). In this paper, I identify a number of steps that could be taken to reduce p-hacking in accounting research. I conjecture that facilitating and encouraging replication alone could have profound effects on the quality and quantity of empirical accounting research.

Persistent and difficult methodological issues, such as those highlighted in Ohlson (2025. Empirical accounting seminars: Elephants in the room. Accounting, Economics, and Law: A Convivium 15, 1–8), undermine confidence in the results of empirical studies. Many of these issues relate to the efforts of researchers to defend the statistical significance of their results. However, the seemingly endless combination of methodologies that can be employed make statistical significance a somewhat illusory concept. A more productive approach would be to place greater emphasis on the economic significance of empirical results.

Empirical research is ripe for a reality check, as elegantly put by the “elephants in the room” (Ohlson, 2025. Empirical accounting seminars: Elephants in the room. Accounting, Economics, and Law: Convivium 15 , 1–8) referring to practices to disguise false positives with the aid of statistical engineering. However, the diagnosis points to a deeper problem. The dominant empirical paradigm combines extraordinarily vague hypotheses with ridiculously high desired levels of statistical confidence beatable solely with econometric hacks. Instead, I argue that economic magnitudes measure meaningful theoretical constructs and require far less than conventional significance levels for measurements of sufficient importance. Precisely estimating that an effect is close to zero can be more meaningful than a noisy but significant coefficient. I make several actionable proposals: (1) report standard errors rather than conventional statistical significance (stars) or t-stats, (2) discuss target significance levels likely to change priors and could much higher than weak significance for unsettled questions, (3) report precisely estimated zeros and power analyses, and (4) anchor empirical design on formal theory justified with precise references or structural models.

Ohlson (2025. Empirical accounting seminars: Elephants in the room. Accounting, Economics, and Law: A Convivium 15, 1–8) laments that the evidentiary quality of empirical accounting research is low due to inappropriate methods and practices, leaving seminar attendees and readers unpersuaded by presented or published articles. He suggests that the norms of the profession prevent a public recognition and discussion of those issues, thereby sustaining the poor state of empirical accounting research. I agree that some current empirical approaches and norms seem to hamper progress toward more convincing research. I provide a practical suggestion to possibly improve the state of empirical accounting research. I caution though that even with better methods and more honest research practices, we should not expect that any individual research article can provide conclusive answers to important accounting questions. Such progress in knowledge requires a body of high-quality and independent research.

Ohlson (Empirical Accounting Seminars: Elephants in the Room. Accounting, Economics, and Law: A Convivium 15 (1): 1–8) argues that researchers tacitly avoid raising statistics-related ‘elephants’ that could undermine inferences. We offer a balanced perspective, first applauding the remarkable progress made in deriving testable predictions, leveraging modern statistical techniques, and tapping alternative Big Data sources to address issues relevant to practitioners, regulators and academia. While we concur with Ohlson’s elephants, we caution against over-criticism based on statistical design choices, as it risks creating new elephants. Our key lessons: focus on meaningful hypotheses, recognize merits of descriptive studies, balance Type I and II errors in data handling and journal reviewing, employ proper context when interpreting statistical significance and consider economic significance. Overall, though empirical accounting research faces challenges, criticism should not deter innovative research (Type II error in journal reviewing).