In the modern world, with its massive amount of information, it is more important than ever to rely on solid data. The most credible source of reliable information is scientific articles published in peer-reviewed publications. However, even the best journals, such as Science or Nature, can publish articles with erroneous data. To correctly assess the article’s credibility, a reader can look for answers to three main questions, which will indicate the reliability of the information and the correctness of the conclusions made by the researchers.
The first question to be asked is how correct the experiment setting was. Was there a control group, and how did it differ from the test group? In medical research on humans, as a rule, placebo-controlled trials should be conducted. That is, the subjects should not know whether they are in the control group or the study group.
The second question is how statistically reliable the data obtained is. The statistical reliability of information is influenced by factors such as sample size and homogeneity, as well as the presence of correct mathematical data processing. Is a normal or even distribution of results expected? How likely is it that the results are not random but statistically significant? Usually, the probability that the results could come out randomly (so-called p) should not exceed 0.05 or 5% (LeBel et al., 2018).
The third question to be addressed is whether the researchers are somehow interested in the results and how they could inadvertently influence them. Some research may be directly funded by companies or lobbyists, making scientists interested in a particular result (LeBel et al., 2018). In such cases, the study should be “blind” so that the researcher has minimal influence on the result.
Reference
LeBel, E. P., McCarthy, R. J., Earp, B. D., Elson, M., & Vanpaemel, W. (2018). A unified framework to quantify the credibility of scientific findings. Advances in Methods and Practices in Psychological Science, 1(3), 389-402.