This week, we were asked to evaluate the footnote, which stated that due to the fact that the research was explanatory, the level of significance was relaxed to 0.1. If I were to give a response to the authors of the article as a reader, I would draw their attention to several points. First, I would say that increasing tolerance to the possibility of Type 1 error in this situation is acceptable. The researchers conducted an explanatory study, which meant that they studied a phenomenon that had not yet been studied before. This implies that there was little data available to have high-precision estimates. Warner (2012) states that researchers sometimes prefer to use significance levels of 0.1 in explanatory studies. However, there is another matter that should be considered about the conclusions drawn from the significance level.
When responding to the authors of the article, I would point out that it would be incorrect to draw conclusions about the meaningfulness of findings based on the p-value alone. American Statistical Association (ASA, 2016) warns researchers that statistical significance should not be used as the only measure of the meaningfulness of findings. P-values cannot measure how incompatible data are with the statistical model, nor can they measure the probability that the hypothesis is true (ASA, 2016). Additionally, meaningfulness is different from statistical significance because the findings may not be applicable to the real world due to a small effect size (Laureate Education, 2016). Thus, while the choice of the alpha level is acceptable, the conclusion drawn from the p-value may be inadequate.
References
American Statistical Association. (2016). American statistical association releases statement on statistical significance and p-values. Web.
Laureate Education. (2016). Meaningfulness vs statistical significance [Video file]. Baltimore MD: Author.
Warner, R.M. (2012). Applied statistics from bivariate through multivariate techniques (2nd ed.). Sage Publications.